The Influence of the Pool of Candidates on the Performance of Selection and Combination Techniques in Ensembles

In this paper, we propose the use of an immune-inspired approach called opt-aiNet to generate a diverse set of high-performance candidates to compose an ensemble of neural network classifiers. Being a population-based search algorithm, the opt-aiNet is capable of maintaining diversity and finding many high-performance solutions simultaneously, which are known to be desired features when synthesizing an ensemble. Concerning the selection and combination phases, the most relevant selection and combination techniques already proposed in the literature have been considered. The main contribution of this paper is the indication that there is no pair of selection/combination technique that can be considered the best one, because the performance of the obtained ensemble varies significantly with the current composition of the pool of candidates already produced by the generation phase. Notwithstanding, this variability in performance is not restricted to the choice of opt-aiNet as the generative device. As a consequence, to overcome the performance of the best individual classifier, every possible pairs of selection and combination techniques should be tried. Only with such an exhaustive search (notice that the main computational burden is usually related to the generation phase), the performance of the ensemble was invariably superior to the performance of the best individual classifier on four benchmark classification problems.