Forming classifier ensembles with multimodal evolutionary algorithms

Ensemble classifiers have become popular in recent years owing to their ability to produce robust predictive models that generalise well to previously unseen data. In principle, Evolutionary Algorithms (EAs) are well suited to ensemble generation since they result in a pool of trained classifiers. However, in practice they are infrequently used for this purpose. Current research trends in the EA community focus on relatively complex mechanisms for building ensembles, such as co-evolution and multi-objective optimisation. In this paper, we take a back-to-basics approach, studying whether conventional EAs, augmented with simple niching strategies, can be used to form accurate ensembles. We focus on crowding for this, considering both deterministic and probabilistic variants. We also consider the effect of different similarity measures. Our results suggest that simple niching methods can lead to accurate ensemble classifiers and that the choice of similarity measure is not a significant factor. A further study using heterogeneous classifier models within the population showed no added benefit.