Combining different ways to generate diversity in bagging models: An evolutionary approach

Bagging algorithm has been proven to be effective when dealing with on different classification problems. However, the success of Bagging depends strongly on the diversity level reached by the individual classifiers of the ensemble models. Diversity in ensemble can be obtained when the individual classifiers are built using different circumstances, such as parameter settings, training datasets and learning algorithms. This paper presents a new approach which combines these three different ways to obtain high diversity in Bagging models, aiming, as a consequence, to obtain high levels of accuracy for the ensembles. In the proposed approach, in order to obtain the optimal configurations of features and classifiers in Bagging models, we have applied an evolutionary approach composed of two genetic algorithm instances. In order to validate the proposed approach, experiments involving 10 classification algorithms have been conducted, applying the resulting Bagging structures in 5 pattern classification datasets taken from the UCI repository. In addition, we analyze the performance of the resulting Bagging structures in terms of two recently proposed diversity measures, referred to as good and bad.