Genetic Algorithm Based Selective Ensemble with Multiset Representation

Recently, it has been shown that, in ensemble learning, it may be preferable to ensemble some instead of all the classifiers. Various selective ensemble approaches are then designed, where optimization algorithms like genetic algorithm (GA) are used to evolve weights of component classifiers and classifiers with weights greater than a threshold are selected. This paper proposes a novel selective ensemble algorithm which treats each ensemble as a multiset defined over the universe of all the trained classifiers and directly optimizes the ensemble set. Firstly, a classifiers pool U is trained, and a candidate multiset ensemble d is initialized to U. Then GA is employed to evolve the candidate ensemble d. The underlying set of the final optimal ensemble contains the component classifiers that GA has selected and the multiplicities of the classifiers form different "confidence" levels in correct prediction. More trust can then be put on classifiers with higher confidence levels. Experimental results show that the proposed approach achieves much preferable performance to several state-of-the-art selective and non-selective ensemble algorithms while generating ensembles with far smaller size.