Hybridizing Ensemble Classifiers with Individual Classifiers

Two extensive research areas in Machine Learning are classification and prediction. Many approaches have been focused in the induction of ensemble to increase learning accuracy of individual classifiers. Recently, new approaches, different to those that look for accurate and diverse base classifiers, are emerging. In this paper we present a system made up of two layers: in the first layer, one ensemble classifier process every example and tries to classify them; in the second layer, one individual classifier is induced using the examples that are not unanimously classified by the ensemble. In addition, the examples that reach to the second layer incorporate new information added in the ensemble. Thus, we can achieve some improvement in the accuracy level, because the second layer can do more informed classifications. In the experimental section we present some results that suggest that our proposal can actually improve the accuracy of the system.