On the duel between the top 10 percent classifiers versus the bottom 90 percent classifiers

Over the past two decades a number of methods have been proposed for constructing stronger classifiers as an ensemble of simpler individual classifiers. These simpler classifiers are often limited to a single family, and candidates among these are further limited to only elite classifiers, i.e., classifiers having highest performance. Many prior research papers suggest that using the top 10%-20% of elite classifiers in an ensemble should yield stronger classifiers. In this paper, we have explored the top 10% of elite classifiers as well as the bottom 90% of non-elite classifiers, proposed algorithms that can effectively use them to construct much stronger classifiers, and showed that using the bottom 90% can yield comparable and sometimes better performance than using the top 10%. Our algorithm is based on heuristic search algorithms for developing ensembles of diverse classifiers that optimize complementarity among them, and the experiments are made with 9 publicly available datasets using 22 different kinds of classifiers.