Combining Classifiers: From the Creation of Ensembles to the Decision Fusion

Multiple classifier combination methods can be considered some of the most robust and accurate learning approaches. The fields of multiple classifier systems and ensemble learning developed various procedures to train a set of learning machines and combine their outputs. Such methods have been successfully applied to a wide range of real problems, and are often, but not exclusively, used to improve the performance of unstable or weak classifiers. In this tutorial are presented the basic terminology of the field, a discussion on the effectiveness of combination algorithms, the diversity concept, methods for the creation of an ensemble of classifiers, approaches to combine the decisions of each classifier, the recent studies and also possible future directions.