Informational classifier fusion

Classifier combination has proven itself a powerful tool for achieving high recognition rates with otherwise moderately discriminating classifiers. While progress has been made during the last decade in terms of generating powerful classifier ensembles, the actual combination process is not understood yet. In this paper, the author present an information-theoretical solution to classifier combination that integrates the information conveyed by each classifier. The proposed method transforms the likelihood values of a classifier in such a way that they equal the information conveyed, without affecting its individual performance. This implicitly postulates that the elementary sum-rule performs at least as good as any other, more complex combination scheme. The author evaluated his method by combining on-line and off-line Japanese character recognizers, computing a considerable improvement of more than 4.5% compared to the best single recognition rate.