Improving a Neural Network Classifier Ensemble with Multi-Task Learning

Classifier ensembles have been shown to enhance classification as the expected error rate of a set of voting classifiers is known to be less than the expected error rate (r) of one of the individual members of the set, assuming r is uniform across the set and that r < 0.5. This effect is diminished by error redundancies among the individual classifiers; thus, diversity in error patterns across items among classifiers promotes performance. Here we introduce a second classification task to neural network classifiers designed to promote diversity among classifiers while minimizing performance on the main task. The result is an improvement in ensemble performance even if there is a small decrease in individual performance.