Performance Analysis of Majority Vote Combiner for Multiple Classifier Systems

Combining rules in Multiple Classifier Systems (MCS) play a central role in shaping their performance (classification error probability). Many theoretical works are developed to predict the performance using different combining rules. Some of the developed works assumed that classifiers´ outputs are independent, however in practice an ensemble of classifiers shows dependent behavior between each other. In this work, a theoretical model is derived for estimating the misclassification error probability of MCS based on majority vote combiner. In the derivation, we assumed that each classifier produces at its output an estimation of the posterior class probability that has a Gaussian distribution. In addition, we assumed that each classifier has two classes, and the outputs of classifiers are dependent and identically distributed. We validated our model using computer simulations. Results show that the ensemble performance is highly sensitive to class variance while exhibits a smoother behavior against class mean. Also, results show that as the correlation among classifiers´ outputs increases, the probability of classification error degrades exponentially. The trend continues until the performance reaches the behavior of a single classifier regardless of the number of base classifiers used in the ensemble. The proposed model provides a better understanding of the behavior of majority vote combiner in MCS.