Analyzing the balancing of error rates for multi-group classification

This paper reports the relative performance of an experimental comparison of some well-known classification techniques such as classical statistical, artificial intelligence, mathematical programming (MP), and hybrid approaches. In particular, we examine the four-group, three-variable problem and the associated error rates for the four groups when each of the models is applied to various sets of simulated data. The data had varying characteristics such as multicollinearity, nonlinearity, sample proportions, etc. We concentrate on individual error rates for the four groups, i.e., we count the number of group 1 values classified into group 2, group 3, and group 4 and vice versa. The results indicate that in general not only are MP, k-NN, and hybrid approaches relatively better at overall classification but they also provide a much better balance between error rates for the top customer groups. The results also indicate that the MP and hybrid approaches provide relatively higher and stable classification accuracy under all the data characteristics.