Title :
A comparison between criterion functions for linear classifiers, with an application to neural nets
Author :
Barnard, Etienne ; Casasent, David
Author_Institution :
Dept. of Electr. & Comput. Eng., Carnegie-Mellon Univ., Pittsburgh, PA, USA
Abstract :
The error rates of linear classifiers that utilize various criterion functions are investigated for the case of two normal distributions with different variances and a priori probabilities. It is found that the classifier based on the least mean squares (LMS) criterion often performs considerably worse than the Bayes rate. The perceptron criterion (with suitable safety margin) and the linearized sigmoid generally lead to lower error rates than the LMS criterion, with the sigmoid usually the better of the two. Also investigated are the exceptions to the general trends: only if one class is known to have much larger a priori probability or variance than the other should one expect the LMS or perceptron criteria to be slightly preferable as far as error rate is concerned. The analysis is related to the performance of the back-propagation (BP) classifier, giving some understanding of the success of BP. A neural-net classifier, the adaptive-clustering classifier, suggested by this analysis is compared with BP (modified by using a conjugate-gradient optimization technique) for two problems. It is found that BP usually takes significantly longer to train than the adaptive-clustering technique
Keywords :
error statistics; neural nets; pattern recognition; adaptive-clustering classifier; back-propagation classifier; conjugate-gradient optimization technique; criterion functions; error rates; least mean squares; linear classifiers; linearized sigmoid; neural nets; normal distributions; pattern recognition; perceptron criterion; Backpropagation; Classification algorithms; Computerized monitoring; Density functional theory; Design optimization; Least squares approximation; Neural networks; Pattern analysis; Process design; Vectors;
Journal_Title :
Systems, Man and Cybernetics, IEEE Transactions on