Design of minimum-error optimal classifiers for patterns from distributions with Gaussian tails

Adaptive algorithms for designing two-category linear pattern classifiers have been developed and studied in recent years. When the pattern sets are nonseparable, the adaptive algorithms do not directly minimize the number of classification errors, which is the usual goal in pattern classifier design: furthermore, they also are not minimum-error optimal, i.e., they do not generally minimize the probability of error for the classifier. However, the least-mean-square (LMS) adaptive algorithm has been shown to yield classifiers that are asymptotically minimum-error optimal for patterns from Gaussian equal-covariance distributions. A technique is also known for designing asymptotically minimum-error optimal linear classifiers for patterns from Gaussian distributions with unequal covariance matrices. This paper shows that classifiers designed with the "error-correction" algorithms have these same asymptotic properties: the error-correction algorithms are asymptotically minimum-error optimal for patterns drawn from Gaussian equal-covariance distributions and they can be used to design asymptotically minimum-error optimal linear classifiers for patterns from Gaussian distributions with unequal covariance matrices. In addition, because the error-correction algorithms use only part of the patterns in determining the classifier weights, they are asymptotically minimum-error optimal for patterns from distributions that have only Gaussian tails in the regions where their patterns are misclassified or close to misclassified, and that are almost arbitrary elsewhere.