On the nonlinearity of pattern classifiers

This paper presents a novel approach to the analysis of the overtraining phenomenon in pattern classifiers. A nonlinearity measure 𝒩 is introduced which relates the shape of the classification function to the generalization capability of a classifier. Experiments using the k-nearest neighbour rule, a neural classifier and the quadratic classifier show that the introduced measure 𝒩 can be used to study the overtraining behaviour of a classifier. Moreover 𝒩 shows to be a predictor for the local sensitivity of a classifier. Classifiers that have a small local sensitivity are shown to have a low nonlinearity whereas an increased nonlinearity indicates an increase in local sensitivity