Classifier Design via Projection Approximation

In the viewpoint of inverse problem and function approximation theory, we set up a framework for pattern classifier design, wherein the classifier is assumed to be an element of a Reproducing Kernel Hilbert Space (RKHS) continuously defined on the pattern feature space. Based on the RKHS metrics, an orthogonal projection criterion is adopted for pattern feature representation so that optimal generalization capability is ensured for the classifier, with respect to minimum error norm. In addition, by orthogonally projecting the outputs of the target class samples onto the null space of an operator defined by the sample outputs of other classes, the target class is optimally discriminated from other classes, with respect to minimum mean output energy. Combination of the above two criteria yields a criterion for optimal representation and discrimination of pattern features, which yields another classifier wherein the balance between representation and discrimination may be controlled by a parameter. Some experimental results on handwritten digit classification, face recognition, and radar target recognition are given to show the feasibility of the presented classifiers.