Decision rule steered discriminant analysis: A paradigm of unifying dimension reduction and classification into a framework

Dimension reduction (feature extraction) and classification are two elementary tasks in pattern recognition. This paper presents a paradigm of unifying dimension reduction and classification tasks into one framework. We start with a simplest classifier, the nearest (global) mean classifier, and use its decision rule to steer the design of the global mean disciminant analysis (GMDA). GMDA is proven equivalent to the classical Fisher linear discriminant analysis (FLDA). FLDA is thus an optimal feature extractor for the nearest (global) mean classifier. We then consider the nearest local mean classifier and use its decision rule to steer the design of the local mean discriminant analysis (LMDA). LMDA matches the nearest local mean classifier optimally in theory. The proposed LMDA algorithm has two advantages over the current dimension reduction algorithms. First, it has a natural connection to classification. Second, it examines the separability of samples in the transformed space where classifiers works thereby it can achieve more desirable performance. Experiments are done on the CENPARMI handwritten numeral database and the ETH80 object category database and results confirm our idea and the effectiveness of the proposed algorithm.