Factorization towards a classifier

In practice, nonnegative data factorization is often performed for data dimensionality reduction prior to a classification task using a classifier which is effective in low dimensional space such as nearest neighbor classifier. In this work, we propose a novel formulation to learn a multi-class classifier directly through a supervised nonnegative data factorization. This new formulation has the following properties: 1) the nonnegative data matrix is approximated as the product of a nonnegative basis matrix and a coefficient matrix where the nonnegative bases distinctively capture the common characteristics of all classes apart from that specific to individual classes; 2) a regularization term is imposed on nonnegative data factorization so that each datum can be predominantly reconstructed by the common basis vectors and its corresponding class-specific basis vectors; and 3) the coefficient vector for each datum is assumed to be transformed from a mapped kernel space, and the l₂ norm of the class-specific coefficients reveals the relative confidence of classes, which then directly leads to a multi-class classifier. We also present an iterative optimization technique for our formulation and analytically show its convergence property. Extensive experiments on face recognition, head pose estimation, and handwritten digit recognition tasks clearly demonstrate the advantages of the proposed classifier over the conventional two-step approach of nonnegative data factorization followed by a classic classifier.