SVM-Induced Dimensionality Reduction and Classification

Support Vector Machines (SVM) has drawn extensive interests due to its attractive properties, based on which some dimensionality reduction methods have been proposed. However, SVM here only serves as a feature extractor rather than a classifier, the extracted features are in turn used as inputs to other different classifiers. In this paper, a novel and simpler SVM-induced Dimensionality Reduction and Classification framework (SVMDRC) is developed, in which the dimensionality reduction and classification can uniformly be implemented. Specifically, SVMDRC linearly embeds a projection matrix into the classical SVM. By its dual formulation, the projection matrix and the dual variables of SVM are alternatively optimized such that the optimization process makes SVM not only generate a low-dimensional orthogonal subspace spanned by the columns of the matrix but also complete classification for the projected data. It is worth to emphasize that the projection matrix can easily be analytically solved by an eigen-equation, which is as tractable as PCA. Experiments of evaluating the presented method are performed on the benchmark datasets.