Towards the Optimal Discriminant Subspace

Dimensionality reduction is a common practice in many learning and Intelligence applications. However, most existing methods use the dimension of the target subspace as a parameter, making it hard to decide which subspace is optimal for classification. In this paper, we address the challenge of learning the optimal subspace for the nearest neighbor classification. We focus on labeled data and assume that the data for each class lie on respective sub-manifolds. To separate each sub-manifold, the labels of the data are used to learn the subspace where neighboring points of the same class keep close and those of different classes are disassociated. The sub-manifold separating method is first proposed as linear projection. For more complicated nonlinear situation, we generalize the algorithm using the kernel method. A group of experiments on data representation and classification are performed to evaluate he effectiveness of the proposed approaches.