Cluster Preserving Embedding

Most of existing dimensionality reduction methods obtain the low-dimensional embedding via preserving a certain property of the data, such as locality, neighborhood relationship. However, the intrinsic cluster structure of data, which plays a key role in analyzing and utilizing the data, has been ignored by the state-of-the-art dimensionality reduction methods. Hence, in this paper we propose a novel dimensionality reduction method called Cluster Preserving Embedding(CPE), in which the cluster structure of original data is preserved via preserving the robust path-based similarity between pairwise points. We present two different methods to preserve this similarity. One is the Multidimensional Scaling(MDS) way, which tries to preserve similarity matrix accurately, the other one is a Laplacian-style way, which preserves the topological partial order of the similarity rather than similarity itself. Encouraging experimental results on a toy data set and handwritten digits from MNIST database demonstrate the effectiveness of our Cluster Preserving Embedding method.