Extending Manifold Leaning Algorithms by Neural Networks

Manifold learning algorithms have been recently reported superior to classical dimensionality reduction techniques, such as PCA or MDS, in their ability to discover a more meaningful low-dimensional embedding of the high-dimensional samples. However, most of them encounter the problem of extension to novel samples. In this paper, we propose a regression model to extend three well-known manifold learning algorithms, i.e. Isomap, LLE, and Laplacian Eigenmap to novel samples by neural networks. We first examine these algorithms, and then show that the nonlinear dimensionality reduction ability can be acquired by neural networks, thus the extension problem is easily addressed. This model is very flexible and still preserves the nonlinear nature of the manifold leaning algorithms. Experimental results of data visualization and classification are reported, which validate the feasibility of the proposed model.