Bayesian Nonlinear Principal Component Analysis Using Random Fields

We propose a novel model for nonlinear dimension reduction motivated by the probabilistic formulation of principal component analysis. Nonlinearity is achieved by specifying different transformation matrices at different locations of the latent space and smoothing the transformation using a Markov random field type prior. The computation is made feasible by the recent advances in sampling from von Mises-Fisher distributions. The computational properties of the algorithm are illustrated through simulations as well as an application to handwritten digits data.