Local PCA learning with resolution-dependent mixtures of Gaussians

A globally linear model, implied by conventional principal component analysis (PCA), may be insufficient to represent multivariate data in many situations. An important question is then how to find an appropriate partitioning of the data space together with a proper choice of the local numbers of principal components (PCs). We address both problems within a density estimation framework and propose a probabilistic approach which is based on a mixture of subspace-constrained Gaussians. Thereby the number of local PCs depends on a global resolution parameter, which represents the assumed noise level and determines the degree of smoothing imposed by the model. As a result, the model leads to an automatic resolution-dependent adjustment of the optimal principal subspace dimensionalities, which may vary among the different mixture components. Furthermore it provides the optimization with an annealing scheme, which solves the initialization problem and offers an incremental model refinement procedure