Wake-Sleep PCA

In this paper we introduce a coupled Helmholtz machine for principal component analysis (PCA), where sub-machines are related through sharing some latent variables and associated weights. We present a wake-sleep algorithm for PCA (referred to as WS-PCA), leading both generative and recognition weights to converge to principal eigenvectors of a data covariance matrix without rotational ambiguity, in contrast to probabilistic PCA and EM-PCA. Then we also present a kernerlized variation, i.e., a wake-sleep algorithm for kernel PCA (WS-KPCA). The coupled Helmholtz machine provides a unified view of principal component analysis, including various existing algorithms as its special cases. The validity of wake-sleep PCA and KPCA algorithms are confirmed by numerical experiments.