A new way of PCA: integrated-squared-error and EM algorithms

Minimization of the reconstruction error (squared-error) leads to a principal subspace analysis (PSA) which estimates the scaled and rotated principal axes of a set of observed data. In this paper, we introduce a new alternative error, the so called integrated-squared-error, the minimization of which determines the exact principal axes (without rotational ambiguity) of a set of observed data. We consider the properties of the integrated-squared-error in the framework of a coupled generative model, giving efficient EM algorithms for integrated-squared-error minimization. We revisit the generalized Hebbian algorithm (GHA) and show that it emerges from the integrated-squared-error minimization in a single-layer linear feedforward neural network.