A Unified Perspective on Advances of Independent Subspaces: Basic, Temporal, and Local Structures

A general framework of independent subspaces is presented, based on which a number of unsupervised learning topics have been summarized from a unified perspective, featured by different combinations of three basic ingredients. Moreover, advances on these topics are overviewed in three streams, with roadmaps sketched. One consists of studies on the second order independence featured principal component analysis (PCA) and factor analysis (FA), in adaptive and robust implementations as well as with duality and temporal extensions. The other consists of studies on the higher order independence featured independent component analysis (ICA), binary FA, and nonGaussian FA. The third is called mixture based learning that combines the above individual tasks, proportionally or competitively to fulfill a complicated task.