Evaluation of canonical correlation analysis: A Correlation Generation Model

Canonical Correlation Analysis (CCA) is a powerful technique for finding the correlations between two sets of multidimensional variables. Due to its performance in practice, many extensions were brought forward such as least square CCA. However, there is no such a unified solution to compare their performance, i.e. in the sense of extracting canonical correlations. In this paper, we propose a framework to systematically evaluate performance of CCA and its variants. Firstly, a Correlation Generation Model (CGM) is proposed to analyze CCA in three aspects: 1) Why are the multidimensional variables correlated? 2) How are they correlated? 3) How to evaluate this correlation? Based on CGM, it is possible to qualitatively study CCA in terms of accuracy and robustness. Most interestingly, the analysis reveals that CCA actually suffers from the Under Sample Problem (USP), which is often discussed in the machine learning field but ignored in the literature. Finally, experiments based on CGM are performed to evaluate the CCA as well as its variants.