Orthogonal canonical correlation analysis and its application in feature fusion

Canonical correlation analysis (CCA) is an important method for multiple feature extraction and fusion. The canonical projective vectors in classical CCA method satisfy conjugated orthogonality constraints. However, the conjugated orthogonality property is badly affected by the small sample size (SSS) problem so that the projections in the classical CCA method are usually not optimal in such case for recognition purpose. Orthogonality, as a common criterion, is widely used in feature extraction. Moreover, it is less affected by poor estimation of covariance matrix. In this paper, we propose a novel canonical correlation analysis method with orthogonality constraints called Orthogonal CCA (OCCA). We replace the conjugated orthogonality constraints in classical CCA by introducing orthogonality constraints, and then a novel algorithm based on twin eigen decomposition is proposed to gradually obtain the orthogonal canonical projective vectors. Experimental results on UCI multiple feature database, and ORL face database show that OCCA has better recognition rates and robustness than previous feature fusion methods.