Discriminative Canonical Correlation Analysis with Missing Samples

Multimodal recognition emerges when the non-robustness of unimodal recognition is noticed in real applications. Canonical correlation analysis (CCA) is a powerful tool of feature fusion for multimodal recognition. However, in CCA, the samples must be pairwise, and this requirement may not easily be met due to various unexpected reasons. Additionally, the class information of the samples is not fully exploited in CCA. These disadvantages restrain CCA from extracting more discriminative features for recognition. To tackle these problems, in this paper, the class information is incorporated into the framework of CCA for recognition, and a novel method for multimodal recognition, called discriminative canonical correlation analysis with missing samples (DCCAM), is proposed. DCCAM can tolerate the missing of samples and need not artificially make up the missing samples so that its computation is timesaving and space-saving. The experimental results show that 1) DCCAM outperforms the related multimodal recognition methods; and 2) the recognition accuracy of DCCAM is relatively insensitive to the number of missing samples.