Higher-Order Regularized Kernel CCA

Kernel canonical correlation analysis (kernel CCA) is sensitive to the choice of appropriate kernels and associated parameters. To the best of our knowledge there is no general well-founded approach for choosing them. As we demonstrate with Gaussian kernels, the kernel CCA tends to show perfect correlation as the bandwidth parameter of the Gaussian kernel decreases, while it provides inappropriate features with all the data concentrated in a few points. This is caused by the ill-posed ness of the kernel CCA with the 4th order moment of canonical variates becomes large. To overcome this problem, we propose to use constraints on the 4th order moments of canonical variates in addition to the variances. Experiments on synthesized and real world datasets demonstrate that the proposed kernel CCA provides well-posed and robust solution in reasonable ranges of all the hyper parameters.