Dimensionality Reduction with Adaptive Kernels

A kernel determines the inductive bias of a learning algorithm on a specific data set, and it is beneficial to design specific kernel for a given data set. In this work, we propose a kind of new kernel, called locality-adaptive-kernel (LAKE), which adoptively measures the data similarity by considering the geometrical structure of the data set. In theory, we prove that the LAKE is a special marginalized kernel; and intuitively, when the local kernel in LAKE is constrained to be linear, it has the explicit semantic of merging multiple local linear analyzers into a single global nonlinear one. We show in a toy problem that the kernel principal component analysis with LAKE well captures the intrinsic nonlinear principal curve of the data set. Moreover, a large set of experiments are presented to verify that the classification performance is sensitive to the kernel variation; and the extensive face recognition experiments on different databases demonstrate that KPCA and KDA based on LAKE are both superior to those based on traditional fixed kernels