Construct Virtual Samples for Improving Kernel PCA

Though kernel methods have been widely used for feature extraction, it suffers from the problem that its feature extraction efficiency is in inverse proportion to the size of the training sample set. In order to make kernel-methods-based feature extraction computationally more efficient, we propose a novel improvement to the kernel method. This improvement assumes that the discriminant vector in the feature space can be approximately expressed by a certain linear combination of some constructed virtual sample vectors. We determine these virtual sample vectors one by one by using a very simple and computationally efficient iterative algorithm. The algorithm is simple, robust and competitive. When we determine virtual sample vectors, we need only to set the initial values of the virtual sample vectors to random values. The experiments show that our method can achieve the goal of efficient feature extraction as well as a good and stable classification accuracy.