A New Gradient Based Algorithm for Kernel Machine Classifier

A new greedy algorithm is introduced using basic matching pursuit based on minimization of the mean square error (MSE) criterion. Comparing with its previous counterparts i.e. support vector machine (SVM) and relevance vector machine (RVM), in our proposed approach, the kernel mean is not restricted to the training input data. However, in this paper, the kernel mean is chosen in an adaptive manner based on the so-called gradient descent algorithm. The experimental results reveal that the proposed gradient kernel construction outperforms other previous algorithms in terms of scarcity and generalization.