Stochastic optimization for learning Non-Convex Linear Support Vector Machines

In this paper, a fast optimization algorithm was proposed to learn the Non-Convex Linear Support Vector Machines (LSVM-NC) based on stochastic optimization, in which the non-convex function, Ramp Loss, was used to suppress the influence of noisy data in the case of large-scale learning problems. As for solving the non-convex linear SVMs, the traditional methods make use of the ConCave-Convex Procedure (CCCP) based on the Sequential Minimal Optimization (SMO) algorithm from dual, which is a time-consuming process and impractical for learning large-scale problems. To tackle this, we resorted to CCCP based on Stochastic Gradient Descent (SGD) algorithm from primal, and experimental results proved that our method could reduce the training time largely and improve the generalization performance.