Binary classification by minimizing the mean squared slack

The paper presents a new binary classification method based on the minimization of the slack variables energy called the Mean Squared Slack (MSS). We deliver preliminary mathematical results which support the motivation behind our approach. We show that (a) in the linearly separable case the minimum MSS is attained at a separating vector, while (b) the minimizer in the linearly non-separable case is bounded but not zero. The method is conceptually simple: it solves a linear system at each iteration and it converges, typically, within a few iterations. Its complexity is obviously related to the size of the system which, in the linear case, is equal to the input pattern dimension. The method is extended to the non-linear case using kernels. Simulations demonstrate that the method is competitive with respect to computation time, accuracy, and generalization performance compared to state of the art SVM methods.