Cost-Free Learning for Support Vector Machines with a Reject Option

In this work, we investigate into the abstaining classification of binary support vector machines (SVMs) based on mutual information (MI). We obtain the reject rule by maximizing the MI between the true labels and the predicted labels, which is a post-processing method. The gradient and Hessian matrix of MI are derived explicitly so that Newton method is used for the optimization which converges very fast. Different from the existing reject rules of SVM, the present MI-based reject rule does not require any explicit cost information and is under the framework of cost-free learning. As a matter of fact, the cost information embedded in MI can also be derived from the method, which provides an objective or initial reference to users if they want to apply cost-sensitive learning. Numerical results confirm the benefits of the proposed MI-based reject rule in comparison with other reject rules of SVM.