Target adaptation to improve the performance of least-squared classifiers

In classifier design, the squared error criterion is often used as an approximation to more relevant cost functions based on the number of classification errors, due to the relative computational ease of least-squares methods. This practice results in decision boundaries which are sub-optimal in terms of classifier accuracy, often failing to separate even linearly separable classes. We describe a method for choosing target values in such a way as to decrease the undesirable effects of the sum of squared errors criterion. The proposed technique may be used with any least-squares or penalized least-squares training method. We demonstrate its use with linear least-squares classifiers, and give a bound on the number of iterations required for the special case of linearly separable classes