Tuning SVM hyperparameters in the primal

Choosing optimal hyperparameters for Support Vector Machines (SVMs) is quite difficult but extremely essential in SVM design. This is usually done by minimizing estimates of generalization error such as the k-fold cross-validation error or the upper bound of leave-one-out (LOO) error. However, most of the approaches concentrate on the dual optimization problem of SVM. In this paper, we would like to consider the task of tuning hyperparameters in the primal. We derive a smooth validation function from the k-fold cross-validation, then tune hyperparameters by minimizing the smooth validation function using Quasi- Newton optimization technique. Experimental results not only show that our approach is much faster and provides more precise results than grid search method, but also demonstrate that tuning hyperparameters in the primal would be more efficient than in the dual due to advantages provided by the primal.