Efficient parameter selection for support vector machines in classification and regression via model-based global optimization

Support vector machines (SVMs) have become one of the most popular methods in machine learning during the last years. A special strength is the use of a kernel function to introduce nonlinearity and to deal with arbitrarily structured data. Usually the kernel function depends on certain parameters, which, together with other parameters of the SVM, have to be tuned to achieve good results. However, finding good parameters can become a real computational burden as the number of parameters and the size of the dataset increases. In this paper we propose an algorithm to deal with the model selection problem, which is based on the idea of learning an online Gaussian process model of the error surface in parameter space and sampling systematically at points for which the so called expected improvement is highest. Our experiments show that on this way we can find good parameters very efficiently.