A Regression Algorithm Based on AdaBoost

The key to successful machine learning methods is learning quality or accuracy. Boosting has proved to be effective method for improving learning quality of a weak learning algorithm with wide applications to classifications problems. However few successful applications were reported on improving regression quality by boosting. This paper presents a new algorithm for regression based on a boosted support vector machine (SVM) method. A regression problem is first converted to a binary classification problem upon the concept of epsiv-insensitive loss. By applying the idea of AdaBoost algorithm, an optimal classification-plane ensemble is constructed with the converted classification data set. Based on this ensemble a regression estimate function is obtained with equivalence to the original regression problem. The analysis shows that for the regression data set, the number of samples with regression error exceeding epsiv decreased exponentially with the number of boosting iterations. The testing results for an actual data set show that the new algorithm is effective