A geometric conversion approach for boosting regression problem

Boosting is one of the most important developments in machine learning recently, AdaBoost is a prevailing boosting algorithm which receives lots of attention for its effectivity and practicality. Currently the research on boosting is dominated by classification problems. On the other hand, the extension of boosting to regression has received less investigation. In this paper, we present a boosting algorithm for regression in a geometric conversion approach. The algorithm first converts the regression problem to a binary classification one by a geometric operation. Employing confidence-rated AdaBoost on converted classification problem, a separating hyperplane ensemble could be obtained, and it could be transformed to a regression function for original regression problem. We prove that this algorithm decreases the training error exponentially fast. Experiments validate that our method is effective.