Combining Classifiers in a Tree Structure

This paper presents a method for combining classifiers in a tree structure, where each node of the tree contains single hypothesis trained in respective region of the feature space. All base classifiers are then combined using weighted average. Majority vote and Newton-Raphson numerical optimization are used for fitting the coefficients in the additive model. Two loss functions (quadratic and boosting-like exponential) as well as new splitting criteria for inducing the tree are examined within proposed framework. The idea of combining classifiers in a tree structure is then compared with other iteratively built classifiers: Adaboost.MH and MART (multiple additive regression trees). The experiments were conducted with the usage of well-known databases from the UCI Repository and the ELENA project.