An adaptive multiclass boosting algorithm for classification

A large number of practical domains, such as scene classification and object recognition, have involved more than two classes. Therefore, how to directly conduct multiclass classification is being an important problem. Although some multiclass boosting methods have been proposed to deal with the problem, the combinations of weak learners are confined to linear operation, namely weighted sum. In this paper, we present a novel large-margin loss function to directly design multiclass classifier. The resulting risk, which guarantees Bayes consistency and global optimization, is minimized by gradient descent or Newton method in a multidimensional functional space. At every iteration, the proposed boosting algorithm adds the best weak learner to the current ensemble according to the corresponding operation that can be sum or Hadamard product. This process grown in an adaptive manner can create the sum of Hadamard products of weak learners, leading to a sophisticated nonlinear combination. Extensive experiments on a number of UCI datasets show that the performance of our method consistently outperforms those of previous multiclass boosting approaches for classification.