Feature subset selection for efficient AdaBoost training

Working with a very large feature set is a challenge in the current machine learning research. In this paper, we address the feature-selection problem in the context of training AdaBoost classifiers. The AdaBoost algorithm embeds a feature selection mechanism based on training a classifier for each feature. Learning the single-feature classifiers is the most time consuming part of AdaBoost training, especially when large number of features are available. To solve this problem, we generate a working feature subset using a novel feature subset selection method based on the partial least square regression, and then train and select from this feature subset. The partial least square method is capable of selecting high-dimensional and highly redundant features. The experiments show that the proposed PLS-based feature-selection method generates sensible feature subsets for AdaBoost in a very efficient way.