Optimization of the Trade-Off by Artificially Re-sampling for Ensemble Learning

Ensemble learning methods like Bagging and Boosting that train a diversity of classifiers by manipulating the training set given to a base learning algorithm and then combine their predictions in classification. And the trade-off between accuracy and diversity of component classifiers is the crucial factor in determining its generalization error. Both Bagging and Boosting approach the trade-off by bootstrap sampling or re-weighting the existing training set. Some improved methods such as Decorate does this by adding artificial training instances into the existing training set. Although they achieve a suitable trade-off to some extent, they have not focused directly on the goal of find the best trade-off between accuracy and diversity of component classifiers. This paper studies empirically the influence of the trade-off, and an artificially re-sampling method is given by cross-grouping the training set to provide the best trade-off, which felicitously groups the original training set into a number of crossed derived training sets, and a base learning algorithm will be trained on them. The experiments show that the performance of this method is better than that of Boosting and Bagging.