A new ensemble classifier creation method by creating new training set for each base classifier

Base classifier´s classification error and diversity are key factors in performance of ensemble methods. There is usually a trade-off between classification error and diversity in ensemble methods. Decreasing classification error of base classifiers usually makes them less diverse while increasing diversity, results in less accurate base classifiers. This paper proposes a new ensemble classifier generation method which aims to create more diverse base classifiers while making them more accurate. In this approach, training data for base classifiers are built by taking a bootstrap sample of the original training set and then manipulating a set of arbitrary attributes of each pattern. We experimented our ensemble of classifiers on 15 UCI data sets and were able to outperform Bagging, Boosting and Rotation Forest. Moreover, Wilcoxon signed rank test confirms our claim and shows that the proposed method is significantly better than other three methods on these data sets.