Feature-Selected Tree-Based Classification

Feature selection can decrease classifier size and improve accuracy by removing noisy and/or redundant features. However, it is possible for feature selection to yield features that are only partially informative about the classes in the set. These features are beneficial for distinguishing between some classes but not others. In these cases, it is beneficial to divide the large classification problem into a set of smaller problems, where a more specific set of features can be used to classify different classes. Dividing a problem this way is also common when the base classifier is binary, and the problem needs to be reformulated as a set of two-class problems so it can be handled by the classifier. This paper presents a method for multiclass classification that simultaneously formulates a binary tree of simpler classification subproblems and performs feature selection for the individual classifiers. The feature selected hierarchical classifier (FSHC) is tested against several well-known techniques for multiclass division. Tests are run on nine different real data sets and one artificial data set using a support vector machine (SVM) classifier. The results show that the accuracy obtained by the FSHC is comparable with other common multiclass SVM methods. Furthermore, the results demonstrate that the algorithm creates solutions with fewer classifiers, fewer features, and a shorter testing time than the other SVM multiclass extensions.