A Novel Algorithm Based on Conditional Entropy Established by Clustering for Feature Selection

Feature selection is an important issue in machine learning. Rough set theory is one of the important methods for feature selection. In rough set theory, feature selection has already been separately studied in algebra view and information view. Unfortunately, the previously proposed methods based on information entropy for feature selection only focus on the discrete datasets. However, how to effectively discretize the continuous datasets is also full of challenge, since this method may lead to loss of some useful information. To overcome this disadvantage, in this paper, we introduce a novel algorithm based on conditional entropy by clustering strategy for feature selection (ACECFS). In ACECFS, the projected data corresponding to each feature is appropriately separated into several clusters at first, and then the conditional entropy for a set of features is conveniently computed by the clusters and corresponding feature list is generated, hence an effectively relevant and compact feature subset can be obtained from the ranked feature list. Experiments show the effectiveness of ACECFS.