A Novel Hypothesis-Margin Based Method Incorporating Minimal-Redundancy Criterion for Feature Selection

Simba is a recently proposed algorithm based on hypothesis-margin for feature selection, it uses maximizing hypothesis-margin as a criterion for evaluating the effectiveness of a feature subset, in this way an effective feature subset can be efficiently obtained by employing the stochastic gradient ascent strategy. However, this algorithm still can not eliminate completely those redundant features. To overcome this drawback, in this paper, we propose a novel hypothesis-margin based method for feature selection incorporating minimal-redundant criterion (Rsimba). In Rsimba, after getting the weights of features by employing hypothesis-margin strategy, the mutual information criterion induced by clustering is introduced for removing those redundant features, in this way an effectively relevant feature subset can be efficiently obtained. Experiments show that the classification performance induced by Rsimba is better than that induced by Simba on all benchmark data sets.