A Dilemma in Assessing Stability of Feature Selection Algorithms

In realm, feature selection is an effective means for handling high-dimensional data that becomes increasingly abundant. The stability of a feature selection algorithm is becoming crucial for determining the fitness of the algorithm. Below, we review existing methods of stability assessment and analyse how they assess the stability of a feature selection algorithm. A common approach is to evaluate the similarity between the selected subsets of features produced by that algorithm over different training samples or over distributed datasets. We point out challenges facing the existing evaluation methods and suggest how to improve stability assessment of feature selection algorithms.