Robust Relief-Feature Weighting, Margin Maximization, and Fuzzy Optimization

A latest advance in Relief-feature-weighting techniques is that the iterative procedure of Relief can be approximately expressed as a margin maximization problem, and therefore, its distinctive properties can be investigated with the help of optimization theory. Being motivated by this advance, the Relief-feature-weighting algorithm is investigated for the first time within a fuzzy-optimization framework. A new margin-based objective function that incorporates three fuzzy concepts, namely, fuzzy-difference measure, fuzzy-feature weighting, and fuzzy-instance force coefficient, is introduced. By the application of fuzzy optimization to this new margin-based objective function, several useful theoretical results are derived, based upon which, a set of robust Relief-feature-weighting algorithms are proposed for two-class data, multiclass data, and, then, online data. As demonstrated by extensive experiments in synthetic datasets, the University of California at Irvine (UCI)-benchmark datasets, cancer-gene-expression datasets, and face-image datasets, the proposed algorithms were found to be competitive with the state-of-the-art algorithms and robust for datasets with noise and/or outliers.