Discriminant feature extraction for parametric and non-parametric classifier

Feature extraction (FE) is considered as preserving the value of the discriminant function for a given classifier which uses a posteriori probabilities P(ω_i|X) while reducing dimensionality. For classification minimizing Bayes´ error, a posteriori probabilities would be the best features. In this feature space, the probability of error is the same as in the original space, assuming Bayes´ classifier. The authors consider FE as eliminating features which have no impact on the value of the discriminant function and propose an FE algorithm which eliminates those irrelevant features and retains only useful features. The proposed algorithm does not deteriorate even when there is no difference in the mean vectors or covariance matrices, and it can be used for both parametric and nonparametric classifiers