A novel evaluation function for feature selection based upon information theory

Feature selection methods play a significance role during classification of data having high dimensions of features. The feature selection methods select most relevant subset of features that describe data appropriately. Mutual Information (MI) based upon information theory is one of metric used for measuring relevance of features. This paper analyzes various feature selection methods based upon MI for (1) Different evaluation function; (2) Consideration of redundancy relevance and class conditional interaction information for measuring net relevance of features. Various research gaps identified are: (1) Computation of MI from the whole sample space instead of unclassified sample subspace. (2) Consideration of relevance of features only or tradeoff between relevance & redundancy, but class conditional interaction of features is ignored. In this paper, we propose a novel generalized evaluation function using MI for feature selection. The proposed evaluation function measures the net relevance of candidate feature as linear combination of relevance, redundancy and class conditional interaction information. The proposed evaluation function is based on the principle of maximal relevance, minimal redundancy and maximal interaction information of features.