A conditional independence perspective of variable selection

Variable selection is a necessary preprocessing stage in many applications, such as regression and classification, to reduce computational cost, to avoid curse of dimensionality and to improve generalization. A filter type approach to variable selection employs statistical criteria such as dependence to quantify the importance of a variable. In this paper we discuss the use of conditional independence as a criteria for variable selection, and describe a forward selection and a backward elimination based approach using this notion. We introduce two measures of conditional independence, describe their respective estimators and apply them in the variable selection task. We also provide a brief overview of the available variable selection methods and compare the proposed methods with these methods.