On node selection for classification in correlated data sets

Consider a system which can be in a finite number of states. Given a large number of characteristics which are measured, representing the system, we are concerned with the selection of a subset of characteristics of (small) given cardinality, for which the classification of the system according to one of the states in the state set is optimal according to the Rayleigh quotient criterion. This problem is relevant in various scenarios where a few explanatory variables have to be selected from a large set of candidates, including sensor selection in sensor networks, classification in image processing, and feature selection in data mining for bioinformatics applications. We show that the optimization amounts to finding the submatrix of the features covariance matrix for which the sum of elements of the inverse is maximized, and we present bounds which relate this optimization to a similar metric based on elements of the original covariance matrix.