Information theoretic covariance complexity and its relation to pattern recognition

The discrimination information of a set of stochastic vectors is utilized to define a measure of the information theoretic complexity of the associated covariance matrix. Bounds are developed that relate the covariance complexity to the maximum distance, or speed, between the characteristic roots of the covariance. A computationally efficient algorithm is devised for finding the elements of a diagonal operator for minimizing the covariance complexity when the covariance is of the Toeplitz form. Experimental results show that the algorithm reduces the information theoretic complexity and also is capable of appreciably increasing the maximum ratio of the characteristic roots of the covariance. A preprocessing operation of this type is important in data compression or feature selection for pattern recognition.