Principal Components Selection by the Criterion of the Minimum Mean Difference of Complexity

Based on the concept of complexity or minimum description length developed by Kolmogorov, Rissanen, Wallace, and others, an index of predictive power is proposed as a criterion to select the principal components of a random vector distributed in a parametric family. This criterion, when applied to the principal components selection, considers the lost information due to the reduction of the parameters as well as the observed variables. The principal components, obtained by minimizing the index of predictive power, turn out to be identical to the classical principal components when the assumed distribution is normal. A test procedure for the principal components selection is constructed and discussed. Finally, principal components for a type of ϵ-contaminated normal family are given, and are shown to converge to those of the normal distribution. Results from a simulation study are also presented.