On the behavior of information theoretic criteria for model order selection

The Akaike (1974) information criterion (AIC) and the minimum description length (MDL) are two well-known criteria for model order selection in the additive white noise case. Our aim is to study the influence on their behavior of a large gap between the signal and the noise eigenvalues and of the noise eigenvalue dispersion. Our results are mostly qualitative and serve to explain the behavior of the AIC and the MDL in some cases of great practical importance. We show that when the noise eigenvalues are not clustered sufficiently closely, then the AIC and the MDL may lead to overmodeling by ignoring an arbitrarily large gap between the signal and the noise eigenvalues. For fixed number of data samples, overmodeling becomes more likely for increasing the dispersion of the noise eigenvalues. For fixed dispersion, overmodeling becomes more likely for increasing the number of data samples. Undermodeling may happen in the cases where the signal and the noise eigenvalues are not well separated and the noise eigenvalues are clustered sufficiently closely. We illustrate our results by using simulations from the effective channel order determination area