Title :
On the behavior of information theoretic criteria for model order selection
Author :
Liavas, Athanasios P. ; Regalia, Phillip A.
Author_Institution :
Dept. of Comput. Sci., Ioannina Univ., Greece
fDate :
8/1/2001 12:00:00 AM
Abstract :
The Akaike (1974) information criterion (AIC) and the minimum description length (MDL) are two well-known criteria for model order selection in the additive white noise case. Our aim is to study the influence on their behavior of a large gap between the signal and the noise eigenvalues and of the noise eigenvalue dispersion. Our results are mostly qualitative and serve to explain the behavior of the AIC and the MDL in some cases of great practical importance. We show that when the noise eigenvalues are not clustered sufficiently closely, then the AIC and the MDL may lead to overmodeling by ignoring an arbitrarily large gap between the signal and the noise eigenvalues. For fixed number of data samples, overmodeling becomes more likely for increasing the dispersion of the noise eigenvalues. For fixed dispersion, overmodeling becomes more likely for increasing the number of data samples. Undermodeling may happen in the cases where the signal and the noise eigenvalues are not well separated and the noise eigenvalues are clustered sufficiently closely. We illustrate our results by using simulations from the effective channel order determination area
Keywords :
digital simulation; eigenvalues and eigenfunctions; information theory; white noise; AIC; Akaike information criterion; MDL; additive white noise; channel order determination; data samples; information theoretic criteria; minimum description length; model order selection; noise eigenvalue dispersion; noise eigenvalues; overmodeling; signal eigenvalue; simulations; Additive white noise; Arithmetic; Colored noise; Computer science; Covariance matrix; Eigenvalues and eigenfunctions; Noise robustness; Signal analysis; Signal processing;
Journal_Title :
Signal Processing, IEEE Transactions on