Information theory measures with application to model identification

The identification of the order of a model which is fitted to data is of central importance. We have considered in this paper an information theoretic criterion, the AIC, which has found considerable use in many diverse applications. Our discussion is presented with various points in mind. First of all, since the theory of the AIC appears in the statistical literature in a complex form, we have attempted to present the salient points of the development in a simplified manner coupled with a geometrical interpretation which we hope will illuminate the nature of this criterion. Furthermore, we have illustrated the development with an analytical example and have compared the AIC to other measures which have been proposed in the literature. Diverse applications of the AIC, which we have investigated, include time series modeling, parametric inverse problems, and spectral analysis. We have found the AIC to be both a versatile and, what is particularly important, a robust criterion.