AR(∞) estimation and nonparametric stochastic complexity

Let H* be the transfer function of a linear stochastic system such that H* and its inverse are in H^∞(D). Writing the system as an AR(∞) system, the best AR (k) approximation of the system is estimated using the method of least squares. A useful representation theorem for the parameter estimation error is presented. The effect of undermodeling and parameter uncertainty (due to estimation) on honest prediction, and the optimal choice of k, are questioned. This question is answered and the result is applied to the AR approximation of ARMA systems. The excess of the mean of the nonparametric stochastic complexity with respect to the AR class of an ARMA system with zeros less than 1/β in moduli is found asymptotically to be less than σ²log²N/logβ after N samples