One universally efficient estimation of the first-order autoregressive parameter and universal data compression

A universal nearly efficient estimator is proposed for the first-order autoregressive (AR) model where the probability distribution of the driving noise is unknown. It is shown that universal estimators for the AR model can be derived from universal data compression algorithms and universal tests for randomness. In other words, estimators derived appropriately from efficient universal codes can be expected to inherit good estimation performance under some conditions. The proposed estimator has a simple information-theoretic interpretation related to universal coding, which can be easily generalized to the higher-order case and to other parametric models, e.g. the one-sample location model, the two-sample location model, and the linear regression model