Abstract :
In this talk we establish an equivalence between the MDL principle and a maximum unconditional likelihood principle, which thereby links statistical modeling in an inextricable manner with universal coding. Generalizing the Shannon information, defined for a single distribution, we define the Stochastic Complexity of a string of data, relative to a parametric family of distributions, to be the greatest lower bound for the number of binary digits with which the observed data can be encoded. This bound also sets the greatest lower bound for the prediction errors that result when the data are predicted. A recently derived approximation to stochastic complexity for regression problems has been extended to a new model selection criterion for the ARMA models.
