Inference by optimal encoding

It is argued that the ideas of optimal coding theory offer a general approach to the problem of inductive statistical inference. The coding approach supposes that the data are to be encoded and transmitted to a receiver who must be able to decode the message and recover the data exactly. The message is to be in two parts. The first will contain an assertion of the systematic pattern, in the form of a specification of a statistical model together with estimated values of any unknown parameters it contains. The second part comprises the data encoded using the specified pattern. If a simple model is used, the first part will be short but the second will be relatively long, as the data will not be well-fitted by the model. Increasing the complexity of the model will lengthen the first part but shorten the second since the pattern can be better exploited to encode the data more efficiently. Considering the total length of the message gives a very simple and natural tradeoff between complexity and goodness-of-fit