Knowledge Versus Information Contents

While the information contents of a binary string x can be measured by its prefix Kolmogorov complexity K(x), it is not clear how to measure the knowledge stored in x. In this paper, we argue that the knowledge contained by x is relative to the hypothesis assumed to explain x. So, if H is a hypothesis for x, we suggest to measure the knowledge in x by K(H). The absolute knowledge in x is K(H₀), where H₀ is a simplest model capable of explaining x. Using Bayes´ rule and Solomonoff ´s universal distribution, we obtain K(x) = K(H) + K(x | H). We interpret K(H) as the knowledge part in x and K(x / H) as the random aspect (accidental information) in x relative to H. Furthermore, we provide a simple explanation for Kolmogorov´s innovative proposal for a non-probabilistic approach to statistics and model selection. We observe that the expression used by Kolmogorov to describe positively probabilistically random objects is a rewrite of Bayes´ rule combined with approximations based on Solomonoff ´s universal distribution. We revisit the role of algorithmic sufficient statistic in the theory of hypothesis selection and prediction, especially as related to Kolmogorov´s structure function and non-stochastic objects. Also, We derive a fundamental result relating Kolmogorov´s structure function and two of its variants.