DocumentCode
1145490
Title
Redundancy of universal coding, Kolmogorov complexity, and Hausdorff dimension
Author
Takahashi, Hayato
Author_Institution
Inst. of Stat. Math., Tokyo, Japan
Volume
50
Issue
11
fYear
2004
Firstpage
2727
Lastpage
2736
Abstract
We study asymptotic code lengths of universal codes for parametric models. We show a universal code whose code length is asymptotically less than or equal to that of the minimum description length (MDL) code. Especially when some of the parameters of a source are not random reals, the coefficient of the logarithm in the formula of our universal code is less than that of the MDL code. We describe the redundancy in terms of Kolmogorov complexity and Hausdorff dimension. We show that our universal code is asymptotically optimal in the sense that the coefficient of the logarithm in the formula of the code length is minimal. Our universal code can be considered to be a natural extension of the Shannon code and the MDL code.
Keywords
Bayes methods; encoding; probability; redundancy; Bayes methods; Hausdorff dimension; Kolmogorov complexity; MDL code; Shannon code; apriori probability; asymptotic code length; minimum description length; parametric model; stochastic complexity; universal code; Fractals; Information theory; Materials science and technology; Mathematics; Maximum likelihood estimation; Parametric statistics; Physics; Stochastic resonance; Upper bound; A priori probability; Bayes methods; Hausdorff dimension; Kolmogorov complexity; MDL; minimum description length; stochastic complexity; universal coding;
fLanguage
English
Journal_Title
Information Theory, IEEE Transactions on
Publisher
ieee
ISSN
0018-9448
Type
jour
DOI
10.1109/TIT.2004.836663
Filename
1347358
Link To Document