• DocumentCode
    1145490
  • Title

    Redundancy of universal coding, Kolmogorov complexity, and Hausdorff dimension

  • Author

    Takahashi, Hayato

  • Author_Institution
    Inst. of Stat. Math., Tokyo, Japan
  • Volume
    50
  • Issue
    11
  • fYear
    2004
  • Firstpage
    2727
  • Lastpage
    2736
  • Abstract
    We study asymptotic code lengths of universal codes for parametric models. We show a universal code whose code length is asymptotically less than or equal to that of the minimum description length (MDL) code. Especially when some of the parameters of a source are not random reals, the coefficient of the logarithm in the formula of our universal code is less than that of the MDL code. We describe the redundancy in terms of Kolmogorov complexity and Hausdorff dimension. We show that our universal code is asymptotically optimal in the sense that the coefficient of the logarithm in the formula of the code length is minimal. Our universal code can be considered to be a natural extension of the Shannon code and the MDL code.
  • Keywords
    Bayes methods; encoding; probability; redundancy; Bayes methods; Hausdorff dimension; Kolmogorov complexity; MDL code; Shannon code; apriori probability; asymptotic code length; minimum description length; parametric model; stochastic complexity; universal code; Fractals; Information theory; Materials science and technology; Mathematics; Maximum likelihood estimation; Parametric statistics; Physics; Stochastic resonance; Upper bound; A priori probability; Bayes methods; Hausdorff dimension; Kolmogorov complexity; MDL; minimum description length; stochastic complexity; universal coding;
  • fLanguage
    English
  • Journal_Title
    Information Theory, IEEE Transactions on
  • Publisher
    ieee
  • ISSN
    0018-9448
  • Type

    jour

  • DOI
    10.1109/TIT.2004.836663
  • Filename
    1347358