Title :
Limit results on pattern entropy
Author :
Orlitsky, A. ; Santhanam, N.P. ; Viswanathan, K. ; Zhang, J.
Abstract :
We determine the entropy rate of patterns of i.i.d. strings and show that they satisfy an asymptotic equipartition property. We prove that for discrete distributions the entropy rate of patterns equals that of the distribution, and that for distributions with continuous probability q, the entropy rate of patterns equals that of a modified distribution where the continuous probability is assigned to a new discrete element. One implication of these results is that for discrete distributions the conditional entropy rate of the sequence when its pattern is known is zero. We address only distributions with finite entropy.
Keywords :
data compression; entropy; sequences; statistical distributions; asymptotic equipartition property; continuous probability; discrete distributions; entropy rate; i.i.d. strings; limit results; pattern entropy; sequence; universal compression; Entropy; H infinity control; Probability density function; Speech; Sufficient conditions;
Conference_Titel :
Information Theory Workshop, 2004. IEEE
Print_ISBN :
0-7803-8720-1
DOI :
10.1109/ITW.2004.1405282