Title :
Nonparametric entropy estimation for stationary processes and random fields, with applications to English text
Author :
Kontoyiannis, I. ; Algoet, P.H. ; Suhov, Yu.M. ; Wyner, A.J.
Author_Institution :
Dept. of Electr. Eng., Stanford Univ., CA, USA
fDate :
5/1/1998 12:00:00 AM
Abstract :
We discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a Doeblin-type mixing condition. The estimators are Cesaro averages of longest match-lengths, and their consistency follows from a generalized ergodic theorem due to Maker (1940). We provide examples of their performance on English text, and we generalize our results to countable alphabet processes and to random fields
Keywords :
entropy; estimation theory; information theory; pattern matching; source coding; stochastic processes; Cesaro averages; Doeblin-type mixing condition; English text; countable alphabet processes; entropy rate; generalized ergodic theorem; longest match-lengths; mean consistency; nonparametric entropy estimation; performance; pointwise consistency; random field; stationary ergodic process; stationary processes; Algorithm design and analysis; Compression algorithms; Convergence; Data compression; Entropy; Geometry; Information systems; Laboratories; Pattern matching; Upper bound;
Journal_Title :
Information Theory, IEEE Transactions on
