Title :
An inequality on entropy
Author :
Mceliece, Robert J. ; Yu, Zhong
Author_Institution :
Dept. of Electr. Eng., California Inst. of Technol., Pasadena, CA, USA
Abstract :
The entropy H(X) of a discrete random variable X of alphabet size m is always non-negative and upper-bounded by log m. In this paper, we present a theorem which gives a non-trivial lower bound for H(X). We show that for any discrete random variable X with range R={x0,…,xm-1}, if pi=Pr{X=xi} and p0⩾p1⩾…pm-1, then H(X)⩾(2logm)/(m-1)Σi=0m-1ipi, with equality iff (i) X is uniformly distributed, i.e., pi=1/m for all i, or trivially (ii) p0=1, and p i=0 for 1⩽i⩽m-1
Keywords :
entropy; random processes; statistical analysis; alphabet size; discrete random variable; entropy; equality; inequality; nontrivial lower bound; uniformly distribution; Bridges; Constraint theory; Distributed computing; Entropy; Mathematics; Postal services; Random variables; Upper bound;
Conference_Titel :
Information Theory, 1995. Proceedings., 1995 IEEE International Symposium on
Conference_Location :
Whistler, BC
Print_ISBN :
0-7803-2453-6
DOI :
10.1109/ISIT.1995.550316