Title :
A Probabilistic Upper Bound on Differential Entropy
Author :
Learned-Miller, Erik ; DeStefano, Joseph
Author_Institution :
Dept. of Comput. Sci., Univ. of Massachusetts, Amherst, MA
Abstract :
A novel probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the unknown distribution is required. Previous distribution-free bounds on the cumulative distribution function of a random variable given a sample of that variable are used to construct the bound. A simple, fast, and intuitive algorithm for computing the entropy bound from a sample is provided.
Keywords :
entropy codes; statistical distributions; cumulative distribution function; differential entropy; one-dimensional distribution; probabilistic upper bound; Computer science; Distribution functions; Engineering profession; Entropy; Pervasive computing; Physics; Probability distribution; Random variables; Statistical learning; Upper bound; Convex optimization; differential entropy; entropy; entropy bound; string-tightening algorithm;
Journal_Title :
Information Theory, IEEE Transactions on
DOI :
10.1109/TIT.2008.929937
