Title :
On the estimation of entropy for unknown support size
Author :
Schober, Steffen ; Mansour, Ayman
Author_Institution :
Inst. of Commun. Eng., Ulm Univ., Ulm, Germany
Abstract :
We consider the problem of estimating the entropy of a distribution P of unknown support size N provided with k independently drawn samples, where we are mainly focused on the low sampling regime where k <; N. By extending a method of Hausser and Strimmer [1], which assumes a known support size, by using the support size estimator of Chao and Lee [2], we obtain a simple estimator that performs equally good as the methods of Nemenman, Shafee, and Bialek [3] and Chao and Chen [4].
Keywords :
entropy; estimation theory; Chao-and-Lee support size estimator; Hausser-and-Strimmer method; distribution entropy estimation problem; low sampling regime; unknown support size; Bayesian methods; Chaotic communication; Conferences; Entropy; Information theory; Maximum likelihood estimation;
Conference_Titel :
Information Theory Workshop (ITW), 2012 IEEE
Conference_Location :
Lausanne
Print_ISBN :
978-1-4673-0224-1
Electronic_ISBN :
978-1-4673-0222-7
DOI :
10.1109/ITW.2012.6404729
