Entropy estimation using MDL and piecewise constant density models

We compare the performance of MDL-based estimators of the differential entropy of scalar random variables to several state-of-the-art entropy estimators. The estimators studied determine the entropy of a piecewise constant probability density function whose complexity the number of intervals of constant density value, or ldquobinsrdquo is automatically adjusted to the sample size. The estimators are based on an efficient implementation of the Maximum Likelihood Estimator that has been recently proposed in the literature. Simulation studies for several data distribution show that if the MDL penalty is conveniently defined, performance compares favorably with the state-of-the-art entropy estimators tested, while at the same time providing a compact model for the data set analyzed.