On the convergence of Shannon differential entropy, and its connections with density and entropy estimation

This work extends the study of convergence properties of the Shannon differential entropy, and its connections with the convergence of probability measures in the sense of total variation and direct and reverse information divergence. The results relate the topics of distribution (density) estimation, and Shannon information measures estimation, with special focus on the case of differential entropy. On the application side, this work presents an explicit analysis of the density estimation, and differential entropy estimation, for distributions defined on a finite-dimension Euclidean space ( R d , B ( R d ) ) . New consistency results are derived for several histogram-based estimators: the classical product scheme, the Barronʹs estimator, one of the approaches proposed by Gyِrfi and Van der Meulen, and the data-driven partition scheme of Lugosi and Nobel.