An approximate method for Bayesian entropy estimation for a discrete random variable

This article proposes an approximated Bayesian entropy estimator for a discrete random variable. An entropy estimator that achieves least square error is obtained through Bayesian estimation of the occurrence probabilities of each value taken by the discrete random variable. This Bayesian entropy estimator requires large amount of calculation cost if the random variable takes numerous sorts of values. Therefore, the present article proposes a practical method for calculating an Bayesian entropy estimate; the proposed method utilizes approximation of the entropy function by a truncated Taylor series. Numerical experiments demonstrate that the proposed entropy estimation method improves estimation precision of entropy remarkably in comparison to the conventional entropy estimation method.