Entropy and Kullback-Leibler divergence estimation based on Szegö´s theorem

In this work, a new technique for the estimation of the Shannon´s entropy and the Kullback-Leibler (KL) divergence for one dimensional data is presented. The estimator is based on the Szegö´s theorem for sequences of Toeplitz matrices, which deals with the asymptotic behavior of the eigenvalues of those matrices, and the analogy between a probability density function (PDF) and a power spectral density (PSD), which allows us to estimate a PDF of bounded support using the well-known spectral estimation techniques. Specifically, an AR model is used for the PDF/PSD estimation, and the entropy is easily estimated as a function of the eigenvalues of the autocorrelation Toeplitz matrix. The performance of the Szegö´s estimators is illustrated by means of Monte Carlo simulations and compared with previously proposed alternatives, showing a good performance.