An approximation to the distribution of finite sample size mutual information estimates

In this paper, the distribution of mutual information between two discrete random variables is approximated by means of a second-order Taylor series expansion. Approximative expressions for the distribution of mutual information (MI) between independent random variables, conditional MI between conditionally independent variables, and MI between (weakly) dependent random variables are derived. These distributions are functions of the available sample size and the number of realisations of the random variables only; knowledge of the variables´ PMF is not required. The results are verified numerically for various cases. Exemplary application ideas in statistics and communications engineering are proposed.