Mutual information of contingency tables and related inequalities

For testing independence it is very popular to use either the χ²-statistic or G²-statistics (mutual information). Asymptotically both are χ²-distributed so an obvious question is which of the two statistics that has a distribution that is closest to the χ²-distribution. Surprisingly the distribution of mutual information is much better approximated by a χ²-distribution than the χ²-statistic. For technical reasons we shall focus on the simplest case with one degree of freedom. We introduce the signed log-likelihood and demonstrate that its distribution function can be related to the distribution function of a standard Gaussian by inequalities. For the hypergeometric distribution we formulate a general conjecture about how close the signed log-likelihood is to a standard Gaussian, and this conjecture gives much more accurate estimates of the tail probabilities of this type of distribution than previously published results. The conjecture has been proved numerically in all cases relevant for testing independence and further evidence of its validity is given.