A high probability bound on the mutual information across an observed Discrete Memoryless Channel

This paper introduces several new high probability bounds on the average mutual information, I (X; Y ) between the input and output of a Discrete Memoryless Channel (DMC) based on a set of observed input-output samples. We assume that channel-input probability mass function and channel transition probability mass functions are unknown. We compare a Bayesian-Dirichlet distribution model against a computationally-tractable distribution-free high-probability bound to determine (1) the rate of information possibly flowing across the channel and (2) the rate information could flow across the channel. The performance (as a function of N, the number of observed samples) is compared via simulation. The rate of convergence is towards to ´true´ mutual information rate shown to be O (log(N)/N).