Title :
On the Randomness of Independent Experiments
Author :
Holenstein, Thomas ; Renner, Renato
Author_Institution :
Inst. of Theor. Comput. Sci., ETH Zurich, Zurich, Switzerland
fDate :
4/1/2011 12:00:00 AM
Abstract :
Smooth entropies characterize basic information-theoretic properties of random variables, such as the number of bits required to store them or the amount of uniform randomness that can be extracted from them (possibly with respect to side information). In this paper, explicit and almost tight bounds on the smooth entropies of n-fold product distributions, Pn, are derived. These bounds are expressed in terms of the Shannon entropy of a single distribution, P . The results can be seen as an extension of the asymptotic equipartition property (AEP).
Keywords :
data compression; entropy; random processes; Shannon entropy; asymptotic equipartition property; data compression; n-fold product distribution; random variable; smooth entropy; Data compression; Data mining; Encoding; Entropy; Markov processes; Probability distribution; Random variables; Data compression; nonasymptotic information theory; quantitative asymptotic equipartition property (AEP); randomness extraction; smooth entropies;
Journal_Title :
Information Theory, IEEE Transactions on
DOI :
10.1109/TIT.2011.2110230