Title :
A Bit of Information Theory, and the Data Augmentation Algorithm Converges
Author_Institution :
Dept. of Stat., Univ. of California, Irvine, CA
Abstract :
The data augmentation (DA) algorithm is a simple and powerful tool in statistical computing. In this note basic information theory is used to prove a nontrivial convergence theorem for the DA algorithm.
Keywords :
information theory; statistical analysis; data augmentation algorithm; information theory; nontrivial convergence theorem; statistical computing; Bayesian methods; Convergence; Density measurement; Entropy; Extraterrestrial measurements; Information geometry; Information theory; Monte Carlo methods; Probability; Sampling methods; Gibbs sampling; I-projection; Kullback–Leibler divergence; Markov chain Monte Carlo; Pinsker´s inequality; information geometry; relative entropy; reverse I-projection; total variation;
Journal_Title :
Information Theory, IEEE Transactions on
DOI :
10.1109/TIT.2008.929918
