The transformed Variational Bayes approximation

The purpose of this paper is to develop parameter transformation strategies that improve the accuracy of the Variational Bayes (VB) approximation. The idea is to find a transformed metric in which coupling between parameters is reduced. Parameter orthogonalization methods from the statistical literature provide a starting point, but these are extended in the paper to yield transformed distributions more amenable to the independence approximation imposed by VB. Among other contexts, the Transformed VB approximation is applied successfully to (i) the normal inverse-gamma distribution central to Bayesian inference for (auto)regressive models, and (ii) the nonlinear context of sinusoidal frequency inference, where promising performance enhancements are reported.