Convergence Analysis of the EM Algorithm and Joint Minimization of Free Energy

Although the expectation-maximization (EM) algorithm has been popularly used for its computational convenience, it has been recognized that the EM algorithm works slowly in certain situations. In this study, we analyze the convergence property of the EM algorithm in terms of the minimization of the free energy, and show that the slow convergence is due to the optimization method of the free energy. The analyses suggest a different optimization can be appropriate for situations of slow convergence. Then, we propose a new speeding-up method for optimization of the free energy. The validity of the new method is confirmed by using a simple problem.