Dynamically regularized maximum likelihood learning of Gaussian mixtures

The Gaussian mixture model is widely applied in the fields of data analysis and information processing. Recently, its parameter learning with adaptive model selection, i.e., the adaptive selection of number of Gaussian distributions in the mixture for a given sample dataset, has become an attracting and interesting topic. In this paper, we propose a dynamically regularized maximum likelihood learning (DRMLL) algorithm for Gaussian mixtures with adaptive model selection. The basic idea is that the Bayesian Ying-Yang (BYY) harmony learning is interpreted as the maximum likelihood learning regularized by the average Shannon entropy of the posterior probability per sample scaled by a positive parameter. As this scale parameter dynamically decreases from 1 to 0, the DRMLL algorithm transforms from the BYY harmony learning with adaptive model selection to the final maximum likelihood (ML) learning. It is demonstrated by simulation experiments that the DRMLL algorithm can not only select the correct number of actual Gaussian distributions in a dataset, but also obtain ML estimates of the parameters in the original mixture.