Penalized maximum-likelihood estimation of covariance matrices with linear structure

In this paper, a space-alternating generalized expectation-maximization (SAGE) algorithm is presented for the numerical computation of maximum-likelihood (ML) and penalized ML (PML) estimates of the parameters of covariance matrices with linear structure for complex Gaussian processes. By using a less informative hidden-data space and a sequential parameter-update scheme, a SAGE-based algorithm is derived for which convergence of the likelihood is demonstrated to be significantly faster than that of an EM-based algorithm that has been previously proposed. In addition, the SAGE procedure is shown to easily accommodate penalty functions, and a SAGE-based algorithm is derived and demonstrated for forming PML estimates with a quadratic smoothness penalty