Regularized Covariance Matrix Estimation via Empirical Bayes

An Empirical Bayes formalization of the regularized covariance estimation problem is proposed for (possibly high-dimensional, low-sample) normal variates. A simple iteration is provided to automatically adjust the shrinkage level, which provably converges to the maximum likelihood hyperparameter estimation for any choice of the starting point. The proposed approach is effective and can outperform both MSE-optimized diagonal loading and the Rao-Blackwell Leidot-Wolf estimator in terms of covariance-matrix-specific metrics.