Covariance estimation in two-level regression

This paper considers estimation of covariance matrices in multivariate linear regression models for two-level data produced by a population of similar units (individuals). The proposed Bayesian formulation assumes that the covariances for different units are sampled from a common distribution. Assuming that this common distribution is Wishart, the optimal Bayesian estimation problem is shown to be convex. This paper proposes a specialized scalable algorithm for solving this two-level optimal Bayesian estimation problem. The algorithm scales to datasets with thousands of units and trillions of data points per unit, by solving the problem recursively, allowing new data to be quickly incorporated into the estimates. An example problem is used to show that the proposed approach improves over existing approaches to estimating covariance matrices in linear models for two-level data.