Universal Weighted MSE Improvement of the Least-Squares Estimator

Since the seminal work of Stein in the 1950s, there has been continuing research devoted to improving the total mean-squared error (MSE) of the least-squares (LS) estimator in the linear regression model. However, a drawback of these methods is that although they improve the total MSE, they do so at the expense of increasing the MSE of some of the individual signal components. Here we consider a framework for developing linear estimators that outperform the LS strategy over bounded norm signals, under all weighted MSE measures. This guarantees, for example, that both the total MSE and the MSE of each of the elements will be smaller than that resulting from the LS approach. We begin by deriving an easily verifiable condition on a linear method that ensures LS domination for every weighted MSE. We then suggest a minimax estimator that minimizes the worst-case MSE over all weighting matrices and bounded norm signals subject to the universal weighted MSE domination constraint.