Minimax estimation of deterministic parameters in linear models with a random model matrix

We consider the problem of estimating an unknown deterministic parameter vector in a linear model with a random model matrix, with known second-order statistics. We first seek the linear estimator that minimizes the worst-case mean-squared error (MSE) across all parameter vectors whose (possibly weighted) norm is bounded above. We show that the minimax MSE estimator can be found by solving a semidefinite programming problem and develop necessary and sufficient optimality conditions on the minimax MSE estimator. Using these conditions, we derive closed-form expressions for the minimax MSE estimator in some special cases. We then demonstrate, through examples, that the minimax MSE estimator can improve the performance over both a Baysian approach and a least-squares method. We then consider the case in which the norm of the parameter vector is also bounded below. Since the minimax MSE approach cannot account for a nonzero lower bound, we consider, in this case, a minimax regret method in which we seek the estimator that minimizes the worst-case difference between the MSE attainable using a linear estimator that does not know the parameter vector, and the optimal MSE attained using a linear estimator that knows the parameter vector. For analytical tractability, we restrict our attention to the scalar case and develop a closed-form expression for the minimax regret estimator.