Abstract :
In this paper, we establish upper and lower bounds for some statistical estimation problems through concise information-theoretic arguments. Our upper bound analysis is based on a simple yet general inequality which we call the information exponential inequality. We show that this inequality naturally leads to a general randomized estimation method, for which performance upper bounds can be obtained. The lower bounds, applicable for all statistical estimators, are obtained by original applications of some well known information-theoretic inequalities, and approximately match the obtained upper bounds for various important problems. Moreover, our framework can be regarded as a natural generalization of the standard minimax framework, in that we allow the performance of the estimator to vary for different possible underlying distributions according to a predefined prior
Keywords :
information theory; minimax techniques; random processes; statistical analysis; general randomized estimation method; information-theoretic inequality; lower bound analysis; standard minimax framework; statistical estimation; upper bound analysis; Additives; Bayesian methods; Helium; Information analysis; Minimax techniques; Pattern recognition; Probability; Random variables; Statistical learning; Upper bound; Gibbs algorithm; PAC-Bayes; lower bound; minimax; randomized estimatin; statistical estimation;
