Bayesian high-dimensional screening via MCMC

We explore the theoretical and numerical properties of a fully Bayesian model selection method in the context of sparse high-dimensional settings, i.e., p ≫ n , where p is the number of covariates and n is the sample size. Our method consists of (1) a hierarchical Bayesian model with a novel prior placed over the model space which includes a hyperparameter t n controlling the model size and (2) an efficient MCMC algorithm for automatic and stochastic search of the models. Our theory shows that, when specifying t n correctly, the proposed method yields selection consistency, i.e., the posterior probability of the true model asymptotically approaches one; when t n is misspecified, the selected model is still asymptotically nested in the true model. The theory also reveals insensitivity of the selection result with respect to the choice of t n . In implementations, a reasonable prior is further assumed on t n . Our approach conducts selection, estimation and even inference in a unified framework. No additional prescreening or dimension reduction step is needed. Two novel g -priors are proposed to make our approach more flexible. The numerical advantages of the proposed approach are demonstrated through comparisons with sure independence screening (SIS).