Bayesian error estimation and model selection in sparse logistic regression

Regularized logistic regression models have recently become an important classification tool for high dimensional problems due to their sparseness and embedded feature selection property of the ℓ₁ penalty. However, the degree of sparseness is determined by a regularization parameter λ, whose selection is typically done by cross validation. In this paper we study the applicability of a recently proposed Bayesian error estimation approach for the selection of a proper model along the regularization path. The model selection by the new Bayesian error estimator is experimentally shown to improve the classification accuracy in small sample-size situations, and is able to avoid the excess variability inherent to traditional cross validation approaches.