Heuristic Methods to Combat the Regression Challenges

Nowadays, high–dimensional data in which the number of observations is smaller than the number of parameters, appear in many practical applications such as biosciences, social networks, psychological researches, recommendation systems and so on. In the regression model analysis, the well–known ordinary least–squares estimation may not be applicable when the classical assumptions such as normality of the error terms and full ranking of the design matrix are violated. As known, a successful approach for high–dimension cases is the penalized scheme (such as LASSO) with the aim of obtaining a subset of effective explanatory variables that predict the dependent variable as the best, while setting the other parameters to zero. Here, we review and develop several penalized models to be used in high-dimension regression analysis for high-dimensional data sets. In this paper, we apply eye data to evaluating a strategy for detecting human eye disease.