Resampling-based variable selection technique and its application to model semiconductor E-test data

The selection of a suitable set of variables is always a key concern for successful regression modeling. Though stepwise regression with setting threshold of p-value as the stopping criterion is one of the most popular techniques for variable selection, it usually suffers the over-fitting and sample-dependent problems, especially when the number of data points is small relative to the number of variables. To cope with the over-fitting problem, the data set is separated into two parts: training and testing. In addition to evaluate the p-values, the comparison of prediction errors with respect to various models conducted on testing data is also embedded in the stopping criterion. Furthermore, to cope with the sample-dependent problem, the resampling technique is applied to compare the performance of various sets of variables selected in individual resampling runs. A set of semiconductor E-test data collected from a foundry is used to demonstrate the advantages of the proposed method against stepwise regressions.