Variable selection in wavelet regression models Original Research Article

Variable selection and compression are often used to produce more parsimonious regression models. But when they are applied directly to the original spectrum domain, it is not easy to determine the type of feature the selected variables represent. By performing variable selection in the wavelet domain we show that it is possible to identify important variables as being part of short- or large-scale features. Therefore, the suggested method is to extract information about the selected variables that otherwise would have been inaccessible. We are also able to obtain information about the location of these features in the original domain. In this article we demonstrate three types of variable selection methods applied to the wavelet domain: selection of optimal combination of scales, thresholding based on mutual information and truncation of weight vectors in the partial least squares (PLS) regression algorithm. We found that truncation of weight vectors in PLS was the most effective method for selecting variables. For the two experimental data sets tested we obtained approximately the same prediction error using less than 1% (for Data set 1) and 10% (for Data set 2) of the original variables. We also discovered that the selected variables were restricted to a limited number of wavelet scales. This information can be used to suggest whether the underlying features may be dominated by narrow (selective) peaks (indicated by variables in short wavelet scale regions) or by broader regions (indicated by variables in long wavelet scale regions). Thus, wavelet regression is here used as an extension of the more traditional Fourier regression (where the modelling is performed in the frequency domain without taking into consideration any of the information in the time domain).