An experimental study of several decision issues for feature selection with multi-layer perceptrons

An experimental study of several decision issues for wrapper feature selection with multi-layer perceptrons is presented, namely the stopping criterion, the data set where the saliency is measured and the network retraining before computing the saliency. Experimental results with the sequential backward selection procedure indicate that the increase in the computational cost associated with retraining the network with every feature temporarily removed before computing the saliency is rewarded with a significant performance improvement. Despite being quite intuitive, this idea has been hardly used in practice. Regarding the stopping criterion and the data set where the saliency is measured, the procedure profits from measuring the saliency in a validation set, as reasonably expected. A somehow non-intuitive conclusion can be drawn by looking at the stopping criterion, where it is suggested that forcing overtraining may be as useful as early stopping.