Selection of training samples for learning with hints

Training with hints is a powerful method for incorporating almost any type of prior knowledge into neural network models. In this paper we demonstrate how the hints can be constructed from numerical approximation of the regularization cost function, and discuss the problem of selecting the hint samples. We give a simple algorithm for placing the hint samples in such regions in the input space where the hint error is large, and for selecting the minimum sufficient set of hint samples by removing the correlated samples