Admissibility of memorization learning with respect to projection learning in the presence of noise

In the training of neural networks using the error-backpropagation (BP) algorithm, over-learning phenomenon has been observed. In previous works we showed how over-learning can be viewed as being the result of using the BP criterion as a substitute for some true criterion. There, the concept of admissibility was introduced and discussed conditions for a true criterion admits a substitute criterion. In this paper we provide necessary and sufficient conditions for the projection learning to admit the memorization learning in the presence of noise. Based on these conditions, we devise methods for choosing training sets to prevent over-learning