Create Stable Neural Networks by Cross-Validation

This paper studies how to learn a stable neural network through the use of cross-validation. Cross-validation has been widely used for estimating the performance of neural networks and early stopping of training. Although cross-validation could give a good estimate of the generalisation errors of the trained neural networks, the question of selecting an neural network to use remains. This paper proposes a new method to train a stable neural network by approximately mapping the output of an average of a set of neural networks obtained from cross-validation. Two experiments have been conducted to show how different the generalisation errors of the trained neural networks from cross-validation could be and how stable an neural network would be by learning the average output of a set of neural networks.