Characterizing the error function of a neural network

Several means are discussed for exploring the error function of a multilayer, feedforward neural network. In particular, hyperplane configurations over time and the generalization of the network function to a region of the input space are considered. The analysis provides explanations for empirically observed phenomena, such as flat spots in the energy surface, and improvements in performance with extra hidden units. The implementation of the back-propagation network training algorithm on the massively parallel connection machine computer is discussed. Results are presented of a scaling experiment in which the effect on learning time of the number of hidden units is studied. Also, results are presented of an experiment on adjusting the length of the initial random weight vectors. The explorations of the shape of the energy function in weight space and the hyperplane configuration are discussed. Several methods for obtaining speedup in training times based on the analyses are suggested