Statistical analysis of the single-layer backpropagation algorithm

The authors present a statistical analysis of the steady-state and transient properties of the single-layer backpropagation algorithm for Gaussian input signals. It is based on a nonlinear system identification model of the desired response which is capable of generating an arbitrary hyperplane decision boundary. It is demonstrated that, although the weights grow unbounded, the mean-square error decreases towards zero. These results indicate that the algorithm, on average, quickly learns the correct hyperplane associated with the system identification model. However, the nature of the mean-square error and the corresponding performance surface are such that the perceptron is prevented from correctly classifying with probability one until the weights converge at infinity