Statistical analysis of the single-layer backpropagation algorithm with bias terms: mean weight behavior

This paper analyzes the perceptron mean weight learning behavior for a system identification model with Gaussian input training data and fixed non zero biases for both the perceptron and the unknown system. The analysis is based upon the partial evaluation of certain expectations using Price´s (1958) theorem followed by numerical integration in the mean weight recursions. The mean weight vector is shown to be in the same direction as that of the unknown system. A scalar recursion is derived for the length of the mean weights. The recursion is shown to yield weight vector predictions that are in close agreement with Monte Carlo simulations of the perceptron learning behavior. The stationary points are also accurately predicted by the theoretical model