Stochastic convergence analysis of a two-layer perceptron for a system identification model

This paper summarizes the results of the simulations and analysis of the learning behavior of a simple two-layer perceptron for a nonlinear system identification problem. Although it is difficult to generalize results for nonlinear systems, the analysis may improve our understanding of neural network training. Numerous sub-optimum stationary points occur for this problem and cause difficulties in the correct identification of the unknown system. The sub-optimum convergence points occur in the saturation regions of the various nonlinearities or for pathological cases. The size of the region of suboptimal convergence points may be reduced by increasing the dimensionality of the input data vector. Also, the range for the rate parameter is computed and an improvement to backpropagation is suggested