DocumentCode :
3416297
Title :
A partial analysis of stochastic convergence in a generalized two-layer perceptron with backpropagation learning
Author :
Vaughn, Jeffrey L. ; Bershad, Neil J. ; Shynk, John J.
Author_Institution :
Dept. of Electr. & Comput. Eng., California Univ., Irvine, CA, USA
fYear :
1992
fDate :
31 Aug-2 Sep 1992
Firstpage :
522
Lastpage :
530
Abstract :
The authors study the stationary points of a two-layer perceptron which attempts to identify the parameters of a specific stochastic nonlinear system. The training sequence is modeled as the output of the nonlinear system, with an input comprising an independent sequence of zero mean Gaussian vectors with independent components. The training rule is a limiting case of backpropagation (to simplify the analysis). Equations are given which define the stationary points of the algorithm for an arbitrary output nonlinearity g(x). The solutions to these equations for the outer layer show that, for a continuous g(x), there is a unique solution for the outer layer weights for any given set of fixed hidden layer weights. These solutions do not necessarily yield zero error. However, if the hidden layer weights are also trained, the unique solution for zero error requires that the parameters of the two-layer perceptron exactly match that of the nonlinear system
Keywords :
backpropagation; convergence; feedforward neural nets; arbitrary output nonlinearity; backpropagation learning; generalized two-layer perceptron; hidden layer weights; outer layer weights; partial analysis; stationary points; stochastic convergence; stochastic nonlinear system; training rule; training sequence; zero mean Gaussian vectors; Backpropagation algorithms; Convergence; Cost function; Information processing; Multilayer perceptrons; Nonlinear equations; Nonlinear optics; Nonlinear systems; Stochastic processes; Stochastic systems;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Neural Networks for Signal Processing [1992] II., Proceedings of the 1992 IEEE-SP Workshop
Conference_Location :
Helsingoer
Print_ISBN :
0-7803-0557-4
Type :
conf
DOI :
10.1109/NNSP.1992.253660
Filename :
253660
Link To Document :
بازگشت