Title :
Signal-flow-graph derivation of on-line gradient learning algorithms
Author :
Campolucci, Paolo ; Marchegiani, Andrea ; Uncini, Aurelio ; Piazza, Francesco
Author_Institution :
Dipartimento di Elettronica e Autom., Ancona Univ., Italy
Abstract :
In this paper, making use of the signal-flow-graph (SFG) representation and its known properties, we derive a new general method for backward gradient computation of a system output or cost function with respect to past (or present) system parameters. The system can be any causal, in general nonlinear and time-variant dynamic system represented by a SFG, in particular any feedforward or recurrent neural network. In this work we use discrete time notation, but the same theory holds for the continuous time case. The gradient is obtained by the analysis of two SFGs, the original one and its adjoint. This method can be used both for online and off-line learning. In the latter case using the mean square error cost function, our approach particularises to Wan´s method (1996) that is not suited for online training of recurrent networks. Computer simulations of nonlinear dynamic systems identification will also be presented to assess the performance of the algorithm resulting from the application of the proposed method in the case of locally recurrent neural networks
Keywords :
feedforward neural nets; learning (artificial intelligence); recurrent neural nets; signal flow graphs; SFG; backward gradient computation; causal nonlinear time-variant dynamic system; discrete time notation; feedforward neural network; locally recurrent neural networks; mean square error cost function; nonlinear dynamic systems identification; off-line learning; online gradient learning algorithms; recurrent neural network; signal-flow-graph representation; Application software; Computer simulation; Cost function; Feedforward neural networks; Flow graphs; Mean square error methods; Neural networks; Nonlinear dynamical systems; Recurrent neural networks; System identification;
Conference_Titel :
Neural Networks,1997., International Conference on
Conference_Location :
Houston, TX
Print_ISBN :
0-7803-4122-8
DOI :
10.1109/ICNN.1997.614186