Title :
A normalized gradient algorithm for an adaptive recurrent perceptron
Author :
Chambers, Jonathon A. ; Sherliker, Warren ; Mandic, Danilo R.
Author_Institution :
Dept. of Electr. & Electron. Eng., Imperial Coll. of Sci., Technol. & Med., London, UK
Abstract :
A normalized algorithm for on-line adaptation of a recurrent perceptron is derived. The algorithm builds upon the normalized backpropagation (NBP) algorithm for feedforward neural networks, and provides an adaptive learning rate and normalization for a recurrent perceptron learning algorithm. The algorithm is based upon local linearization about the current point in the state-space of the network. Such a learning rate is normalized by the squared norm of the gradient at the neuron, which extends the notion of normalized linear algorithms to the nonlinear case
Keywords :
adaptive systems; backpropagation; convergence of numerical methods; feedforward neural nets; gradient methods; least mean squares methods; recurrent neural nets; state-space methods; LMS based nonlinear algorithm; adaptive learning rate; adaptive recurrent perceptron; convergence; feedforward neural networks; local linearization; normalized backpropagation algorithm; normalized gradient algorithm; normalized learning rate; normalized linear algorithms; on-line adaptation; real-time gradient based algorithm; recurrent perceptron learning algorithm; stability; state-space; Adaptive filters; Backpropagation algorithms; Convergence; Lagrangian functions; Least squares approximation; Neural networks; Nonlinear filters; Recurrent neural networks; Stability; State-space methods;
Conference_Titel :
Acoustics, Speech, and Signal Processing, 2000. ICASSP '00. Proceedings. 2000 IEEE International Conference on
Conference_Location :
Istanbul
Print_ISBN :
0-7803-6293-4
DOI :
10.1109/ICASSP.2000.861988
