A normalized gradient algorithm for an adaptive recurrent perceptron

Author

Chambers, Jonathon A. ; Sherliker, Warren ; Mandic, Danilo R.

Author_Institution

Dept. of Electr. & Electron. Eng., Imperial Coll. of Sci., Technol. & Med., London, UK

Volume

1

fYear

2000

fDate

2000

Firstpage

396

Abstract

A normalized algorithm for on-line adaptation of a recurrent perceptron is derived. The algorithm builds upon the normalized backpropagation (NBP) algorithm for feedforward neural networks, and provides an adaptive learning rate and normalization for a recurrent perceptron learning algorithm. The algorithm is based upon local linearization about the current point in the state-space of the network. Such a learning rate is normalized by the squared norm of the gradient at the neuron, which extends the notion of normalized linear algorithms to the nonlinear case

Keywords

adaptive systems; backpropagation; convergence of numerical methods; feedforward neural nets; gradient methods; least mean squares methods; recurrent neural nets; state-space methods; LMS based nonlinear algorithm; adaptive learning rate; adaptive recurrent perceptron; convergence; feedforward neural networks; local linearization; normalized backpropagation algorithm; normalized gradient algorithm; normalized learning rate; normalized linear algorithms; on-line adaptation; real-time gradient based algorithm; recurrent perceptron learning algorithm; stability; state-space; Adaptive filters; Backpropagation algorithms; Convergence; Lagrangian functions; Least squares approximation; Neural networks; Nonlinear filters; Recurrent neural networks; Stability; State-space methods;

fLanguage

English

Publisher

ieee

Conference_Titel

Acoustics, Speech, and Signal Processing, 2000. ICASSP '00. Proceedings. 2000 IEEE International Conference on

Conference_Location

Istanbul

ISSN

1520-6149

Print_ISBN

0-7803-6293-4

Type

conf

DOI

10.1109/ICASSP.2000.861988

Filename

861988