A unifying view of stochastic approximation, Kalman filter and backpropagation

Author

Capobianco, Enrico

Author_Institution

Dept. of Stat., Padova Univ., Italy

fYear

1995

fDate

31 Aug-2 Sep 1995

Firstpage

87

Lastpage

94

Abstract

In this paper the relationships between the stochastic approximation, the Kalman filter and the backpropagation algorithms are investigated. We show that when the neural network architecture at hand can be formalized such that the approximation of the optimum for a nonlinear objective function is the problem for which we seek a solution, then both stochastic approximation techniques and appropriate Kalman filters can be employed in order to reach the goal but the latter can also handle various structural characteristics of the stochastic processes involved and suggest a more efficient two-step estimator

Keywords

Kalman filters; backpropagation; feedforward neural nets; function approximation; state-space methods; stochastic processes; Kalman filter; backpropagation; feedforward neural network; nonlinear objective function; parameter estimation; state space; stochastic approximation; stochastic processes; Approximation algorithms; Backpropagation algorithms; Least squares approximation; Least squares methods; Neural networks; Newton method; Parameter estimation; Recursive estimation; Statistics; Stochastic processes;

fLanguage

English

Publisher

ieee

Conference_Titel

Neural Networks for Signal Processing [1995] V. Proceedings of the 1995 IEEE Workshop

Conference_Location

Cambridge, MA

Print_ISBN

0-7803-2739-X

Type

conf

DOI

10.1109/NNSP.1995.514882

Filename

514882