A parameter estimation approach to artificial neural network weight selection for nonlinear system identification

A unified framework for artificial neural network (ANN) training algorithms applied to nonlinear system identification based on considering weight selection as a parameter estimation problem is presented. Three existing ANN training strategies are reviewed within this framework, including gradient-descent backpropagation, the extended Kalman algorithm, and the recursive-least-squares method. A strikingly different approach to error backpropagation is presented, resulting in the development of a novel method of backward signal propagation and target state generation for embedded layers. The technique is suitable for implementation with a linear Kalman-based update algorithm and is applied with a unique method of covariance modification for the elimination of transients associated with initial conditions. Experimental nonlinear identification results demonstrate a greatly increased rate of convergence in comparison with backpropagation. The new algorithm displayed similar rates of parameter convergence and a decreased computational overhead compared with the extended Kalman algorithm