A Computationally Efficient Algorithm for Training Recurrent Connectionist Networks

The primary goal of this paper is to investigate the training of recurrent networks for control and signal processing applications. This paper first a characterizes a class of network architectures that are well suited for the incremental learning of nonlinear multivariable dynamic mappings, and then presents a general, computationally efficient algorithm for training this class of recurrent networks. The learning algorithm is a local modification of the Extended Kalman Filter that views the network as a parametric model of a nonlinear dynamic system. Computational efficiency of the learning scheme is achieved by exploiting local properties of the network architectures. The ability of this algorithm to train recurrent networks successfully is demonstrated by way of two examples.