Estimation of the training efficiency of recurrent neural networks

In our studies of the capabilities of neural networks, we have relied on time-lagged recurrent neural networks (TLRNN) to learn to emulate the behavior of complex dynamic systems. In this study, we take data from the physical system and train a suitable TLRNN to convergence. We then use that trained neural network to generate a set of noise-free data over the same input manifold. A variety of disturbances are introduced into the generated data so that for a given disturbance a minimum RMS error may be computed from the deviations of the perturbed outputs from the true outputs. These perturbed outputs are then used as targets to train an identically structured TLRNN to determine how close to the global minimum the training proceeds. The results indicate that, depending on the type of the noise introduced, the global Kalman filter method as well as properly formulated gradient descent method produce TLRNNs which have RMS errors deviated from the global minimum from less than 3% to about 10%