A comparison of recurrent neural network learning algorithms

Selected recurrent network training algorithms are described, and their performances are compared with respect to speed and accuracy for a given problem. Detailed complexity analyses are presented to allow more accurate comparison between training algorithms for networks with few nodes. Network performance for predicting the Mackey-Glass equation is reported for each of the recurrent networks, as well as for a backpropagation network. Using networks of comparable size, the recurrent networks produce significantly better prediction accuracy. The accuracy of the backpropagation network is improved by increasing the size of the network, but the recurrent networks continue to produce better results for the large prediction distances. Of the recurrent networks considered, Pearlmutter´s off-line training algorithm produces the best results.