On the design of nonlinear speech predictors with recurrent nets

A dynamic, nonlinear speech predictor trained with real-time recurrent learning (RTRL) can achieve 2-2.5 dB better predictive gain than a conventional linear predictor. The drawback of the RTRL is that it requires a great deal of computation. For a predictor consisting of N recurrent units, the computational complexity is about O(N⁴). We propose a simplified RTRL by investigating the evolution process of the gradient in a recurrent net and reduce the computational complexity to O(N³). On a number of prediction tasks with speech signals, we show that that the simplified RTRL obtains the same prediction accuracy as the RTRL algorithm