A measure theoretical analysis of learning algorithms for recurrent neural networks

Some learning algorithms of continuous dynamical systems for recurrent neural networks are proved to be the probabilistic-descent method. Using the concept of invariant measure, it is shown that the algorithms based on the gradient method are equivalent to the backpropagation method in the sense of average. Some numerical examples are also given to confirm the theoretical results.