Title :
Simple algorithm for recurrent neural networks that can learn sequence completion
Author :
Szita, István ; Lõrincz, András
Author_Institution :
Dept. of Information Syst., Eotvos Lorand Univ., Budapest, Hungary
Abstract :
We can memorize long sequences like melodies or poems and it is intriguing to develop efficient connectionist representations for this problem. Recurrent neural networks have been proved to offer a reasonable approach here. We start from a few axiomatic assumptions and provide a simple mathematical framework that encapsulates the problem. A gradient-descent based algorithm is derived in this framework. Demonstrations on a benchmark problem show the applicability of our approach.
Keywords :
gradient methods; learning (artificial intelligence); mathematical analysis; recurrent neural nets; axiomatic assumptions; connectionist representations; gradient-descent based algorithm; mathematical framework; recurrent neural network; sequence completion learning; Backpropagation algorithms; Chaos; Electronic mail; Humans; Information systems; Mathematical model; Neural networks; Prediction methods; Recurrent neural networks; Signal processing algorithms;
Conference_Titel :
Neural Networks, 2004. Proceedings. 2004 IEEE International Joint Conference on
Print_ISBN :
0-7803-8359-1
DOI :
10.1109/IJCNN.2004.1379895
