Title :
An EM-based algorithm for recurrent neural networks
Author :
Ma, Sheng ; Ji, Chuanyi
Author_Institution :
Dept. of Electr. Comput. & Syst. Eng., Rensselaer Polytech. Inst., Troy, NY, USA
Abstract :
A stochastic model is established for fully-connected recurrent neural networks with sigmoid units based on Gibbs distributions. The EM (expectation-maximization) algorithm with a mean field approximation is then applied to train recurrent networks through hidden state estimation. The resulting EM-based algorithm, which reduces training the original recurrent network to training a set of individual feedforward neurons, simplifies the original training process and reduces the training time
Keywords :
approximation theory; feedforward neural nets; learning (artificial intelligence); recurrent neural nets; state estimation; stochastic processes; EM-based algorithm; Gibbs distributions; backpropagation; expectation-maximization algorithm; feedforward neurons; fully-connected recurrent neural networks; hidden state estimation; mean field approximation; recurrent networks training; recurrent neural networks; sigmoid units; stochastic model; training time reduction; Computer networks; Convergence; Information theory; Jacobian matrices; Neural networks; Neurons; Recurrent neural networks; Testing;
Conference_Titel :
Information Theory, 1995. Proceedings., 1995 IEEE International Symposium on
Conference_Location :
Whistler, BC
Print_ISBN :
0-7803-2453-6
DOI :
10.1109/ISIT.1995.531524
