Title :
Fast α-weighted EM learning for neural networks of module mixtures
Author :
Matsuyama, Yasuo ; Furukawa, Satoshi ; Takeda, Naoki ; Ikeda, Takayuki
Author_Institution :
Dept. of Electr. Eng. & Comput. Sci., Waseda Univ., Tokyo, Japan
Abstract :
A class of extended logarithms is used to derive α-weighted EM (α-weighted expectation-maximization) algorithms. These extended EM algorithms (WEMs, α-EMs) have been anticipated to outperform the traditional (logarithmic) EM algorithm on speed. The traditional approach falls into a special case of the new WEM. In this paper, general theoretical discussions are given first. Then, clear-cut evidence that shows faster convergence than the ordinary EM approach are given for the case of mixture-of-expert neural networks. This process takes three steps. The first step is to show specific algorithms. Then, the convergence is theoretically checked. Thirdly, experiments on the mixture-of-expert learning are tried to show the superiority of the WEM. Besides the supervised learning, the unsupervised case for a Gaussian mixture is also examined. Faster convergence of the WEM is observed again
Keywords :
Hessian matrices; Newton-Raphson method; convergence of numerical methods; estimation theory; learning (artificial intelligence); least squares approximations; neural nets; probability; α-weighted expectation-maximization learning; Gaussian mixture; mixture-of-expert learning; mixture-of-expert neural networks; Clustering algorithms; Concrete; Convergence of numerical methods; Equations; History; Information theory; Microwave integrated circuits; Neural networks; Structural engineering;
Conference_Titel :
Neural Networks Proceedings, 1998. IEEE World Congress on Computational Intelligence. The 1998 IEEE International Joint Conference on
Conference_Location :
Anchorage, AK
Print_ISBN :
0-7803-4859-1
DOI :
10.1109/IJCNN.1998.687221
