Title :
The LMS algorithm with momentum updating
Author :
Shynk, John J. ; Roy, Sumit
Author_Institution :
Dept. Electr. & Comput. Eng., California Univ., Santa Barbara, CA, USA
Abstract :
Several modifications of the well-known least-mean-square (LMS) algorithm have been proposed for improved operation. The authors analyze one such recent innovation that corresponds to the ordinary LMS algorithm with an additional momentum term, parameterized by the factor alpha . The analysis of convergence in the mean yields some novel behavior insofar that it leads to complex eigenvalues of the transition matrix for the mean weight vector. The convergence in the mean-square analysis demonstrates that instability will occur as alpha tends closer to 1, a result not predicted by the analysis of convergence in the mean.<>
Keywords :
convergence of numerical methods; eigenvalues and eigenfunctions; least squares approximations; matrix algebra; LMS algorithm; complex eigenvalues; convergence; least mean square algorithm; mean weight vector; momentum updating; transition matrix; Adaptive algorithm; Algorithm design and analysis; Computational complexity; Contracts; Convergence; Eigenvalues and eigenfunctions; Least squares approximation; Multi-layer neural network; Neural networks; Technological innovation;
Conference_Titel :
Circuits and Systems, 1988., IEEE International Symposium on
Conference_Location :
Espoo, Finland
DOI :
10.1109/ISCAS.1988.15485
