On the statistical efficiency of LMS algorithms

Author

Widrow, Bernard

Author_Institution

Dept. of Electr. Eng., Stanford Univ., CA, USA

Volume

1

fYear

2004

fDate

7-10 Nov. 2004

Abstract

Summary form only given. This paper describes the statistical efficiency of LMS algorithms. In this work, two gradient descent adaptive algorithms are compared, the LMS algorithm and the LMS/Newton algorithm. LMS is simple and practical and is used in many applications worldwide. LMS/Newton is based on Newton´s method and the LMS algorithm. LMS/Newton is optimal in the least squares sense. It maximizes the quality of its adaptive solution while minimizing the use of training data. Many least squares adaptive algorithms have been devised over the years, but no other least squares algorithm can give better performance, on average, than LMS/Newton. Furthermore, LMS algorithm is related to the famous backpropagation algorithm used for training neural networks.

Keywords

adaptive filters; backpropagation; gradient methods; least mean squares methods; neural nets; statistical analysis; LMS algorithm; LMS-Newton algorithm; adaptive filter; backpropagation algorithm; gradient descent adaptive algorithm; neural network training; statistical efficiency; Adaptive algorithm; Autocorrelation; Backpropagation algorithms; Eigenvalues and eigenfunctions; Least squares approximation; Least squares methods; Neural networks; Performance evaluation; Testing; Training data;

fLanguage

English

Publisher

ieee

Conference_Titel

Signals, Systems and Computers, 2004. Conference Record of the Thirty-Eighth Asilomar Conference on

Print_ISBN

0-7803-8622-1

Type

conf

DOI

10.1109/ACSSC.2004.1399099

Filename

1399099