Almost-sure convergence of adaptive algorithms by projections

A convergence proof is discussed for the normalized least-mean-square (LMS) algorithm for ergodic inputs. The approach is based on interpreting the algorithm as a sequence of relaxed projection operators by which the key contraction property is derived. The proof technique is strongly motivated by physical intuition, and provides additional insight into LMS-type algorithms under ergodic inputs. Embedded in the development is a slight generalization to a random time-varying gain parameter. This allows the incorporation of variations such as the LMS and signed LMS algorithms.<>