Control-theoretic design of the LMS and the sign algorithms in nonstationary environments

The feedback structure of the LMS (least mean squared) algorithm proposed by B. Widrow et al. (Proc. IEEE, vol.64, p.1151-62, Aug. 1976) is reexamined from a control system design viewpoint. The minimization of the misadjustments due to gradient noise and lag can then be recast as the disturbance rejection and tracking problems in control. A frequency-domain approach to the latter problems that has the advantages of transparency, ease of computation, and generality compared with the time-domain approach previously used is presented. With the same set of assumptions of white input and a Markovian plant, it is shown that the optimum step size obtained by the present approach is identical to that obtained by Widrow. Applying the same approach, the optimum step size of a simplified version of the LMS algorithm-the sign algorithm-is derived for the case when the plant is slowly varying and the input signals are Gaussian