Analysis of stability and performance of adaptation algorithms with time-invariant gains

Adaptation laws that track parameters of linear regression models are investigated. The considered class of algorithms apply linear time-invariant filtering on the instantaneous gradient vector and includes least mean squares (LMS) as its simplest member. The asymptotic stability and steady-state tracking performance for prediction and smoothing estimators is analyzed for parameter variations described by stochastic processes with time-invariant statistics. The analysis is based on a novel technique that decomposes the inherent feedback of adaptation algorithms into one time-invariant loop and one time-varying loop. The impact of the time-varying feedback on the tracking error covariance can be neglected under certain conditions, and the performance analysis then becomes straightforward. Performance analysis in the presence of a non-negligible time-varying feedback is performed for algorithms that use scalar measurements. Convergence in mean square error (MSE) and the MSE tracking performance is investigated, assuming independent consecutive regression vectors. Closed-form expressions for the tracking MSE are thereafter derived without this independence assumption for a subclass of algorithms applied to finite impulse response (FIR) models with white inputs. This class includes Wiener LMS adaptation.