Title :
Error bound analysis of the least-mean-squares algorithm in linear models
Author :
Jingyi Zhu ; Spall, James C.
Author_Institution :
Dept. of Appl. Math. & Stat., Johns Hopkins Univ., Baltimore, MD, USA
Abstract :
Time-varying formulation is pervasive in dynamic control systems, where tracking parameters characterizing dynamic properties is central task. General conditions for exponential stability of such systems have been fully discussed in prior work of others. In this paper we build practical (computable) error bound analysis of the stochastic gradient algorithm when the loss function is time-dependent and quadratic in the parameters, as arising from standard linear regression model. The long term goal is to address this problem in general nonlinear models. This paper is the first step towards this aim.
Keywords :
asymptotic stability; gradient methods; least mean squares methods; linear systems; nonlinear control systems; regression analysis; stochastic processes; time-varying systems; dynamic control system; error bound analysis; exponential stability; least-mean-squares algorithm; linear regression model; loss function; nonlinear model; stochastic gradient algorithm; time-varying formulation; Adaptation models; Algorithm design and analysis; Covariance matrices; Heuristic algorithms; Indexes; Least squares approximations; Stochastic processes; Adaptive Control; Constant Gain; LMS; Least-Mean-Squares Algorithm;
Conference_Titel :
Information Sciences and Systems (CISS), 2015 49th Annual Conference on
Conference_Location :
Baltimore, MD
DOI :
10.1109/CISS.2015.7086870
