Error bound analysis of the least-mean-squares algorithm in linear models

Author

Jingyi Zhu ; Spall, James C.

Author_Institution

Dept. of Appl. Math. & Stat., Johns Hopkins Univ., Baltimore, MD, USA

fYear

2015

fDate

18-20 March 2015

Firstpage

1

Lastpage

6

Abstract

Time-varying formulation is pervasive in dynamic control systems, where tracking parameters characterizing dynamic properties is central task. General conditions for exponential stability of such systems have been fully discussed in prior work of others. In this paper we build practical (computable) error bound analysis of the stochastic gradient algorithm when the loss function is time-dependent and quadratic in the parameters, as arising from standard linear regression model. The long term goal is to address this problem in general nonlinear models. This paper is the first step towards this aim.

Keywords

asymptotic stability; gradient methods; least mean squares methods; linear systems; nonlinear control systems; regression analysis; stochastic processes; time-varying systems; dynamic control system; error bound analysis; exponential stability; least-mean-squares algorithm; linear regression model; loss function; nonlinear model; stochastic gradient algorithm; time-varying formulation; Adaptation models; Algorithm design and analysis; Covariance matrices; Heuristic algorithms; Indexes; Least squares approximations; Stochastic processes; Adaptive Control; Constant Gain; LMS; Least-Mean-Squares Algorithm;

fLanguage

English

Publisher

ieee

Conference_Titel

Information Sciences and Systems (CISS), 2015 49th Annual Conference on

Conference_Location

Baltimore, MD

Type

conf

DOI

10.1109/CISS.2015.7086870

Filename

7086870