A fundamental approach to the convergence analysis of least squares algorithms

The literature dealing with the question of convergence of the least squares (L.S.) identification algorithm [1-10] is usually utilizing the properties of the sequential estimator, e.g. the fact that the sequence of estimates is a matringale process, if the noise is an independent sequence, has been used to establish convergence in [10]. In this paper emphasis is put on the fact that the least squares estimates are obtained by minimizing a (quadratic) cost functional. Convergence results for sequence of random variables obtained by minimizing a parameterized random sequence with respect to its parameter are presented. These results in turn are utilized to establish strong convergence (w.p.l and m.s.) of the L.S. procedure under milder conditions than those in previous proofs. Landau´s recursive algorithms [12-14] are shown to be variations of the L.S. and thus their convergence is also established. The self tuning regulator [19-22] is also discussed and the importance of the use of the L.S. procedure in it is demonstrated. The importance of this paper, beyond extending previous convergence results is in its approach - utilizing the foundation on which L.S. procedures are based.