The Averaged, Overdetermined, and Generalized LMS Algorithm

This paper provides and exploits one possible formal framework in which to compare and contrast the two most important families of adaptive algorithms: the least-mean square (LMS) family and the recursive least squares (RLS) family. Existing and well-known algorithms, belonging to any of these two families, like the LMS algorithm and the RLS algorithm, have a natural position within the proposed formal framework. The proposed formal framework also includes - among others - the LMS/overdetermined recursive instrumental variable (ORIV) algorithm and the generalized LMS (GLMS) algorithm, which is an instrumental variable (IV) enable LMS algorithm. Furthermore, this formal framework allows a straightforward derivation of new algorithms, with enhanced properties respect to the existing ones: specifically, the ORIV algorithm is exported to the LMS family, resulting in the derivation of the averaged, overdetermined, and generalized LMS (AOGLMS) algorithm, an overdetermined LMS algorithm able to work with an IV. The proposed AOGLMS algorithm overcomes - as we analytically show here - the limitations of GLMS and possesses a much lower computational burden than LMS/ORIV, being in this way a better alternative to both algorithms. Simulations verify the analysis.