Optimality in the choice of the convergence factor for gradient-based adaptive algorithms

The convergence and the adaptation speed of gradient-based adaptive algorithms are controlled by the chosen value for the convergence factor μ. In this paper, the existence of an optimal value for this convergence factor is investigated for two classes of algorithms. A proof is first presented for the general case of the complex adaptive-linear-combiner (ALC). The results are applied to the complex and real LMS algorithms. This is followed by a second proof for algorithms which are linear only in a subset of their adaptive coefficients. These cases are found in IIR applications such as the hybrid-recursive, lattice-recursive, and recursive algorithms using the direct realization IIR. For each case, the optimal value is shown to be generated using instantaneous signal estimates. The resulting adaptive algorithms become self-optimizing in terms of their convergence factor, and dependence on incoming training signal levels is reduced. Moreover, a correction factor is introduced in each case to regulate the adaptation process and accommodate practical applications where additive signals are present with the desired signal.