Interpretation and convergence speed of two recently introduced adaptive filters (FEDS/RAMP)

Fast Euclidean direction search (FEDS) (T. Bose et al., 2002) and recursive adaptive matching pursuit (RAMP) (J.H. Husoy, 2003) are two recently introduced algorithms for adaptive filtering characterized by low computational complexity, good convergence, and numerical robustness. While conceived from quite different perspectives, we point out in the first part of the paper, that both algorithms are closely related and can be interpreted as different variants of 1) a matching pursuit procedure applied to a particular over determined equation set, 2) a constrained least squares (LS) optimization problem, and 3) a Gauss-Seidel like iterative solution procedure applied to a normal equation. Both FEDS and RAMP have been demonstrated experimentally to possess excellent convergence behavior in several application scenarios. However, a tool for predicting the convergence of these algorithms based on second order statistics is lacking. In the second part of the paper such a tool is presented. Finally, we present simulation results showing good agreement between predicted and actual learning curves for both FEDS and RAMP.