Bi-iterative least square versus bi-iterative singular value decomposition for subspace tracking

We first revisit the problem of optimal low-rank matrix approximation, from which a bi-iterative least square (Bi-LS) method is formulated. We then show that the Bi-LS method is a natural platform for developing subspace tracking algorithms. Comparing to the well known bi-iterative singular value decomposition (Bi-SVD) method, we demonstrate that the Bi-LS method leads to much simpler (and yet equally accurate) linear complexity algorithms for subspace tracking. This gain of simplicity is a surprising result while, as we show, the reason behind it is also surprisingly simple.