Analytical bounds on least squares and total least squares methods for the linear prediction problem

The least squares (LS) and total least squares (TLS) methods are commonly used to solve the linear prediction equations in frequency estimation problems. The authors examine how the noise, increasing the number of equations, or augmenting the system may reduce the sensitivity of the noise subspace, and thus provide improved estimation of the polynomial coefficients. Specifically, they provide an analytical lower and new upper bound for the difference between the LS and TLS solutions, which explains their similarities/differences in a high/low SNR environment. Numerical simulation results show that the bounds are sharp. The analysis is intimately linked to the concept of the subspace angle in perturbation theory for the orthogonal projection methods.<>