Fixed-point error analysis of the lattice and the Schur algorithms for the autocorrelation method of linear prediction

Levinson recursions, Schur recursions, and lattice recursions provide three different ways of computing reflection coefficients for a stationary time series. Of these three approaches, only the Schur and lattice algorithms are scaled algorithms that can be implemented in fixed-point arithmetic. The authors study the finite arithmetic properties of the Schur and lattice recursions. They derive error variances for the reflection coefficients and present experimental results which agree very closely with the analytical results. They show analytically and experimentally that lattice recursions have a significant accuracy advantage over Schur recursions when the problem is ill-conditioned or, equivalently, when the absolute values of the reflection coefficients are close to one. This result is not surprising since the lattice recursions, which compute reflection coefficients by QR-factoring a Toeplitz data matrix, are `square-root´ recursions with respect to Schur recursions, which compute reflection coefficients by Cholesky-factoring a Toeplitz correlation matrix that is quadratic in the data