Comparison of the characteristics of linear least squares and orthonormal expansion in estimation

It is frequently assumed in signal processing applications that expansion of a sequence by a weighted sum of mutually orthonormal sequences yields weighting coefficients that are identical to the estimates of the parameters which maximize the likelihood function if the linear sum of sequences is chosen as a model. Although this may be a valid approximation when signal-to-noise ratios are large, it is not generally the case and may lead to erroneous results when substantial noise exists. This paper explores the relationship between orthonormal expansion and linear least squares estimation. In doing so, the conditions under which orthonormal expansion coefficients are maximum likelihood estimates are identified. Several interesting properties related to both techniques are also revealed. The results are relevant to a wide range of signal processing applications such as the discrete Fourier transform and linear prediction theory and can be extended to non-linear least squares estimation. This should make the results of interest to those involved with the analysis of noisy data.