On the Threshold Region Mean-Squared Error Performance of Maximum-Likelihood Direction-of Arrival Estimation in the Presence of Signal Model Mismatch

The mean squared error (MSE) performance prediction of maximum-likelihood (ML) direction-of-arrival (DOA) angle estimation has been studied extensively. Previous analyses consider Cramer-Rao Bounds, sensitivity/asymptotic [in signal-to-colored noise ratio (SNR)] local error performance prediction that includes the impact of finite samples effects and additive signal modeling errors (mismatch), and prediction of the low SNR threshold region performance of ML DOA (without mismatch). Analysis of the adaptive array ML DOA (without mismatch) scenario has also been considered. The goals of this present analysis include the following: (i) to extend prediction of the asymptotic and threshold region MSE performance of ML to include a general form of deterministic signal model mismatch, and (ii) to begin looking at the threshold region performance of ML DOA estimation from an information-theoretic perspective, (iii) to determine if the classic work of Huber on model misspecification, although primarily asymptotic in nature, provide new insights into this finite sample problem. This initial work will focus on the DOA estimation of a single deterministic planewave signal in known colored noise and brief consideration will be given to the more complex scenario of an adaptive array in which the colored noise covariance must be estimated