Performance Analysis of ML Estimation under Misspecified Numbers of Signals

The maximum likelihood (ML) approach for estimating direction of arrival (DOA) plays an important role in array processing. Its consistency and efficiency have been well established in literature. A common assumption is that the number of signals is known. In many applications, this information is not available and needs to be estimated. However, the estimated number of signals may not equal the true number of signals. Therefore, it is important to know whether the ML estimator provides any relevant information about the true parameters. In a previous study, the ML estimator was shown to converge to a well defined vector whose components coincide with the true parameters. In this work, we investigate the impact of model mismatch on estimation accuracy. Applying the theory of misspecified nonlinear regression models, we derive a compact formula for the asymptotic covariance matrix. Our analysis and simulation show that the variance increases when the number of signals is incorrectly chosen