Pseudo roots and the effects of model order overestimation on the ESPRIT algorithm

The work studies the distribution of the ESPRIT roots in the complex z plane. The real source roots are tightly clustered about their true values, while the pseudo roots corresponding to overestimated non-existing sources are distributed broadly with a definite phase angle bias away from the real source roots. The ESPRIT roots distributions can be coupled with Neyman-Pearson tests for CFAR source detection/system model order estimation. The work includes a mathematical derivation of the asymptotic distribution of the pseudo roots