Improved cumulant-based methods for direction finding with mutual coupling effect

We have recently proposed a fourth-order cumulant (FOC)-based method for direction-of-arrival (DOA) estimation in uniform linear arrays (ULAs) with mutual coupling. However, it is found that this method has limited performance in terms of the root-mean-square error (RMSE) in the case where the mutual coupling effect is not significant (but not ignorable). Motivated by this fact, two improved methods are presented in this paper. According to the properties of cumulants, a FOC matrix of the received data is firstly introduced. An improved DOA estimator based on the shift-invariant property as in the ESPRIT method is then developed. To further improve the performance, the other improved method that exploits the matrix rank-reduction property is finally proposed. Although grid search is required in the second improved method, the search space is restricted to a substantially reduced size since the coarse DOA estimates are provided by the first improved one. Simulation results show that the RMSE of DOA estimation can be reduced by means of the presented methods.