Convergence analysis of finite alphabet beamformers for digital cochannel signals

In this paper, we examine the convergence behavior of finite alphabet (FA) beamformers. The two most popular implementations of FA beamformers for digital communication signals are the iterative least-squares with projection (ILSP) and the minimum mean-square error (MMSE) beamformer. To facilitate the analysis, for the binary communications case, we derive closed-form expressions for the mean weight vector, signal-to-noise ratio, and signal-to-interference ratio for both the ILSP and MMSE beamformers in terms of the bit-error rate (BER) performance at each iteration. Next, we generalize the analysis for the M-ary pulse-amplitude modulation and M-ary phase-shift keying cases. We show that both ILSP and MMSE beamformers have previously unreported bias terms in the array response vectors which are functions of the BER for each iteration. Furthermore, as the BER becomes arbitrarily small, we show that our solutions converge to the well-known asymptotic expressions widely published in the literature. Next, we provide a geometric interpretation of the effects of the noise bias vector in terms of angles between subspaces. Based on our analysis, we were able to develop necessary and sufficient conditions for convergence in the mean. We conclude with Monte-Carlo simulations to validate our analysis.