Performance analysis of space-time block codes over keyhole Nakagami-m fading channels

In multiple-input-multiple-output (MIMO) fading environments, degenerate channel phenomena, called keyholes or pinholes, may exist under the realistic assumption that the spatial fading is uncorrelated at the transmitter and the receiver, but the channel has a rank-deficient transfer matrix. In this paper, we analyze the exact average symbol error rate (SER) of orthogonal space-time block codes (STBCs) with M-PSK and M-QAM constellations over Nakagami-m fading channels in the presence of the keyhole. We derive the moment generating function (MGF) of instantaneous signal-to-noise ratio (SNR) after space-time block decoding (signal combining) in such channels. Using a well-known MGF-based analysis approach, we express the average SER of the STBC in the form of single finite-range integrals whose integrand contains only the derived MGF. Numerical results show that the keyhole significantly degrades the SER performance of the STBC from idealistic behaviors in independent identically distributed MIMO channels.