On the symbol error probability of general order rectangular qam in nakagami-m fading

Recently, Beaulieu, following an ingenious concept, presented a closed-form expression for a useful integral, which was used for the evaluation of the symbol error probability (SEP) of general order rectangular quadrature amplitude modulation (QAM) in slow Rayleigh fading. In this letter, these results are extended to Nakagami-m fading channels, deriving a novel closed-form formula for the average over Nakagami-m fading of the product of two Gaussian Q-functions, which can be efficiently used to study the impact of fading severity on the error performance of general rectangular QAM constellations