On the probability of symbol error for two-dimensional signal constellations with non-uniform decision regions

Computing the probability of symbol error for two-dimensional (2-D) signal constellations where the decision regions have arbitrary shapes is cumbersome as it involves integral expressions of special functions. In this paper we present closed-form expressions that can be used to evaluate the probability of symbol error for such signal constellations in additive white Gaussian noise (AWGN) channels. We show that, for 2-D signal constellations, the received signal magnitude has a Rice distribution, and use this property to calculate the symbol error probability and to evaluate it numerically using either the Marcum Q-function or an approximate expression for the Bessel function. The proposed approach is illustrated on two specific examples of nonuniform constellations, and the resulting symbol error probabilities are compared to the corresponding union bound values.