Optimization of Two-Dimensional Signal Constellations in the Presence of Gaussian Noise

A considerable amount of literature exists on the problem of selecting an efficient set of digital signals with in-phase and quadrature components for use in a suppressed carrier data transmission system. However, the signal constellation which minimizes the probability of error in Gaussian noise, under an average power constraint, has not been determined when the number of signals is greater than two. In this paper an asymptotic (large signal-to-noise ratio) expression, of the minimum distance type, is derived for the error rate. Using this expression, a gradient-search procedure, which is initiated from several randomly chosen -point arrays, converges in each case to a locally optimum constellation. The algorithm incorporates a radial contraction technique to meet the average signal power constraint. The best solutions are described for several values of and compared with well-known signal formats. As an example, the best locally optimum 16-point constellation shows an advantage of about 0.5 dB in signal-signal-to-noise ratio over quadrature amplitude modulation. The locally optimum constellations are the vertices of a trellis of (almost) equilateral triangles. As , it is rigorously proved in the Appendix that the optimum constellations tend toward an equilateral structure, and become uniformly distributed in a circle.