Optimal Modulation for Known Interference

We present a symbol-by-symbol approach to the problem of canceling known interference at the transmitter in a communication system. In the envisioned system, the modulator maps an information symbol (taken from a finite alphabet) and an interference symbol (from the complex field) onto a transmitted constellation point. The demodulator picks the information symbol (as a function of the received symbol) which minimizes the average error probability. We find the optimal modulator-demodulator pair, in the minimum-probability-of-symbol-error sense, via an iterative optimization procedure, for fixed average transmit power. We illustrate that the new scheme can perform close to the no-interference bound, and in particular that it outperforms Tomlinson-Harashima precoding, which is a classical but suboptimal solution to the problem under study.