Transmitter design for data transmission in the presence of a data-like interferer

In many digital communications systems, crosstalk, rather than additive noise, is the primary channel impairment. For such systems, it is known that the spectral support of the optimum transmitter is not, in general, restricted to a Nyquist set, in contrast to the case for the additive-noise channel. Nevertheless, the problem of determining the optimum transmitter shaping function for the crosstalk channel without the Nyquist restriction is a difficult one, and has so far remained unsolved. Motivated by current interest in the high-speed digital subscriber line (HDSL) and related crosstalk-dominated applications, we explore a subcase of this problem in which only a single interferer is present. When applied to HDSL-like systems with a single (or dominant) interferer, our analysis and numerical results confirm that wider-than-Nyquist transmitters provide a large performance advantage over Nyquist-limited transmitters. Several interesting and counter-intuitive results also arise. For example, PAM and QAM systems operating at the same spectral efficiency do not, in general, perform identically over the crosstalk channel, despite their essential equivalence in additive noise. We explain why this is so, and show that for channels qualitatively similar to the HDSL wire-pair, QAM has a significant advantage over PAM at high data rates. Finally, we show how the characteristics of HDSL-like channels can be exploited by optimizing the symbol rate.<>