ISI-Free Cochannel Interference Whitening for Bandlimited Fading Channels

The problem of cochannel interference mitigation using interference-plus-noise whitening receiver design in the presence of intersymbol interference (ISI) is considered. The effect of ISI on the signal-to-interference-plus-noise ratio (SINR) of the interference whitening receiver is examined. Then, two methods are proposed to maximize the SINR without introducing ISI. In the first method, the transmitter and receiver filters are designed to maximize the SINR while their overall spectrum maintains a given Nyquist spectrum to avoid ISI. In the second method, the transmitter filter is assumed to be fixed and only the receiver filter is designed to achieve maximum SINR without introducing ISI. The SINR of the ISI-free SINR-maximizing filter is then analytically compared with that of the conventional matched filter receiver and the interference whitening receiver. Numerical results are presented for the cases when standard raised-cosine and Beaulieu-Tan-Damen pulses are used in the system.