The effect of mismatch on decision-feedback equalization and Tomlinson-Harashima precoding

This paper examines the output SNR in decision-feedback equalizer (DFE) with channel mismatch at the receiver and under finite filter length constraints. A lower bound on the achievable rates for Tomlinson-Harashima (1971, 1972) precoding (THP) is also presented under mismatch and finite filter length constraints. With channel mismatch at both transmitter and receiver an asymptotic upper bound for a two-tap channel is derived when the input is i.i.d. uniform. Under finite filter length constraints, a four-tap maximum phase channel shows that the best solution is simply a linear equalizer (LE) for both DFE and THP systems. We derive the general conditions that a channel impulse response must satisfy to have LE as the optimal solution under a given finite length constraints and input SNR.