On the use of tentative decisions to cancel intersymbol interference and nonlinear distortion (with application to magnetic recording channels)

We develop a theory of the error rate of receivers that use tentative decisions for adaptive cancellation of intersymbol interference (ISI). We formulate precise conditions under which tentative decisions can be effectively used to cancel linear or nonlinear ISI, and verify the predictions of the theory by Monte Carlo simulations on four typical channels. In the case of magnetic recording channels, we show that the conditions for effective cancellation are satisfied by the precursor interference generated by inductive read heads, resulting in an improvement of the noise margin of 2 dB at the output of a symbol-by-symbol detector or 1 dB at the output of a Viterbi decoder at a bit error rate of 10^-9. This technique has been incorporated in an experimental VLSI partial response maximum-likelihood (PRML) receiver, and the results confirmed by laboratory measurements. We also show that no improvement is obtained when this technique is used to compensate nonlinear distortion, which is another common impairment of magnetic recording channels, and we establish fundamental limits for the improvement achievable by any nonlinear equalization technique in magnetic recording