Optimum Detection of Quantized PAM Data Signals

The degree of complexity of a digital signal processor is closely related to the precision with which samples of an incoming analog waveform are represented. There is considerable interest in determining how coarse this representation can be without seriously degrading performance from that of an ideal processor of unquantized samples. This question is examined for a receiver of noisy, linearly distorted pulse amplitude modulation (PAM) signals. An optimum [maximum likelihood (ML)] detector, analogous to the Viterbi detector for unquantized samples, is derived for the case of a quantized sample sequence. Performance is evaluated under the assumption of high signal-to-noise ratio (SNR), and the resultant error probability is a good approximation for coarse quantization, and an upper bound for any degree of quantization. For a specified error probability, the degree of quantization suggested by this approach is conservative. Since receiver complexity is closely associated with the length of the digital representation of an input sample, an upper bound on receiver complexity is also suggested. Numerical evaluation of the error probability is quite tedious for an arbitrary channel; however, system performance may be readily evaluated for partial-response (PR) signaling. For the PR channels