Optimum linear filtering of analog signals in noisy channels

Channel noise in analog communication systems can be minimized by the use of linear filters which modify the spectrums of the transmitted and received modulation signals. This technique has been known for many years by engineers who design communication and recording equipment. However, their solution to the problem has been along empirical lines, based on improper criteria and procedures. Generally, the existence of an analytic solution has not been appreciated. Although the basic solution can be expressed simply, the result is not intuitively obvious, suggesting that seat-of-the-pants methods may have been significantly less than optimum. In this paper expressions are derived for five classes of optimum filtering. Although some equivalent formulas have been obtained in earlier papers according to the standard criterion of minimizing the mean-square error, it is claimed that this criterion is not germane in a communication sense. Particular attention is given to the reciprocal-filter design, which has the unique property of being expressed independent of the SNR. In transmitting 8-kHz speech, such filtering is found to make a 5-dB improvement against "flat" noise and a 15-dB improvement against "FM" noise. Applicability of the results to a peak- as well as an average-power limitation is discussed. Among the results is a new design formula for the case where several physical effects limit the modulation waveform (such as in disc recording).