Design of binary bandlimited signals imbedded in colored Gaussian noise and intersymbol interference

This paper considers the design of binary bandlimited receiver signals as well as the design of the receiver filter for signal immunity to intersymbol interference and additive colored Gaussian noise. The design problem is treated in the frequency domain. The maximum probability of error is minimized by choosing signals with zero intersymbol interference. When transmission is below the Nyquist rate, bandlimited signals with gradual cutoff are designed. Other waveforms, with intersymbol interference but possessing no sequences of unbounded amplitude, are also investigated. Examples are given demonstrating the application of the theory.