Asymptotic error performance of modulation codes in the presence of severe intersymbol interference

The minimum distance and hence the asymptotic error probability of trellis modulation codes that have been subjected to severe channel filtering are evaluated. A general algorithm is proposed for this purpose and applied to one class of coded modulations. Using a frequency-domain interpretation of the Euclidean distance, it is shown that under extreme channel band limitations, the error event that causes the minimum distance is characterized by a spectral null at DC. Numerical results show that for filtered continuous-phase-modulation codes, decreasing the modulation index or smoothing the frequency pulse does not improve the error performance. For some of these modulations, channel filtering can actually improve the minimum distance. Bandpass and low-pass filtering are contrasted. The numerical results are explained by power spectrum considerations