Error probability of reduced-state sequence estimation for trellis-coded modulation on intersymbol interference channels

The error probability of reduced-state sequence estimation (RSSE) for trellis-coded modulation on intersymbol interference channels is evaluated. In the literature, the minimum distance of ideal RSSE (without error propagation) has been used for approximating the error probability of real RSSE, but this approximation can be very poor. A union bound on the error probability for ideal RSSE is shown to provide a much better approximation than the minimum distance. A method based on an error super-state diagram and a stack algorithm is proposed for calculating the union bound