Analysis of Optimum Detector of Trellis Coded MPSK in Phase Noise Channels

This paper presents a novel analytical expression which approximates the bit error rate (BER) of the joint phase and symbol maximum a posteriori (JMAP) sequence estimator for: trellis code modulation (TCM) with M-ary phase shift keying (MPSK) modulation; any arbitrary phase noise model (i.e., not limited to the Wiener process); and either a matched or mismatched decoder. First, we derived convenient closed-form expressions for approximating the pairwise error probability of two code sequences for both matched and mismatched decoders. Since the expressions are formulated either in the time or in the frequency domain, it is possible to indicate the contribution of every frequency in the phase noise spectrum. We then applied the union bound on the code sequence pairwise errors. The analytical expression was tight (usually <;1dB) for MPSK constellations with M ≥ 4 and code rates ≥ 0.5. Once developed, the expressions will assist designers to consider the influence on receiver performance of the code characteristics, decoder implementation and RF synthesizer phase noise. It further enables joint optimization of the RF synthesizer, the code and the decoder for achieving the lowest error rate or other design targets.