Soft Metrics and Their Performance Analysis for Optimal Data Detection in the Presence of Strong Oscillator Phase Noise

In this paper, we address the classical problem of maximum-likelihood (ML) detection of data in the presence of random phase noise. We consider a system, where the random phase noise affecting the received signal is first compensated by a tracker/estimator. Then the phase error and its statistics are used for deriving the ML detector. Specifically, we derive an ML detector based on a Gaussian assumption for the phase error probability density function (PDF). Further without making any assumptions on the phase error PDF, we show that the actual ML detector can be reformulated as a weighted sum of central moments of the phase error PDF. We present a simple approximation of this new ML rule assuming that the phase error distribution is unknown. The ML detectors derived are also the aposteriori probabilities of the transmitted symbols, and are referred to as soft metrics. Then, using the detector developed based on Gaussian phase error assumption, we derive the symbol error probability (SEP) performance and error floor analytically for arbitrary constellations. Finally we compare SEP performance of the various detectors/metrics in this work and those from literature for different signal constellations, phase noise scenarios and SNR values.