System Performance of High-Order Optical DPSK and Star QAM Modulation for Direct Detection Analyzed by Semi-Analytical BER Estimation

Recently, higher order modulation formats are intensively investigated to further increase spectral efficiency for building next generation optical transport networks. Direct detection receivers are capable of detecting arbitrary modulation formats with differentially encoded phases such as differentially phase shift keying (DPSK) and star shaped quadrature amplitude modulation (Star QAM) formats. In an earlier publication of the authors (M. Seimetz , ??Optical systems with high-order DPSK and star QAM modulation based on interferometric direct detection,?? J. Lightw. Technol. , vol. 25, no. 6, pp. 1515-1530, Jun. 2007) a system analysis was performed where performance was mainly characterized by eye opening penalties. Here, these investigations are extended. A tool for semi-analytical BER estimation is developed allowing to calculate the BER down to small values such as 10^-9 for a wide range of modulation formats, as well as for different receiver structures. Using this module, the back-to-back OSNR requirements are calculated. CD and SPM tolerances are characterized by optical signal-to-noise ratio (OSNR) penalties at BER = 10^-9 . As far as the authors know, this is the first analysis of the transmission characteristics of optical 16DPSK and Star 16QAM based on the BER. Moreover, further novel aspects such as optimization of the optical and electrical receiver filter bandwidths are investigated.