Optimal criterion of decision feedback equaliser coefficients in optical communication system

Decision feedback equaliser (DFE) for dispersion compensation is a cost-effective element in optical communication OOK systems. The most common but sub-optimal optimisation criterion for the DFE weights is based on minimising the mean square error (MSE). Here, we derive an analytical expression for the probability of error with DFE and examine the optimal optimisation criterion, minimum bit-error rate (MBER) for DFE in optical communication system. A BER performance comparison analysis shows that the MBER optimisation criterion introduces improvements over the conventional minimum MSE optimisation criterion.