Optimization of transmitter and receiver filters for digital communication systems using mean field annealing

A technique for finding transmitter and receiver filters for a wide class of digital communication systems which minimize the bit-error rate (BER) is presented. The technique uses Monte Carlo simulation to estimate the BER and mean field annealing (MFA) to optimize the pulse shapes. Modeling of the link can be as complex as simulation will allow, while MFA is resistant to the statistical variation in the BER estimate from the simulation. Initially, the MFA technique was applied to a binary symmetric channel in a nonsimulation environment, and an approximate analysis of the behavior of MFA for this problem was performed. In a more complex example, MFA was coupled with Monte Carlo simulation techniques to find near-optimal transmit and receive filters for a satellite communications link, taking 6 CPU hours on a DECstation 3100. The BER of the link was found to be as much as three orders of magnitude lower when using the MFA-constructed optimal filters than when using filters from other comparison results. For this example, the pulse shapes obtained using MFA exhibit a low BER even as the parameter controlling the nonlinearity of the satellite-link model is varied over a wide range, thus showing the solution is robust