The joint optimization of transmitter and receiver filters for data transmission systems using mean field annealing

A technique is presented for finding transmitter and receiver filters for a wide class of data transmission systems which minimize the bit error rate (BER). The techniques uses Monte Carlo simulation to estimate the BER, coupled with mean field annealing (MFA) to optimize the pulse shapes. Modeling of the link can be as complex as simulation will allow, while MFA is resistant to the statistical scatter in the BER estimate from the simulation. In a complex example, MFA is coupled with Monte Carlo simulation techniques to find near optimal transmit and receive filters for a satellite communications link, taking 6 CPU hours on a DECstation 3100. The bit error rate of the link is found to be as much as three orders of magnitude lower when using the MFA-constructed optimal filters than when using filters constructed by gradient descent and simulated annealing