Optimal-filter approach for nonlinear power amplifier modeling and equalization

This paper presents an optimal-filter method for application to the equalization and modeling of nonlinear power amplifiers. The method uses auto-spectral and cross-spectral densities of measured amplifier input/output time-domain waveforms. An optimal filter response is calculated which best approximates a given nonlinearity as quantified by a new and useful linear coherence function metric. This response can in turn be used to determine optimal equalizer responses. It is also shown that by adding an optimal filter to the output of the standard two-box amplifier nonlinear model, waveform accuracy can be improved by 3.5 dB. The approach is applied to a 20-GHz traveling-wave tube amplifier using 9.6 Gbps 16-APK input signals.