Configuration and performance of modulated filter banks

The performance issues relating to the transmultiplexers previously synthesized by the authors (see Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing, Albuquerque, NM, April 1990) are examined. The transmultiplexers consist of modulated filter banks based on one or two low-pass prototypes. The limitations of the configured systems regarding intersymbol interference and crosstalk suppression arising from the use of practical filters are analyzed. Based on these observations, a design technique for FIR (finite-impulse response) prototypes that takes the practical degradations into account is formulated. The procedure involves the unconstrained optimization of an error function. The resulting performance is compared with minimax filters. For the one-prototype systems, the method is superior because it leads to better intersymbol interference and crosstalk suppression with a smaller number of filter taps. In the case of the transmultiplexer with two prototypes, the main advantage of the design method is the inclusion of crosstalk terms in the error function. It is noted that the five transmultiplexers can be converted into subband systems. How the design approach formulated for transmultiplexers carries over to the subband systems is shown