Optimization techniques for cosine-modulated filter banks based on the frequency-response masking approach

An optimization technique for designing cosine-modulated filter banks using the frequency-response masking approach is proposed. In the given method, we perform minimization in the least-squares and minimax senses, subject to direct and aliasing transfer-function constraints. For optimization, a quasi-Newton algorithm with line search is used, based on sequential quadratic programming. Simplified analytical expressions to impose the interference constraints are proposed. The results are lower levels of aliasing transfers for a pre-determined filter order, or reduced filter complexity for given levels of interference. Specifications that were unfeasible due to lengthy FIR filter requirements are now achieved using the FRM-CMFB design.