Constrained min-max optimization of finite-precision multi-rate digital filters over the canonical signed-digit coefficient space

This paper is concerned with the development of a new modified branch-and-bound technique for the constrained min-max optimization of multi-rate digital filter transfer functions over the canonical signed-digit (CSD) coefficient space. The proposed optimization technique employs a nonsmooth minimization approach for the underlying continuous optimization problems. The resulting optimization technique can handle the simultaneous optimization of the magnitude and group-delay frequency response characteristics. Moreover, it can handle both inequality and equality constraints on the magnitude and group-delay frequency response characteristics. In addition, it is guaranteed to converge to a solution (if a solution exists). Finally, in this technique, both the coefficient wordlength and the number of nonzero digits can be specified at the outset. The proposed optimization technique has been incorporated in a software package called Papillon 97. The effectiveness and practical application of Papillon 97 is demonstrated by applying it to the optimization of a digital CODEC filter transfer function