Modeling and analysis of vector-quantized M-channel subband codecs

This paper demonstrates that the scalar nonlinear gain-plus-additive noise quantization model can be used to represent each vector quantizer in an M-band subband codec. The validity and accuracy of this analytic model is confirmed by comparing the calculated model quantization errors with actual simulation of the optimum LBG vector quantizer. We compute the mean squared reconstruction error (MSE) which depends on N the number of entries in each codebook, k the length of each codeword, and on the filter bank coefficients. We form this MSE measure in terms of the equivalent scalar quantization model and find the optimum FIR filter coefficients for each channel in the M-band structure for a given bit rate, given filter length, and given input signal correlation model. Specific design examples are worked out for a 4-tap filter in a two-band paraunitary structure. Theoretical results are confirmed by extensive Monte Carlo simulation