Asymptotic MSE Distortion of Mismatched Uniform Scalar Quantization

Asymptotic formulas are derived for the mean-squared error (MSE) distortion of N-level uniform scalar quantizers designed to be MSE optimal for one density function, but applied to another, as N → ∞. These formulas, which are based on the Euler-Maclaurin formula, are then applied with generalized gamma, Bucklew-Gallagher, and Hui-Neuhoff density functions as the designed-for and applied-to densities. It is found that the mismatch between the designed-for and applied-to densities can disturb the delicate balance between granular and overload distortions in optimal quantization, with the result that, generally speaking, the granular or overload distortion dominates, respectively, depending on whether the applied-to density function has a lighter or heavier tail than the designed-for density. Specifically, in the case of generalized gamma densities, a variance mismatch makes overload distortion dominate for an applied-to source with a slightly larger variance, whereas a shape mismatch can tolerate a wider variance difference while retaining the dominance of the granular distortion. In addition, for the studied density functions, the Euler-Maclaurin approach is used to derive asymptotic formulas for the optimal quantizer step size in a simpler, more direct, way than previous approaches.