Estimation and noisy source coding

The minimum-mean-squared-error encoding of a noisy source is considered within the context of alphabet-constrained data compression. Just as in the block-coding formulation, the optimum source coder consists of an optimum estimator followed by optimum source coding of the resulting estimates. However, unlike the block approach, the alphabet-constrained viewpoint admits estimators based on the past history of source observations outside the current block of interest. If delayed encoding is allowed, the estimator is an optimum smoother. For Gauss-Markov sources, the encoding performance is characterized in terms of the estimation error covariance, and it is demonstrated that for moderate block sizes, significant reduction in mean-squared error can be achieved compared to the block coder performance. Extensive experiments are reported for vector quantization of noisy speech using the Y.L. Linde, A. Buzo, and R.M. Gray (IEEE Trans. Commun., vol.COM-28, p.84-95, 1980) training mode vector quantizer. The results indicate that the alphabet-constrained estimator, which is a Kalman filter, is superior to the block estimator, and, in particular, that adaptivity is critical for good performance over a variety of speech sources