Causal source coding of stationary sources with high resolution

Neuhoff and Gilbert (1982) defined a causal lossy source code as a system where the reconstruction of the present source sample is restricted to be a function of the present and past source samples, while the code stream itself may be non-causal and have variable rate. They showed that for stationary and memoryless sources, optimum causal source coding is achieved by time-sharing at most two entropy coded scalar quantizers. We extend this result to general real valued stationary sources with finite differential entropy rate, in the limit of small distortions. We show that for the mean square distortion, the optimum causal encoder at high resolution is a fixed uniform quantizer followed by a sequence entropy coder. Thus, the cost of causality is the “space filling loss” of the uniform quantizer, i.e., (1/2)log(2πe/12)≈0.254 bit. This generalizes the well known result of Gish and Pierce on asymptotically optimal entropy constrained scalar quantization