Approaching optimality in spatially scalable video coding: From resampling and prediction to quantization and entropy coding

This paper builds on our recent work on optimal prediction in spatially scalable video coding, and is inspired by earlier work in our lab on optimal approaches for quality (or SNR) scalability. The approach we propose herein complements the optimal enhancement-layer prediction, enabled by transform domain resampling that ensures the base layer information is maximally accessible and usable at the enhancement layer despite their differing signal resolutions, with an optimal approach to quantization and entropy coding that exploits all available information, encapsulated in the appropriate conditional distribution for transform coefficients, to yield a unified coding engine for spatial scalability. For such quantizers to fully exploit base layer information, the enhancement layer transform block size must proportionally match the signal block transformed at the base layer. The overall system incorporates switching that applies the full estimation-theoretic quantizer and entropy coder at the right block size, but may optionally employ other block sizes where it defaults to optimal prediction followed by standard quantization. It is experimentally shown that the proposed scheme provides considerable performance gains over conventional codec and other leading competitors.