Joint rate-distortion optimization of transform coefficients for spatial Scalable Video Coding using SVC

The recently finalized scalable video coding (SVC) extension of the H.264/AVC video coding standard supports encoding of video sequences at different spatial resolutions within one single scalable bit-stream. In order to achieve an improved coding efficiency compared to simulcast, SVC allows so-called "inter-layer prediction," where texture, motion, and residual data of the lower resolution layer are exploited for the encoding of the higher resolution layer. In this paper, we extend the ideas presented in our previous work by M. Winken et al (2007), and describe an approach how the dependencies caused by the inter-layer residual prediction can be utilized in the selection of the transform coefficient values for the lower resolution layer. Our experimental results show gains of 0.5 dB luma PSNR compared to our previous approach for the high resolution layer with no impact on the base layer coding efficiency.