Rate-distortion optimization for automatic sprite video coding using H.264/AVC

Sprite-based video coding offers higher compression efficiency than conventional block-based hybrid video coders. In sprite coding a sequence is divided into a model of its background, i.e. a so-called background sprite image, and a foreground object sequence. These are then encoded and transmitted to the receiver. At the decoder the output video sequence is synthesized using content previously transmitted. The background information of the video sequence is, other than the foreground object sequence, encoded indirectly, meaning for a sequence with N frames one single image is encoded and not N background frames. The background for a single frame is reconstructed by coordinate transformation at the decoder. An important issue to address here is an appropriate rate-distortion technique for optimization of the sprite-based coding approach, since foreground and background are encoded independently. In this paper, the Lagrangian cost function is considered for rate-constrained encoder control. Experimental evaluation shows the correct use of the optimization method and the gain in terms of rate-distortion performance compared to H.264/AVC.