Video Coding with Spatio-Temporal Texture Synthesis

This paper presents a video coding scheme in which some texture regions are selectively removed at the encoder and recovered by synthesis at the decoder. We present region removal utilizing conventional block-based motion information rather than global motion field. Removed regions including their motion information are not coded at the encoder. We propose spatio-temporal patch-searching in texture synthesis at the decoder to recover the removed regions. Our approach is not content based and is flexible and generic to be implemented. The scheme has been integrated into H.264/AVC and achieves up to 38.8% bitrate saving at similar visual quality levels compared with H.264/AVC.