A texture replacement method at the encoder for bit-rate reduction of compressed video

We propose a method for texture replacement in video sequences. Our method, which is applied at the encoder side, consists of removal of texture from selected regions of the original frames, synthesis of new texture, and mapping of the new texture back onto the segmented regions. The texture removal stage employs highly effective color-based angular maps. The texture analysis and texture synthesis stages make use of steerable pyramids. The latter stage also employs constraints that are derived using a vocabulary and grammar for color pattern similarity evaluation that have been introduced previously. Because they have different characteristics than those of the original textures, the synthesized textures can be coded more effectively. Consequently and most importantly, significantly reduced bit rates of the compressed video sequences with the texture replaced are obtained as compared to those of the original sequences. Moreover, because the synthesized textures have similar perceptual characteristics to those of the original textures, the video sequences with the texture replaced are also visually similar to the original sequences. Even more, because it is performed at the encoder and it does not have any impact on the decoder, our texture replacement method is cost effective. We illustrate its performance and computational efficiency using movie sequences.