Long-Term Background Reconstruction with Camera in Motion

Obtaining a dynamically reconstructed background sprite image is an important and challenging task for video applications such as object-based video coding, tracking and video object segmentation. This paper proposes a novel system for video background reconstruction with camera in motion. Firstly, the proposed algorithm uses feature point pairs from current frame and a reference frame for reliable global motion estimation (GME) where a dynamic reference frame update strategy is utilized. Once the global motion transformation matrix is obtained, each frame is warped into a projective plane where a statistic-based background modeling algorithm is applied and the background sprite image is produced. Compared with conventional sprite generation methods, the proposed algorithm: (1) provides efficient and reliable GME using feature points; (2) significantly reduces the accumulated warping error by a dynamic reference frame update mechanism; (3) produces accurate background pixel reconstruction using a statistic analytical model. The experimental results are presented and analyzed to show the robustness and accuracy of the proposed system.