Video editing using figure tracking and image-based rendering

We describe a new approach to video editing based on the semi-automatic segmentation of video into multiple layers and the composition of layers using image-based rendering. Using figure tracking and background motion estimation, we can segment a moving figure and reconstruct the background. Using geometrically-correct pixel reprojection, layers can be composited on the basis of the geometry of the underlying scene and the position of a virtual camera. We have implemented a prototype editing system called SpliceWorld.