Depth image based compositing for stereo 3D

Compositing is an essential tool in modern film making. Established workflows exist for single-view video compositing, however new problems that demand new solutions arise when considering stereoscopic compositing. We investigate new methods for compositing live action stereo 3D, given two stereo camera systems and a corresponding depth map. We break the process up into three main steps; First, we show how to use trifocal tensors to robustly project 3D content from one video into another. Second, we analyze different image-based rendering methods for drawing objects to be composited into the new video. Finally, we describe a novel super-pixel based depth-test that increases robustness to errors in depth map accuracy. A user study was conducted to validate different steps of this process.