Advanced 3D Video Object Synthesis Based on Trilinear Tensors

In this paper an advanced method to synthesize 3D video objects is presented. Firstly, the knowledge about the implicit scene geometry is determined by a sparse set of point correspondences. Additionally an accurate camera calibration based on Zhangs method and model is carried out. With the help of matching constraints obtained from epipolar geometry a dense disparity map is estimated using a modification of Birchfield and Tomasi´s pixel-to-pixel stereo algorithm. Using these information we synthesize novel views. The mapping method we employ is called trifocal transfer and based upon the usage of trilinear tensors introduced by Shashua and Avidan. Our synthesis method avoids the issue of indirect view specification and allows the user to move freely in space. The whole system chain is designed for future real-time TV applications. A modular software system we call ReVOGS (realistic video object generations system) is the base for high level applications. We conclude the paper with an outlook of future system improvements