Augmented reality with occlusion rendering using background-foreground segmentation and trifocal tensors

Over the past years, video communication has emerged as one of the most popular form of communication. Developing the technology for merging the real and virtual image streams and 3D models for better communication is an active research topic. In this paper, we propose a novel method of real time augmentation of 3D objects in 3D scenes using trifocal tensors. The augmentation is robust to arbitrary camera motion. The proposed technique does not require camera calibration, use of fiducials and prior knowledge of structural model of the scene. Also, using background-foreground segmentation, the occlusion issue in augmented reality is addressed. We show several results of the successful working of our algorithm in real life situations. The technique works on a real time video from a USB camera, creative Webcam III on a P IV 1.6 GHz system without any special hardware support.