A Real-Time Virtual Environment Rendering System

In this paper, we describe a rendering system that is capable of capturing and rendering dynamic environments at about 20 fps using a commercially available PC (1.8 GHz single processor) and IEEE 1394 cameras (each with resolution of 320times240). It does not rely on any additional custom hardware. Such a system permits streaming applications that involve real-time view synthesis from multiple cameras, such as 3D teleconferencing or chatting. View synthesis is performed using the closest two cameras, and relies on plane sweeping to produce a photoconsistent virtual image. In addition, an alpha matte is estimated at the virtual view in order to improve the rendering quality at the foreground/background boundaries. We illustrate the performance of our system on several dynamic environment examples.