High-presence, low-bandwidth, apparent 3D video-conferencing with a single camera

Small digital video cameras have become increasingly common, appearing on portable consumer devices such as cellular phones. The widespread use of video-conferencing, however, is limited in part by the lack of bandwidth available on such devices. Also, video-conferencing can produce feelings of discomfort in conversants due to a lack of co-presence. Current techniques to increase co-presence are not practical in the consumer market due to the costly and elaborate equipment required (such as stereoscopic displays and multicamera arrays). To address these issues, this paper describes a real-time, full frame-rate video-conferencing system that provides simulated three- dimensionality via motion parallax in order to achieve a higher level of co-presence. The system uses a deformable 3D face model to track and re-synthesize each user´s face using only a single monocular camera, so that only the (few tens of) parameters of the model need be transferred for each frame. This both provides motion- tracking for the simulated 3D experience and reduces the bandwidth requirements of the video-conference to the order of a few hundred bytes per frame. Bandwidth and processor usage for the algorithms are reported. Possible implications and applications of this technology are also discussed.