Video-rate capture of dynamic face shape and appearance

This paper presents a system for simultaneous capture of video sequences of face shape and colour appearance. Shape capture uses a projected infrared structured light pattern together with stereo reconstruction to simultaneously acquire full resolution shape and colour image sequences at video rate. Displacement mapping techniques are introduced to represent dynamic face surface shape as a displacement video. This unifies the representation of face shape and colour. The displacement video representation enables efficient registration, integration and spatiotemporal analysis of captured face data. Results demonstrate that the system achieves video-rate (25 Hz) acquisition of dynamic 3D colour faces at PAL resolution with an rms accuracy of 0.2 mm and a visual quality comparable to the captured video.