Reanimating Faces in Images and Video

This paper presents a method for photo-realistic animation that can be applied to any face shown in a single image or a video. The technique does not require example data of the person’s mouth movements, and the image to be animated is not restricted in pose or illumination. Video reanimation allows for head rotations and speech in the original sequence, but neither of these motions is required. In order to animate novel faces, the system transfers mouth movements and expressions across individuals, based on a common representation of different faces and facial expressions in a vector space of 3D shapes and textures. This space is computed from 3D scans of neutral faces, and scans of facial expressions. The 3D model’s versatility with respect to pose and illumination is conveyed to photo-realistic image and video processing by a framework of analysis and synthesis algorithms: The system automatically estimates 3D shape and all relevant rendering parameters, such as pose, from single images. In video, head pose and mouth movements are tracked automatically. Reanimated with new mouth movements, the 3D face is rendered into the original images