High Quality Binocular Facial Performance Capture from Partially Blurred Image Sequence

Existing methods on passive facial performance capture assume that the input images are well captured. They merely consider how to deal with motion blurred images in the input sequence, which is very common in the image capture process. This paper presents a collection of novel algorithms and a thereby resulting system to reconstruct high quality facial dynamic geometry even from a partially blurred image sequence. In our method, we adopt binocular cameras to capture a stereo sequence. With this sequence, we first estimate depth map for each frame using a state-of-the-art stereo matching method. Then, based on the estimated depth map sequence, we track facial motion by leveraging constraints of both optical flow and geometry. In this step, a blur detection and regularization algorithm are devised to adaptively keep both shape and details. Finally, we synthesize temporal mesoscopic geometry on the blurred region from clear image texture of neighboring frames. We conduct extensive experiments on several sequences containing facial performance with blurred region, and the results demonstrate the effectiveness and robustness of our algorithms.