Orthogonal-Blendshape-Based Editing System for Facial Motion Capture Data

In this work, we present a data-driven 3D facial motion capture editing system that uses the automated construction of an orthogonal-blendshape face model and constrained weight propagation. Given a collected facial motion capture data set, we start by performing a region-based principal component analysis (PCA) decomposition and constructing a truncated PCA space spanned by the largest eigenvector for each anatomical region of the human face ( for example, the left eyebrow). We then construct an orthogonal-blendshape face model as follows: each eigenvector of an anatomical region corresponds to a blendshape basis, so we regard its PCA coefficient as its blendshape weight. Our orthogonal model also minimizes the blendshape interference issue that can affect the efficiency of the overall approach.Finally, we transform each frame of a new facial motion capture sequence into blendshape weights by projecting it into the PCA spaces mentioned earlier on a per-region basis. As such, modifying the blendshape weights (PCA coefficients) is equivalent to editing the underlying motion capture sequence.