Human face modeling for facial image synthesis using optimization-based adaptation

This paper presents a novel optimization-based adaptation method for efficient reconstruction of human facial models for animation from range scan data. A prototype model with layered anatomical structure of the skin and muscles serves as the starting point for our adaptation algorithm. Based on a series of measurements between a subset of sparse anthropometric landmarks specified on the prototype model and scan data, an automated global adaptation adapts the size, position, and orientation of the prototype model to align it with the scanned surface. A local adaptation is then carried out to deform the skin mesh of the prototype model to fit all of its vertices to the scan data. We formulate an optimization problem to solve for local adaptation by minimizing an energy function that is a weighted combination of three measures: proximity of transformed vertices to the scan data, surface smoothness, and proximity of facial features at corresponding locations on the prototype model and scan data. On the adapted model, the underlying muscle structure is transferred, such that the model can be animated immediately to synthesize various facial expressions using the same muscle parameters.