Super-Resolution Face View Synthesis using a Mobile Face Capture System

During face-to-face collaboration people frequently monitor the other´s facial expressions to determine their current state of attention, mood, and comprehension. Capturing a frontal view of the face of mobile users in multi-user collaborative environments has been a challenge for several years. A mobile social presence system has been proposed that captures two side views of the face simultaneously and generates a frontal view in real-time. The face is modeled using an active appearance model (AAM) and a mapping of the side model to the frontal model is constructed from training. Frontal views are then generated by applying this mapping to the fitted side model during collaboration. Only a few model coefficients are transmitted for the synthesized facial frames, providing a highly compressed stream. In this paper we present a performance analysis by evaluating the proposed system under limited resolution run-time conditions.