Pose estimation and conversion to front viewing facial image using 3D head model

Surveillance cameras are usually mounted near ceiling and pointing downward at an angle. Face images acquired usually are not frontal instead of faces looking downward or sideways. However, the face images collected in databases are frontal face images posing a problem of face recognition. Using skin color and shape analysis to detect the face, eyes and mouth then location of the head determined in 3D space based on perspective projection. Face parts not visible by the camera including parts facing away from the camera and that are obstructed by other face parts, that can be estimated from the angle between the surface normal and the vector pointing toward the camera. For converting to front viewing, two camera calibration matrices were established, one for physical camera situated right in front of the detected head and the other for virtual camera. By using calibration matrices location of the projection of a point in the 3D space on the image planes determined. For each face part (a triangle) that is visible by both cameras, we find its image in the face seen by the physical camera, apply affine transform to change its shape and size, and paste it onto its location seen by the virtual camera to accomplish pose conversion to a front viewing face. We verified that for a tilt angle (looking downward) ranging from 0 to 40 degrees, and/or a slant angle (face turning left or right) ranging from -60 degrees to +60 degrees, our approach is able to convert non-front viewing face to a front viewing face.