Robust real-time 3D head tracking based on online illumination modeling and its application to face recognition

This paper investigates the estimation of 3D head poses and its identity authentication using a simple ellipsoid model. To achieve robust motion estimation even under time-varying lighting conditions, we incorporate illumination correction into the conventional 3D model-based tracking framework with a single camera. In addition, by computing the illumination bases online from the registered face images, after estimating the 3D head poses, user-specific illumination bases can be obtained, and therefore illumination-robust tracking without a prior learning process can be possible. Furthermore, our unified tracking is approximated as a linear least-squares problem; a closed-form solution is then provided. Therefore, it can be executed in real-time at 20 frames per second. After recovering full motion of the head, we can register face images with pose variations into stabilized (frontal) view images which are suitable for pose-robust face recognition. To verify the feasibility and applicability of our proposed 3D head-tracking framework, we performed extensive experiments with three sets of challenging image sequences.