Stereo head/face tracking and pose estimation

A real-time disparity- and intensity-based head/face tracking system is presented. It integrates stereo, watershed and gradient techniques to track the head/face. The head is tracked if the facial features are not available, e.g. when the person is away from the stereo head or when the head is in the profile view. The face and 3D pose are tracked once the facial features, e.g. eyebrows, eyes, mouth and nostrils, are available. A histogram-based disparity segmentation method is developed to locate the person of interest. A novel watershed-based method is proposed in which the head can be separated from the connected head-and shoulder component. The facial features are extracted within the facial region using a new method called "anti max-median filter". The face contour is refined using the features and the gradient model developed in, however the head in our approach is allowed to be in arbitrary direction. A commercially available stereo system was used to implement the head/face tracking. Our approach automatically initializes without user intervention and can be re-initialized automatically if tracking of the 3D face pose is lost. The results on our face database and real image video sequences illustrate the effectiveness of the method.