Face pose estimation based on 3D detection of pupils and nostrils

Face orientation was determined by measuring 3D positions of pupils and nostrils using the stereo camera method. The two cameras were set at relatively low positions in order to grab the nostril image of a user. Two sets of several LEDs were arranged around the apertures of the two cameras, respectively. The two sets of the light sources were lit alternately and synchronously with the odd/even signal. This illumination condition alternately produced so-called bright and dark pupils in both camera images every field. Differentiation of the bright and dark pupil images in each camera image made it easy to detect the pupils by image processing. A large window was applied to a possible position of the nostrils, which were estimated by the two detected pupils. When the face was illuminated by the light source attached to another camera each other, the nostrils were darker than the other parts in the window. Accordingly, the nostrils can be detected easily. However, when the horizontal angle of the face is large, one nostril may be undetected. So the position of the undetected nostril was estimated using the ratio of the distance between the two nostrils to the distance between the two pupils. The ratio was obtained from an initial calibration procedure. The 3D positions of the two pupils and two nostrils were calculated by stereo matching. A composition vector of the normal vectors of the plane including the two pupils and the left nostril and the plane including the two pupils and the right nostril were defined as the face orientation. The experimental results showed that for all of three subjects head orientation detection was possible for a horizontal large angle of ±45 deg and for a vertical angel of ±15 deg.