Real time notch based face detection, tracking and facial feature localization

This research paper deals with the real time implementation of face detection, tracking and facial feature localization in video sequence that is invariant to scale, translation, and (plusmn45) rotation transformation. The proposed system contains two parts, visual guidance and face/non-face classification. The visual guidance phase detects the skin region using the cue present in monocular video (motion and color). Fusion of motion and color cues is done to classify skin color. The classified skin color is passed to morphological operations (recurring erosion and dilation) to remove the noise and holes from the classified skin regions. The contiguous regions are extracted using union-structure component labeling algorithm and scale normalization is applied by nearest neighbor interpolation method. Using the aspect ratio of width and height size, region of interest (ROI) is obtained and then passed to face/non-face classifier. Notch (Gaussian) based templates are used to find circular darker regions in ROI keeping the fact in view that face texture is coarser in nature due to eyes, lips and nostril darker portions. Various size of notch based templates are matched with ROI using normalized cross correlation (NCC), to get matching score (for the classification of input region as face/non-face). The visual guidance reduces the search space and helps to detect and track skin region whereas each region is categorized as face/non-face using face/non-face classifier. The classified face region is handed over to facial feature localization phase, which uses YCbCr eyes/lips mask for face feature localization