A local region based approach to lip tracking

Lip tracking has played a significant role in a lip reading system. In this paper, we present a local region based approach to lip tracking, which consists of two phases: (i) lip contour extraction for the first lip frame, and followed by (ii) lip tracking in the subsequent lip frames. Initially, we construct a localized color active color model provided that the foreground and background regions around the object are locally different in color space. In the first phase, we find a combined semi-ellipse around the lip as the initial evolving curve and compute the localized energies for curve evolution such that the lip image is separated into lip and non-lip regions. Then, we utilize a 16-point deformable model (Wang et al., 2004 [20]) with geometric constraint to achieve lip contour extraction. In the second phase, we present a dynamic selection of the radius of local regions associated with the extracted lip contour of the previous frame to realize lip tracking. The proposed approach not only adapts to the lip movement, but it is also robust against the appearance of teeth, tongue and black hole. Extensive experiments show the efficiency of the proposed lip tracking algorithm in comparison with the existing methods.