Multi-View Face Tracking with Factorial and Switching HMM

Dynamic face pose change and noise make it difficult to track multi-view faces in a cluttering environment. In this paper, we propose a graphical model based method, which combines the factorial and the switching hidden Markov model (HMM). Our method integrates a generic face model with general tracking methods. Two sets of states, corresponding to appearance model and generic face model respectively, are factorized in the HMM. The measurements on different states are fused in a probabilistic framework to improve the tracking accuracy. To handle pose change, model switching mechanism is applied. The pose model with the highest probabilistic score is selected. Then pose angles are estimated from those pose models and propagated during tracking. The factorial and switching model allows to track small faces with frequent pose changes in a cluttering environment. A Monte Carlo method is applied to efficiently infer the face position, scale and pose simultaneously. Our experiments show improved robustness and good accuracy