HMM recognition of expressions in unrestrained video intervals

The paper discusses the application of a facial expression recognition system in unrestrained video intervals. The system is based on the modeling of expressions by means of hidden Markov models. The observations used to create the models are the MPEG-4 standardized facial animation parameters (FAPs). The FAPs of a video sequence are first extracted and then analyzed using a semi-continuous HMM. The basic recognizer (Pardas, M. et al., Proc. 2002 IEEE Conf. Acoustics, Speech and Signal Processing, vol.4, p.3624-7, 2002) shows good performance in distinguishing entire expressions, previously marked, in video sequences. We now describe the adaptation of the technique to deal with unrestrained expression intervals in video sequences, that is, expressions whose boundaries have not been previously marked in the scene. We have taken advantage of the symmetry of the expressions to extract a new HMM topology and we have trained the models without requiring to record and analyse a new database. We have also used a discarding process with the aim of eliminating expressions not included in the models, together with intervals without any expressions at all. The combination of the presented techniques is suitable for temporal block sampling with later expression classification or discarding.