Video classification based on HMM using text and faces

Video content classification and retrieval is a necessary tool in the current merging of entertainment and information media. With the advent of broadband networking, every consumer will have video programs available on-line as well as in the traditional distribution channels. Systems that help in content management have to discern between different categories of video in order to provide for fast retrieval. In this paper we present a novel method for video classification based on face and text trajectories. This is based on the observation that in different TV categories there are different face and text trajectory patterns. Face and text tracking is applied to arbitrary video clips to extract faces and text trajectories. We used Hidden Markov Models (HMM) to classify a given video clip into predefined categories, e.g., commercial, news, sitcom and soap. Our preliminary experimental results show classification accuracy of over 80% for HMM method on short video clips. This paper describes continuity-based face and text detection and tracking in video for the above HMM classification method.