Semantic classification of movie scenes using finite state machines

The problem of classifying scenes from feature films into semantic categories is addressed and a robust framework for this problem is proposed. It is proposed that the finite state machines (FSM) are suitable for detecting and classifying scenes and their usage is demonstrated for three types of movie scenes: conversation, suspense and action. This framework utilises the structural information of the scenes together with the low-level and mid-level features. Low level features of the video including motion and audio energy and a mid-level feature, body, are used in this approach. The transitions of the FSMs are determined by the features from each shot in the scene. The FSMs have been experimented on over 80 clips and convincing results have been achieved.