Graphical framework for action recognition using temporally dense STIPs

Graphical models have been shown to provide a natural framework for modelling high level action transition constraints, and to simultaneously segment and recognize a sequence of actions. Spatio-temporal interest points (STIPs) have been proposed as suitable features for action detection. These interest points are typically mapped to a set of codewords, and actions are detected by accumulating the codeword weights or by learning suitable classifiers. Existing methods for interest point detection provide a sparse representation of actions and require a costly exhaustive search over the entire spatio-temporal volume for action classification. Our contribution here is two-fold -first, we combine the interest point models of actions with pedestrian detection and tracking using a conditional random field (CRF); second, we extend existing interest point detectors to provide a dense action representation while minimizing spurious detections. The larger number of interest points and the high-level reasoning provided by the CRF allows us to automatically recognize action sequences from an unsegmented stream, at real time speed. We demonstrate our approach by showing results comparable to state-of-the-art for action classification on the standard KTH-action set, and also on more challenging cluttered videos.