Multi-class action recognition based on inverted index of action states

A fast inverted index based algorithm is introduced for multi-class action recognition. At first, we compute the shape-motion features of the automatically localized actor. Secondly, a binary state tree is built by hierarchically clustering of the extracted features, and the action states are the cluster centers. Then videos are represented as sequences of states by searching the state binary tree. With the labeled state sequences, we create the inverted index tables. During testing, the state and the state transition scores are computed by querying the inverted index tables. With the learned weight, we compute an action recognition score vector. The recognized action class is the index of the maximum score element. Our key contribution is that we propose a fast inverted index based multi-class action recognition approach. Experiments on several challenging data sets confirm the performance of this approach.