Activity recognition by learning structural and pairwise mid-level features using random forest

This paper presents a novel random forest based method to build mid-level features describing spatial and temporal structure information for activity recognition. Our model consists of two separate parts, spatial part and temporal part, which are employed to capture the distinctive characteristics in spatial and temporal domains of activity analysis. In the spatial part, densely sampled low level features are passed through the first level random forest and concatenated structurally to form spatial mid-level features. In the temporal part, we use results from the first level random forest on sparsely sampled interest points to build pairwise mid-level features. The second level random forests operate on all the mid-level features and compute scores for these two parts. Then final recognition is based on the weighted sum of these two parts. Our method smoothly fuses both spatial and temporal information and builds more descriptive models, which can better represent human activities in large variations. Experimental results show that our method achieves promising performance on three available action and facial expression datasets.