Integrating joint and surface for human action recognition in indoor environments

Action recognition has a long research history, despite several contributed approaches have been introduced, it remains a challenging task in computer vision. In this paper, we present a uniform fusion framework for action recognition, which integrates not only the local depth cues but also the global depth cues. Firstly, the action recognition task is formulated as the maximize the posterior probability, and then the observation for the original action is decomposed into the sub-observations for each individual feature representation strategy of the original action. For the local depth cues, the joints inside the human skeleton is employed to model the local variation of the human motion. In addition, the normal of the depth surface is utilized as the global cue to capture the holistic structure of the human motion. Rather than using the original feature directly, the support vector machine model learning both the discriminative local cue (i.e., the joint) and the discriminative global cue (i.e., the depth surface), respectively. The presented approach is validated on the famous MSR Daily Activity 3D Dataset. And the experimental results demonstrate that our fusion approach can outperform the baseline approaches.