View-Invariant 3D Action Recognition Using Spatiotemporal Self-Similarities from Depth Camera

The problem of viewpoint changes is an important issue in the study of human action recognition. In this paper, we propose the use of spatial features in a spatiotemporal self-similarity matrix (SSM) based on action recognition that is robust in viewpoint changes from depth sequences. The spatial features represent a discriminative density of 3D point clouds in a 3D grid. We construct the spatiotemporal SSM for the spatial features that change along with frames. To obtain the spatiotemporal SSM, we compute the Euclidean distance of each spatial feature between two frames. The spatiotemporal SSM represents similarity of human action robust in viewpoint changes. Our proposed method is robust in viewpoint changes and various length of action sequence. This method is evaluated on ACTA2 dataset containing the multi-view RGBD human action data, and MSRAction3D dataset. In the experimental validation, the spatiotemporal SSM is a good solution for the problem of viewpoint changes in a depth sequence.