Motion Local Ternary Pattern for distributed multi-view human action recognition

Multi-view human action recognition has gained a lot of attention in recent years for its superior performance as compared to the single view recognition. In this paper, we propose algorithms for the real-time realization of human action recognition in distributed camera networks (DCNs). We first present a new method for fast calculation of motion information by Motion Local Ternary Pattern (Mltp) that is tolerant to illumination change, robust in homogeneous region and computationally efficient. Next, we combine the local interest point detector with Mltp to generate 3D patches containing motion information and introduce two feature descriptors for the extracted 3D patches. Taking advantage of the proposed Mltp, 3D patches generated from background can be further removed automatically and thus the foreground patches can be highlighted. Finally, the histogram representations based on Bag-of-Words modeling, are transmitted from local cameras to the base station for classification. At the base station, a probability model is produced to fuse the information from various views and a class label is assigned accordingly. Compared to the existing algorithms, the proposed methods have three advantages: 1) no preprocessing is required; 2) communication among cameras is unnecessary; and 3) positions and orientations of cameras do not need to be fixed. We further evaluate both descriptors on the most popular multi-view action dataset IXMAS. Experimental results indicate that our approaches repeatedly achieve state-of-the-art results when various numbers of views are tested. In addition, our approaches are tolerant to the various combination of views and benefit from introducing more views at the testing stage.