Human action recognition with extremities as semantic posture representation

In this paper, we present an approach for human action recognition with extremities as a compact semantic posture representation. First, we develop a variable star skeleton representation (VSS) in order to accurately find human extremities from contours. Earlier, Fujiyoshi and Lipton proposed an image skeletonization technique with the center of mass as a single star for rapid motion analysis. Yu and Aggarwal used the highest contour point as the second star in their application for fence climbing detection. We implement VSS and earlier algorithms and compare their performance over a set of 1000 frames from 50 sequences of persons climbing fences to analyze the characteristic of each representation. Our results show that VSS performs the best. Second, we build feature vectors out of detected extremities for hidden Markov model (HMM) based human action recognition. On the data set of human climbing fences, we achieved excellent classification accuracy. On the publicly available Blank et al. data set, our approach showed that using only extremities is sufficient to obtain comparable classification accuracy against other state-of-the-art performance. The advantage of our approach lies in the less time complexity with comparable classification accuracy.