Learning features for action recognition and identity with deep belief networks

Feature extraction is a crucial part of computer vision. In this paper, we present a novel method that can automatically extract relevant features from video for action recognition and identity of human who makes the action, in single framework. We propose a watermark embedding in a video to represent a human identity as a 2-D wavelet transform. The feature extraction consists of a Deep Belief Network (DBN) on Discrete Fourier Transforms (DFTs) of the tracked features in a video. We then use the activations of the trained network as inputs for a non-linear Support Vector Machine (SVM) classifier. In particular, the learned features are used to solve the task of action recognition and identity. The method significantly reduces computational cost without scarifying any recognition accuracy. We test our algorithm on the KTH human motion dataset. Our results reflect the promise of our approach. Moreover, we obtain high classification accuracy on the KTH dataset that compares favorably against state-of-the-art action classifiers using hand-designed features.