Visual learning and recognition of sequential data manifolds with applications to human movement analysis

Human motion analysis is increasingly attracting much attention from computer vision researchers. This paper aims to address the task of human gait and activity analysis from image sequences by learning and recognition of sequential data under a general integrated framework. Human movements generally exhibit intrinsically nonlinear spatiotemporal characteristics in the high-dimensional ambient space. An attractive framework, which we explore here, is to: (1) Extract simple and reliable features from image sequences. (2) Find a low-dimensional feature representation embedded in high-dimensional image data. (3) Then characterize/classify the motions in this low-dimensional feature space. We examine two simple alternatives for step 1: silhouette and a distance transformed silhouette; and three quite different methods for step 3: Gaussian mixture models (GMM) based classification, a matching-based approach with the mean Hausdorff distance, and continuous hidden Markov models (HMM) based modelling and recognition. The core is step 2 where we choose to use LPP (locality preserving projections), an optimal linear approximation to a nonlinear spectral embedding technique (i.e., Laplacian eigenmap). In essence our aim is to see whether this core, together with simple approaches to steps 1 and 3, can solve problems across several types of human gait and activity. To see how well the proposed framework performs, we carry out extensive experiments in three related domains: human activity recognition, abnormal gait analysis, and gait-based human identification. The experimental results show that the proposed framework performs well across all three areas.