Action classification in polarimetric infrared imagery via diffusion maps

This work explores the application of a nonlinear dimensionality reduction technique known as diffusion maps for performing action classification in polarimetric infrared video sequences. The diffusion maps algorithm has been used successfully in a variety of applications involving the extraction of low-dimensional embeddings from high-dimensional data. Our dataset is composed of eight subjects each performing three basic actions: walking, walking while carrying an object in one hand, and running. The actions were captured with a polarized microgrid sensor operating in the longwave portion of the electromagnetic (EM) spectrum with a temporal resolution of 24 Hz, yielding the Stokes traditional intensity (S₀) and linearly polarized (S₁, S₂) components of data. Our work includes the use of diffusion maps as an unsupervised dimensionality reduction step prior to action classification with three conventional classifiers: the linear perceptron algorithm, the k nearest neighbors (KNN) algorithm, and the kernel-based support vector machine (SVM). We present classification results using both the low-dimensional principal components via PCA and the low-dimensional diffusion map embedding coordinates of the data for each class. Results indicate that the diffusion map lower-dimensional embeddings provide a salient feature space for action classification, yielding an increase of overall classification accuracy by ~40% compared to PCA. Additionally, we examine the utility that the polarimetric sensor may provide by concurrently performing these analyses in the polarimetric feature spaces.