The Human Action Image

Recognizing a person´s motion is intuitive for humans but represents a challenging problem in machine vision. In this paper, we present a multi-disciplinary framework for recognizing human actions. We develop a novel descriptor, the Human Action Image (HAI): a physically-significant, compact representation for the motion of a person, which we derive from first principles in physics using Hamilton´s Action. We embed the HAI as the Motion Energy Pathway of the latest Neurobiological model of motion recognition. The Form Pathway is modelled using existing low-level feature descriptors based on shape and appearance. Experimental validation of the theory is provided on the well-known Weizmann and USF Gait datasets.