View independent recognition of human-vehicle interactions using 3-D models

Recognition of human-vehicle interactions is a challenging problem. The occlusion by vehicles and motion of humans contribute to the difficulty. In this paper, we present a novel approach for the view independent recognition of human-vehicle interactions. The shape based matching of synthetic 3D vehicle models is used for accurate localization of vehicles and for the specification of regions-of-interest (e.g. doors). In the proposed method, the system transforms the optical flow field based on the position of doors and the direction of a vehicle. This enables the system to extract view-independent features. Histogram of oriented optical flow (HOOF) and histogram of oriented gradient (HOG) characterize the optical flow and gradient, respectively. A support vector machine (SVM) classifier is trained for these view-independent features. Consequently, the system recognizes the interactions of a person entering a vehicle and getting out of a vehicle. Our method is applied to a dataset of human-vehicle interactions taken from 8 different viewpoints, composed of 120 video clips. The experimental results show that the system recognizes sequences of complex human-vehicle interactions with a high recognition rate of 86%.