Pedestrian Recognition by Using a Dynamic Modality Selection Approach

Despite many years of research, pedestrian recognition is still a difficult, but very important task. It was proved that concatenating information from multi-modality images improves the recognition accuracy, but with a high computational cost. We present a modality selection approach, which is able to dynamically select the most discriminative modality for a given image and furthermore use it in the classification process. Firstly, we extract kernel descriptor features from a given image in three modalities: intensity, depth and flow. Secondly, we dynamically determine the most suitable modality for that image using both: a modality pertinence classifier and a decision confidence indicator. Thirdly, we classify the image in the selected modality using a linear SVM approach. Numerical experiments are performed on the Daimler benchmark dataset consisting of pedestrian and non-pedestrian bounding boxes captured in outdoor urban environments and indicate that our model outperforms all the individual-modality classifiers and the model based on a posterior fusion of multi-modality decisions. Moreover, the proposed selection model is a promising and less computational expensive alternative to the concatenation of multi-modality features prior to classification.