Stereo-Based Framework for Pedestrian Detection with Partial Occlusion Handling

The pedestrian detection literature has been recently extended by the availability of large-scale multisensory datasets, able to capture complementary aspects of the objects of interest, namely, appearance, motion, and depth. In this paper, we exploit this multimodal scenario to propose a new set of composite descriptors dubbed CO², CO-variances of visual features and CO-occurrences of depth fields. Covariances of visual features allow us to integrate at low-level heterogeneous visual cues related to intensity and texture. Co-occurrences of depth fields are brand new descriptors, which use range information for characterizing the global shape of a pedestrian while being also able to identify its occluded parts. This paper illustrates how these descriptors can be instantiated and combined together, improving detection capabilities taking also benefit from the proper handling of occlusions. Experimental results show that CO², fed into a standard discriminative classification system, set state-of-the-art performances on recent multi-modal intensity- and stereo-based pedestrian datasets.