A 3D laser and vision based classifier

This paper presents a method for modelling semantic content in scenes, in order to facilitate urban driving. More specifically, it presents a 3D classifier based on Velodyne data and monocular color imagery. The system contains two main components: a ground model and an object model. The ground model is a novel extension of elevation maps using Conditional Random Fields. It allows estimation of ground type (grass vs. asphalt) in addition to modelling the geometry of the scene. The object model involves two segmentation procedures. The first is a novel extension of elevation maps to a hierarchical clustering algorithm. The second is a new algorithm for defining regions of interest in images, which reasons jointly in the 3D Cartesian frame and the image plane. These two procedures provide a segmentation of the objects in the 3D laser data and in the images. Based on the resulting segmentation, object classification is implemented using a rule based system to combine binary deterministic and probabilistic features. The overall 3D classifier is tested on logs acquired by the MIT Urban Grand Challenge 2007 vehicle. The classifier achieves an accuracy of 89% on a set of 500 scenes involving 16 classes. The proposed approach is evaluated against seven other standard classification algorithms, and is shown to produce superior performance.