Multi-view terrain classification using panoramic imagery and LIDAR

The focus of this work is addressing the challenges of performing object recognition in real world scenes as captured by a commercial, state-of-the-art, surveying vehicle equipped with a 360° panoramic camera in conjunction with a 3D laser scanner (LIDAR). Even with state-of-the-art surveying equipment, there is colour saturation and very dark regions in images, as well as some degree of time-varying misalignment between the point cloud data and imagery due to, for instance, imperfect tracking of sensor pose. Moreover, there are frequent occlusions due to both static and moving objects. These issues are inherently difficult to avoid and therefore need to be dealt with in a more robust fashion. This is where the contribution of the paper is; that is, the development of a consensus method that can intelligently incorporate feature responses from multiple views and reject those that are not very descriptive. It is shown that the overall performance in a ten class problem is increased from 70.5% for a simple 2D-3D classification system, to 77.5%. Subsequently, an enhanced CRF which has become robust using the misclassifications of training data and equipped with the probabilities of the adjacent points, was applied to the system and further improved its performance to 82.9%. The experiments were performed on a challenging dataset captured both in summer and winter.