Visible-infrared fusion in the frame of an obstacle recognition system

In this article we propose different fusion schemes using information provided by visible and infrared images for road obstacle SVM-based classification. Three approaches for the fusion of VIS and IR information are presented. The early fusion yields a feature vector integrating at the feature level both visual and infrared information. The obtained bimodal feature vector is used as input to an SVM-based classification scheme. The intermediate fusion, which is performed at the kernel level combines different simple kernels of the SVM classifier in order to obtain a multiple kernel (MK). The late fusion combines matching scores of individual obstacle recognition modules in order to improve the system´s final decision. In this late fusion case two methods have been considered to calculate the optimum weighting parameter: an Adaptive Fusion of Scores (AFScores) and a non-Adaptive Fusion of Scores (nAFScores). Comparative results showed that fusion-based obstacle recognition systems outperform monomodal visual and infrared obstacle recognizers. An important advantage of these fusion-based systems is their possibility to adapt to the environmental illumination conditions due to the weighting parameter which can contribute to the adjustments of the system´s final decision.