Cross-Spectral Stereo Matching Based on Local Self-Similarities and Image Moments

Recent developments in the infrared industry and the availability of relatively cheaper infrared cameras attracted more attention towards the simultaneous utilization of visible and infrared images and solving the correspondence problem between them. The Microsoft Kinect sensor represents one of the most widespread examples of low cost multimodal setups. In this context, we investigate the feasibility of matching features extracted from cross-spectral stereo cameras using a sparse approach. First, a set of stable features are extracted from both images using the Scale Invariant Feature Transform (SIFT). Then, descriptors are computed from local self-similarities around the selected key points. The combination of image moments is also investigated and expected to improve the matching process. Experimental results show the performance of this approach to the multimodal correspondence challenge.