Registration and fusion for ToF camera and 2D camera reading

Time of Flight (ToF) cameras have become a competitive alternative to traditional distance sensing techniques like laser or stereo vision as it can deliver grayscale images and 3D information simultaneously at high frame-rates. However the low resolution of ToF camera limits the applications for accurate segmentation or classification. This paper presents a fast and robust solution to combine the 3D information of ToF camera and high resolution color image. Firstly, we setup a 2D/3D stereo camera with fixed spatial relation and similar visual field. Based on characteristic of ToF camera and the principles of stereo vision system, the 3D information of ToF camera is registered with high resolution color image by matching the high resolution color image with grayscale image of ToF camera. This method has very low computation cost while the matching accuracy is only determined by physical characteristics or parameters of the 2D/3D stereo camera system without extra computation error. Experiments results have demonstrated the feasibility, efficiency and accuracy of proposed algorithms.