Sparse depth map upsampling with RGB image and anisotropic diffusion tensor

This paper proposes a novel algorithm for up-sampling 2D sparse depth map projected by laser scanner (i.e. Velodyne HDL-64E) using its synchronised RGB image and Anisotropic Diffusion Tensor. We assume each depth-unknown pixel´s depth value derives from all depth-known pixels and their affinity can be measured in a geodesic manner. Specifically, for each depth-unknown point, we compute its geodesic distance to all depth-known points, the cost of each geodesic path is calculated by spatial distance, color aberrance and tensor discrepancy, which compacts with the assumption that color homogenous region corresponds to close or smooth depth value, while depth discontinuity often happens in image edges. To mitigate computation complexity, we further introduced a flexible approximation algorithm in which the complexity is linear to image´s size and can be further reduced w.r.t. different accuracy requirement. Finally, we evaluate our algorithm on both KITTI visual benchmark suite and Middlebury dataset. Experiment shows that, while generating smooth and dense upsampling result, our algorithm retains sharp depth discontinuity even in edges of objects lies very far and few laser scanner points cover it. Besides, our algorithm is parameter-nonsensitive, which frees us from laboriously finding appropriate parameters to get fine result. We hope our work would motivate more research on laser point upsampling and their combination with RGB image, especially in outdoor scenes for autonomous driving.