Filling large holes in LiDAR data by inpainting depth gradients

We introduce a technique to fill large holes in LiDAR data sets. We combine concepts from patch-based image inpainting and gradient-domain image editing to simultaneously fill both texture and structure in a LiDAR scan. We discuss the problems with directly inpainting a depth image, and present a solution to this problem based on inpainting the depth gradients. Once the inpainted depth gradients are obtained, we use an image reconstruction technique to obtain the final 3D scene structure. We present several real-world examples of this technique with excellent results.