An iterative, non-local approach for restoring depth maps in RGB-D images

In this paper, we present a novel iterative median filter based strategy to improve the quality of the depth maps provided by sensors like Microsoft Kinect. The quality of the depth map is improved in two aspects, by filling holes present in the maps and by addressing the random noise. The holes are filled by iteratively applying a median based filter which takes into account the RGB components as well. The color similarity is measured by finding the absolute difference of the neighbourhood pixels and the median value. The hole filled depth map is further improved by applying a bilateral filter and processing the detail layer separately using Non-Local Denoising. The denoised detail layer is combined with the base layer to obtain a sharp and accurate depth map. We show that the proposed approach is able to generate high quality depth maps which can be quite useful in improving the performance of various applications of Microsoft Kinect such as pose estimation, gesture recognition, skeletal and facial tracking, etc.