Depth image filter for mixed and noisy pixel removal in RGB-D camera systems

The commercial RGB-D camera produces color images and their depth maps from a scene in real time. However, the active camera creates mixed depth data near the border of different objects, occasionally losing depth information of shiny and dark surfaces in the scene. Furthermore, noise is added to the depth map. In this paper, a new method is presented to resolve such mixed, lost, and noisy pixel problems of the RGB-D camera. In particular, mixed pixel areas are detected using common distance transform (CDT) values of color and depth pixels, and merged them to lost pixel regions. The merged regions are filled with neighboring depth information based on an edge-stopping convergence function; distance transform values of color edge pixels are used to form this function. In addition, a CDT-based joint multilateral filter (CDT-JMF) is used to remove noisy pixels. Experimental results show that the proposed method gives better performance than conventional hole filling methods and image filters.