Spatial and Temporal Enhancement of Depth Images Captured by a Time-of-Flight Depth Sensor

In this paper, we present a new method to enhance depth images captured by a time-of-flight (TOF) depth sensor spatially and temporally. In practice, depth images obtained from TOF depth sensors have critical problems, such as optical noise existence, unmatched boundaries, and temporal inconsistency. In this work, we improve depth quality by performing a newly-designed joint bilateral filtering, color segmentation-based boundary refinement, and motion estimation-based temporal consistency. Experimental results show that the proposed method significantly minimizes the inherent problems of the depth images so that we can use them to generate a dynamic and realistic 3D scene.