Improvement of segment-based depth estimation using a novel segment extraction

We present a novel segment extraction and segment-based depth estimation technique. Proposed segment extraction technique exploits depth and motion information of segments between frames as well as color information. We firstly divide each frame of reference view into foreground and background areas based on initial depth information obtained from time-of-flight (TOF) camera. Then we extract segments with color information by applying image segmentation technique to each divided area. Moreover, we track the extracted segments between frames in order to maintain depth consistency. We set the disparity search range for local segment-based stereo matching based on the initial depth from TOF camera. Experimental results showed the superior performance of the proposed technique over conventional ones that do not use foreground and background separation or motion tracking of segments especially at the static background and the regions that have depth discontinuities in depth but similar colors.