Title :
Reliability-Based Multiview Depth Enhancement Considering Interview Coherence
Author :
Jinwook Choi ; Dongbo Min ; Kwanghoon Sohn
Author_Institution :
Sch. of Electr. & Electron. Eng., Yonsei Univ., Seoul, South Korea
Abstract :
Color-plus-depth video format has been increasingly popular in 3-D video applications, such as auto-stereoscopic 3-D TV and freeview TV. The performance of these applications is heavily dependent on the quality of depth maps since intermediate views are synthesized using the corresponding depth maps. This paper presents a novel framework for obtaining high-quality multiview color-plus-depth video using a hybrid sensor, which consists of multiple color cameras and depth sensors. Given multiple high-resolution color images and low quality depth maps obtained from the color cameras and depth sensors, we improve the quality of the depth map corresponding to each color view by increasing its spatial resolution and enforcing interview coherence. Specifically, a new up-sampling method considering the interview coherence is proposed to enhance multiview depth maps. This approach can improve the performance of the existing up-sampling algorithms, such as joint bilateral up-sampling and weighted mode filtering, which have been developed to enhance a single-view depth map only. In addition, an adaptive approach of fusing multiple input low-resolution depth maps is proposed based on the reliability that considers camera geometry and depth validity. The proposed framework can be extended into the temporal domain for temporally consistent depth maps. Experimental results demonstrate that the proposed method provides better multiview depth quality than the conventional single-view-based methods. We also show that it provides comparable results, yet much more efficiently, to other fusion approaches that employ both depth sensors and stereo matching algorithm together. Moreover, it is shown that the proposed method significantly reduces bit rates required to compress the multiview color-plus-depth video.
Keywords :
image colour analysis; image enhancement; image fusion; image resolution; image sampling; image sensors; reliability; stereo image processing; three-dimensional television; video cameras; video signal processing; 3D video applications; adaptive approach; autostereoscopic 3D TV; camera geometry; color-plus-depth video format; depth sensors; depth validity; free-view TV; high-quality multiview color-plus-depth video; hybrid sensor; intermediate view synthesis; interview coherence; joint bilateral up-sampling algorithm; low quality depth maps; multiple color cameras; multiple high-resolution color images; multiple input low-resolution depth map fusion approach; reliability-based multiview depth enhancement; single-view depth map; stereo matching algorithm; weighted mode filtering; Calibration; Cameras; Coherence; Color; Image color analysis; Image resolution; Reliability; Color camera; Depth sensor; color camera; depth sensor; depth up-sampling; inter-view coherence; interview coherence; reliability;
Journal_Title :
Circuits and Systems for Video Technology, IEEE Transactions on
DOI :
10.1109/TCSVT.2013.2278160