Title :
Computational Multi-View Imaging with Kinect
Author :
Xinchen Ye ; Jingyu Yang ; Hao Huang ; Chunping Hou ; Yao Wang
Author_Institution :
Sch. of Electron. Inf. Eng., Tianjin Univ., Tianjin, China
Abstract :
The lack of 3-D content has become a bottleneck for the advancement of three-dimensional television (3-DTV), but conventional multicamera arrays for multiview imaging are expensive to setup and cumbersome to use. This paper proposes a lightweight multiview imaging approach with Kinect, a handheld integrated depth-color camera, under the depth-image-based rendering framework. The proposed method consists of two components: depth restoration from noisy and incomplete depth measurements and view synthesis from depth-color pairs. In depth restoration, we propose a moving 2-D polynomial approximation via least squares to suppress quantization errors in the acquired depth values, and propose a progressive edge-guided trilateral filter to fill missing areas of the depth map. Edges extracted from color image are used to predict the locations of depth discontinuities in missing areas and to guide the proposed trilateral filter avoiding filtering across discontinuities. In view synthesis, we propose a low-rank matrix restoration model to inpaint disocclusion regions, fully exploiting the nonlocal correlations in images, and devise an efficient algorithm under the augmented lagrange multiplier (ALM) framework. Disocclusion areas are inpainted progressively from the boundaries of disocclusion with an estimated priority consisting of four terms: warping term, reliability term, texture term, and depth term. Experimental results show that our method restores high quality depth maps even for large missing areas, and synthesizes natural multiview images from restored depth maps. Strong 3-D visual experiences are observed when the synthesized multiview images are shown in two types of stereoscopic displays.
Keywords :
image colour analysis; image restoration; least squares approximations; polynomial approximation; reliability; rendering (computer graphics); stereo image processing; television cameras; three-dimensional television; 3D visual experiences; 3DTV; ALM framework; Kinect; augmented lagrange multiplier framework; color image; computational multiview imaging; depth term; depth-color pairs; depth-image-based rendering framework; disocclusion areas; handheld integrated depth-color camera; least squares; lightweight multiview imaging; low-rank matrix restoration; moving 2D polynomial approximation; multicamera arrays; natural multiview images; nonlocal correlations; progressive edge-guided trilateral filter; reliability term; restored depth maps; stereoscopic displays; texture term; three-dimensional television; view synthesis; warping term; Color; Image color analysis; Image edge detection; Image restoration; Imaging; Quantization (signal); Three-dimensional displays; Kinect; Multiview imaging; depth restoration; matrix completion; view synthesis;
Journal_Title :
Broadcasting, IEEE Transactions on
DOI :
10.1109/TBC.2014.2345931