Segmentation-based view synthesis for three-dimensional video

This paper investigates the use of segmentation in view synthesis for three-dimensional video. View synthesis is the process of generating novel views of a scene, using a set of views as the reference. Recently, several techniques that use depth maps for rendering virtual views have been suggested. However, inaccuracy in depth maps causes annoying visual artifacts in depth-based view synthesis. This paper presents an efficient depth image-based rendering technique based on segmentation using multi-level thresholding. In the proposed algorithm, first all the images are segmented according to the depth and the pixels belonging to different objects are warped and blended independently. Based on multi-level thresholding, an algorithm for finding the ghost contour pixels is provided which simplifies the computations. A novel inpainting method for disocclusions has been introduced which uses the segmented images to find the associated background boundary pixels. The experimental results show that the proposed algorithm improves the PSNR of the synthesized views up to 0.68 dB for the multi-view video test sequences and eliminates the annoying visual artifacts.