Free Viewpoint Video Coding With Rate-Distortion Analysis

To improve free viewpoint video (FVV) coding efficiency and optimize the quality of the synthesized virtual view video, this paper proposes a depth-assisted FVV coding framework and analyzes the rate-distortion (R-D) property of the synthesized virtual view video in FVV coding. In the depth-assisted FVV coding framework, the depth assigned disparity compensated prediction is introduced to exploit the correlation between multiview video (MVV) and depth. To model the R-D property of the synthesized virtual view video, a region-based view synthesis distortion estimation approach is investigated with respect to the distortion of MVV and depth. Subsequently, the general R-D property estimation models of MVV and depth are analyzed. Finally, a rate-allocation scheme is designed to optimize the quantization parameter pair of MVV and depth in FVV coding. The simulation results demonstrate that the proposed depth-assisted FVV coding framework can improve the FVV coding efficiency. The region-based view synthesis distortion estimation approach and the general R-D model are able to precisely approximate the R-D property of synthesized virtual view video in the multiview video plus depth based FVV coding frameworks. The proposed rate-allocation scheme can optimize the overall FVV coding efficiency to achieve a high-quality reconstructed video at the desired viewpoint with a given rate constraint.