Combining single-image and multiview super-resolution for mixed-resolution image plus depth data

In mixed-resolution multiview setups, a scene is captured from various viewpoints with cameras having different spatial resolutions. Compared to full-resolution systems, mixed-resolution setups allow for savings with respect to data transmission, storage, and costs. However, for applications like free viewpoint television, high-quality images are required for all available camera perspectives. Therefore, high-resolution cameras can be used to increase the image quality of a neighboring low-resolution view. Due to occlusions, some parts of the scene are invisible in the high-resolution reference views and thus cannot be directly synthesized from the neighboring perspectives. In this paper, we propose to integrate the idea of single-image super-resolution to better handle occluded areas and thus to improve the super-resolution quality for mixed-resolution multiview images. For a downsampling factor of 4, the proposed method achieves an average gain of 0.53 dB with respect to a comparable multiview super-resolution approach.