Optimization of free viewpoint interpolation by applying adaptive depth plane distributions in plane sweeping a histogram-based approach to a non-uniform plane distribution

In this paper, we present a system to increase performance of plane sweeping for free viewpoint interpolation. Typical plane sweeping approaches incorporate a uniform depth plane distribution to investigate different depth hypotheses to generate a depth map, used in novel camera viewpoint generation. When the scene consists of a sparse number of objects, some depth hypotheses do not contain objects and can cause noise and wasted computational power. Therefore, we propose a method to adapt the plane distribution to increase the quality of the depth map around objects and to reduce computational power waste by reducing the number of planes in empty spaces in the scene. First, we generate the cumulative histogram of the previous frame in a temporal sequence of images. Next, we determine a new normalized depth for every depth plane by analyzing the cumulative histogram. Steep sections of the cumulative histogram will result in a dense local distribution of planes; a flat section will result in a sparse distribution. The results, performed on controlled and on real images, demonstrate the effectiveness of the method over a uniform distribution and allows a lower number of depth planes, and thus a more performant processing, for the same quality.