Minimal projective reconstruction with missing data

The minimal data necessary for projective reconstruction from point correspondences is well-known when the points are visible in all images. In this paper, we formulate and propose solutions to a new family of reconstruction problems from multiple images with minimal data, where there are missing points in some of the images. The ability to handle the minimal cases with missing data is of great theoretical and practical importance. It is unavoidable to use them to bootstrap robust estimation such as RANSAC and LMS algorithms and optimal estimation such as bundle adjustment. First, we develop a framework to parametrize the multiple view geometry, needed to handle the missing data cases. Then we present a solution to the minimal case of 8 points in 3 images, where one of the points is missing in one of the three images. We prove that there are in general as many as 11 solutions for this minimal case. Furthermore, all minimal cases with missing data for 3 and 4 in images are catalogued. Finally we demonstrate the method on both simulated and real images and show that the algorithms presented in this paper can be used for practical problems