Estimating 3D shape from degenerate sequences with missing data

Reconstructing a 3D scene from a moving camera is one of the most important issues in the field of computer vision. In this scenario, not all points are known in all images (e.g. due to occlusion), thus generating missing data. On the other hand, successful 3D reconstruction algorithms like Tomasi & Kanade’s factorization method, require an orthographic model for the data, which is adequate in close-up views. The state-of-the-art handles the missing points in this context by enforcing rank constraints on the point track matrix. However, quite frequently, close-up views tend to capture planar surfaces producing degenerate data. Estimating missing data using the rank constraint requires that all known measurements are “full rank” in all images of the sequence. If one single frame is degenerate, the whole sequence will produce high errors on the reconstructed shape, even though the observation matrix verifies the rank 4 constraint. In this paper, we propose to solve the structure from motion problem with degenerate data, introducing a new factorization algorithm that imposes the full scaled-orthographic model in one single optimization procedure. By imposing all model constraints, a unique (correct) 3D shape is estimated regardless of the data degeneracies. Experiments show that remarkably good reconstructions are obtained with an approximate models such as orthography.