Rank Conditions on the Multiple-View Matrix

Geometric relationships governing multiple images of points and lines and associated algorithms have been studied to a large extent separately in multiple-view geometry. The previous studies led to a characterization based on multilinear constraints, which have been extensively used for structure and motion recovery, feature matching and image transfer. In this paper we present a universal rank condition on the so-called multiple-view matrix M for arbitrarily combined point and line features across multiple views. The condition gives rise to a complete set of constraints among multiple images. All previously known multilinear constraints become simple instantiations of the new condition. In particular, the relationship between bilinear, trilinear and quadrilinear constraints can be clearly revealed from this new approach. The theory enables us to carry out global geometric analysis for multiple images, as well as systematically characterize all degenerate configurations, without breaking image sequence into pairwise or triple-wise sets of views. This global treatment allows us to utilize all incidence conditions governing all features in all images simultaneously for a consistent recovery of motion and structure from multiple views. In particular, a rank-based multiple-view factorization algorithm for motion and structure recovery is derived from the rank condition. Simulation results are presented to validate the multiple-view matrix based approach.