Determining the fundamental matrix with planes: instability and new algorithms

The fundamental matrix is a key concept when working with uncalibrated images and multiple viewpoints. It contains all the available geometric information and enables recovery of the epipolar geometry from uncalibrated perspective views. The problem of its determination from points which lie in several planes is discussed. In that case, there is an homography between coordinates of points in the two images. The use of different criteria to compute the homography is investigated. A very simple and important relation between the homography matrices obtained from the observation of planar surfaces and the fundamental matrix is then established. Using simulations and real images to validate this analysis, it is shown that as a first consequence of this relation, the general methods to compute the fundamental matrix are unstable when the points lie close to planes. New algorithms are proposed to exploit this situation through the use of the previous relation. Their performance is compared to the performance of the general algorithm using a large number of noisy synthetic data and real images