A fast and robust approach to recovering structure and motion from live video frames

This paper describes a fast and robust approach to recovering structure and motion from video frames. It first describes a robust recursive factorization method for affine projection. Using the least median of squares (LMedS) criterion, the method estimates the dominant 3D affine motion and discards feature points regarded as outliers. The computational cost of the overall procedure is reduced by combining this robust statistics-based method with a recursive factorization method that can at each frame provide the updated 3D structure of an object at a fixed computational cost by using the principal component analysis. Experiments with synthetic data and with real image sequences demonstrate that the method can be used to estimate the dominant structure and the motion robustly and in real-time, on an off-the-shelf PC. Finally this paper describes preliminary online experiments on selecting feature points, which have the dominant 3D motion, from live video frames by using a PC-cluster system