Efficient Augmentation of the EKF Structure from Motion with Frame-to-Frame Features

The Extended Kalman Filter (EKF) is still one of the most widely used approaches for small scale Structure from Motion (SFM) and Simultaneous Localization And Mapping (SLAM) problems. However, the EKF does not have the ability to take into account the motion information carried by features matched only between two consecutive frames. This information is valuable because, when used appropriately, it generally enhances the performance of the filter. Two main reasons hinder the direct use of such features in the EKF: their un-initialized 3D location would corrupt the covariance matrix, and the computational cost grows cubically with the number of features. In this paper we present a novel approach to solve those problems. Our approach folds the frame-to-frame information in the filter through a separate update step that can be carried out in linear time. Other advantages of our approach is that it can be introduced to already implemented filters with minimal change. It can be done in a separate thread to further speedup the computation. Additionally, it can be further divided to multiple steps with different sets of features, which permits to reject or accept each step based on some performance criteria and to stay within the budgeted time.