Reconstruction and Accurate Alignment of Feature Maps for Augmented Reality

This paper focuses on the preparative process of retrieving accurate feature maps for a camera-based tracking system. With this system it is possible to create ready-to use Augmented Reality applications with a very easy setup work-flow, which in practice only involves three steps: filming the object or environment from various viewpoints, defining a transformation between the reconstructed map and the target coordinate frame based on a small number of 3D-3D correspondences and, finally, initiating a feature learning and Bundle Adjustment step. Technically, the solution comprises several sub-algorithms. Given the image sequence provided by the user, a feature map is initially reconstructed and incrementally extended using a Simultaneous-Localization-and-Mapping (SLAM) approach. For the automatic initialization of the SLAM module, a method for detecting the amount of translation is proposed. Since the initially reconstructed map is defined in an arbitrary coordinate system, we present a method for optimally aligning the feature map to the target coordinated frame of the augmentation models based on 3D-3D correspondences defined by the user. As an initial estimate we solve for a rigid transformation with scaling, known as Absolute Orientation. For refinement of the alignment we present a modification of the well-known Bundle Adjustment, where we include these 3D-3D-correspondences as constraints. Compared to ordinary Bundle Adjustment we show that this leads to significantly more accurate reconstructions, since map deformations due to systematic errors such as small camera calibration errors or outliers are well compensated. This again results in a better alignment of the augmentations during run-time of the application, even in large-scale environments.