Image-based augmentation of virtual object for handy camera video sequence using arbitrary multiple planes

We propose a novel vision-based registration method for augmented reality with merging of arbitrary multiple planes. In our approach, we require neither artificial markers nor sensors, and estimate the camera rotation and translation by an uncalibrated image sequence in which arbitrary multiple planes in the real world exist. Since the geometrical relationship of those planes is unknown, for merging of them, we assign a 3D coordinate system for each plane independently and construct projective 3D space defined by projective geometry of two reference images. By merging with the projective space, we can use arbitrary multi-planes, and achieve high-accurate registration for every position in the input images.