Stable real-time interaction between virtual humans and real scenes

We present an augmented reality system that relies on purely passive techniques to solve the real-time registration problem. It can run on a portable PC and does not require engineering of the environment, for example by adding markers. To achieve this result, we have integrated robust computer vision techniques into a powerful VR framework. The resulting AR system allows us to produce complex rendering and animation of virtual human characters, and to blend them into the real world. The system tracks the 3D camera position by means of a natural features tracker, which, given a rough CAD model, can deal with complex 3D objects. The tracking method can handle both large camera displacements and aspect changes. We will show that our system works in the cluttered environment of a real industrial facility and can, therefore, be used to enhance manufacturing and industrial processes.