Spatial programming for industrial robots based on gestures and Augmented Reality

The presented spatial programming system provides an assistance system for online programming of industrial robots. A handheld device and a motion tracking system establish the basis for a modular 3D programming approach corresponding to different phases of robot programming: definition, evaluation and adaption. Static and dynamic gestures enable the program definition of poses, trajectories and tasks. The spatial evaluation is done using an Augmented Reality application on a handheld device. Therefore, the programmer is able to move freely within the robot cell and define the program spatially through gestures. The camera image of the handheld is simultaneously enhanced by virtual objects representing the robot program. Based on 3D motion tracking of human movements and a mobile Augmented Reality application, we introduce a novel kind of interaction for the adaption of robot programs. The programmer is enabled to interact with virtual program components through bare-hand gestures. Such sample forms of interaction include translation and rotation applicable to poses, trajectories or tasks representations. Finally, the program is adapted according to the gestural changes and can be transferred from the handheld device directly to the robot controler.