Multimedia and virtual reality techniques for the control of ERA, the first free flying robot in space

The commanding and supervision of complex automation systems for space as well as for terrestrial automation applications is a demanding task. Modern developments in the field of virtual reality (VR) based man machine interfaces have the potential to facilitate such tasks enormously. At the Institute of Robotics Research (IRF) in Dortmund, Germany a variety of practical applications regarding the control of robots over long distances by means of virtual reality based man machine interfaces have been developed and successfully tested. In April 1999, the experiences made were successfully applied to the commanding and supervision of the robot ERA on board the Japanese satellite ETS-VII. In a joint mission between the Japanese Space Agency NASDA, the German Space Agency DLR and IRF, the robot was successfully commanded and supervised by means of "projective virtual reality" methods. The paper describes the key ideas and features of projective virtual reality as well as the metaphors used to augment the virtual world and to make the operation of the robot very intuitive. Before being able to apply the framework of projective virtual reality to this application, two major issues had to be addressed. First, it has to be made sure that the commanded robot can never damage itself or his environment this was addressed by automatic collision avoidance strategies. Secondly, the virtual model has to generated and then to be calibrated-this was addressed by the "TV-view into reality"-metaphor and modern 3D vision algorithms to provide the required online world-model updates.