A multi-modal teaching-advisory system using complementary operator and sensor information

A multimodal teaching-advisory system for sensor-based robot systems is presented that uses complementary information obtained from the operator and sensors. We propose a new framework for on-site robot teaching systems that takes account of the characteristics of humans and sensors. This framework enables the teaching system to integrate a variety of robot task constraints, task specifications, their tolerances as defined by the operator, and sensing information acquired by the sensors in a complementary manner. The system synthesizes all of this information into a form that is easily understood by the operator. Finally, the synthesized information is presented to the operator as multimodal information. By combining this advisory information with the operator´s own information, the desired robot teaching commands can be generated. This system can significantly improve total robot system performance by ensuring high-quality teaching