A multi-agent mobile robot system with environment perception and HMI capabilities

A multi-agent robot system can speed up exploration or search and rescue operations in dangerous environments by working as a distributed sensor network. Each robot (e.g. Eddi Robot) equipped with a combined 2D/3D sensor (MS Kinect) and additional sensors needs to efficiently exchange its collected data with the other group members for task planning. For environment perception a 2D/3D panorama is generated from a sequence of images which were obtained while the robot was rotating. Furthermore the 2D/3D sensor data is used for a Human-Machine Interaction based on hand postures and gestures. The hand posture classification is realized by an Artificial Neural Network (ANN) which is processing a feature vector composed of Cosine-Descriptors (COD), Hu-moments and geometric features extracted of the hand shape. The System achieves an overall classification rate of more than 93%. It is used within the hand posture and gesture based human machine interface to control the robot team.