A generic learning approach to multisensor based control

We propose a concept for integrating sensors in real-time robot control. To increase the controller robustness under diverse uncertainties, the robot systematically generates series of sensor data (as robot state) while memorising the corresponding motion parameters. Based on the collection of (multi-) sensor trajectories, statistical indices like principal components for each sensor type can be extracted. If the sensor data are pre-selected as output relevant, these principal components can be used very efficiently to approximately represent the original perception scenarios. After this dimension reduction procedure, a nonlinear fuzzy controller can be trained to map the subspace projection into the robot control parameters. We apply the approach to a real robot system with two arms and multiple vision and force/torque sensors. These external sensors are used simultaneously to control the robot arm performing insertion and screwing operations. The successful experiments show that the robustness and the precision of robot control can be enhanced by integrating additional sensors using this concept.