A novel optical tracking based tele-control system for tabletop object manipulation tasks

For a robot serving in a complex environment such as in a restaurant, it is difficult to perform a task like tabletop object manipulation completely by itself, in that some information may be missing. An approach to deal with this is to use a tele-control system and method to control the robot or demonstrate. In this paper, a LeapMotion sensor based non-contact tele-control method is developed for a robot to perform tabletop object manipulation tasks. A coordinate system for mapping from the operation space of the LeapMotion sensor to the workspace of the robot is established. A gesture recognition and action generating algorithm is proposed for control or to demonstrate the motion to the robot. To evaluate the performance of the LeapMotion sensor and proposed method for tele-control of a robot, a comprehensive assessment index based on entropy weighting is proposed. Three common tele-control modes, including demonstration mode, teleoperation mode and semi-teleoperation mode, are developed on a PR2 robot. The experimental results show that the proposed tele-control system is more appropriate for use in task demonstration.