Improving imitated grasping motions through interactive expected deviation learning

One of the major obstacles that hinders the application of robots to human day-to-day tasks is the current lack of flexible learning methods to endow the robots with the necessary skills and to allow them to adapt to new situations. In this work, we present a new intuitive method for teaching a robot anthropomorphic motion primitives. Our method combines the advantages of reinforcement and imitation learning in a single coherent framework. In contrast to existing approaches that use human demonstrations merely as an initialization for reinforcement learning, our method treats both ways of learning as homologous modules and chooses the most appropriate one in every situation. We apply Gaussian Process Regression to generalize a measure of value across the combined state-action-space. Based on the expected value, uncertainty, and expected deviation of generalized movements, our method decides whether to ask for a human demonstration or to improve its performance on its own, using reinforcement learning. The latter employs a probabilistic search strategy, based on expected deviation, that greatly accelerates learning while protecting the robot from unpredictable movements at the same time. To evaluate the performance of our approach, we conducted a series of experiments and successfully trained a robot to grasp an object at arbitrary positions on a table.