Reinforcement learning of ball-in-a-cup playing robot

An autonomous robot, which makes decisions based on previous experience and feedback from its sensors is one of the challenges in robotics. Among the most promising framework to bring traditional robotics towards true autonomy is reinforcement learning. However, reinforcement learning in high dimensional spaces usually required to encode robot tasks is extremely difficult. The key idea to speed up the reinforcement learning is to limit the potentially huge search space of the policy by using previous experience and to generalize to new policies from similar cases. In the paper we evaluate this idea on learning of ball-in-a-cup playing robot. First we solve this problem using traditional approach by learning action-value function without using any previous knowledge. Next, we propose learning which relies on generalizing to new policies from similar cases. It is shown, that this latter approach dramatically reduces number cycles needed to learn the appropriate policy.