Reinforcement Learning with Case-Based Heuristics for RoboCup Soccer Keepaway

In this paper we propose to combine Case-based Reasoning and Heuristically Accelerated Reinforcement Learning to speed up a Reinforcement Learning algorithm in a Transfer Learning problem. To do so, we propose a new algorithm called SARSA Accelerated by Transfer Learning - SATL, which uses Reinforcement Learning to learn how to perform one task, stores the policy for this problem as a case-base and then uses the learned case-base as heuristics to speed up the learning performance in a related, but different, task. A set of empirical evaluations were conducted in transferring the learning between two domains with multiple agents: an expanded version of Littman´s simulated robot soccer and the RoboCup Soccer Keep away. A policy learned by one agent in the Littman´s soccer is used to speed up the agent learning in the Keep away soccer. The results show that the use of this new algorithm can lead to a significant improvement in the performance of the learning agents.