Title :
Smoothed Sarsa: Reinforcement learning for robot delivery tasks
Author :
Ramachandran, Deepak ; Gupta, Rakesh
Author_Institution :
Computer Science Dept., University of Illinois at Urbana-Champaign, IL-61801, USA
Abstract :
Our goal in this work is to make high level decisions for mobile robots. In particular, given a queue of prioritized object delivery tasks, we wish to find a sequence of actions in real time to accomplish these tasks efficiently. We introduce a novel reinforcement learning algorithm called Smoothed Sarsa that learns a good policy for these delivery tasks by delaying the backup reinforcement step until the uncertainty in the state estimate improves. The state space is modeled by a Dynamic Bayesian Network and updated using a Region-based Particle Filter. We take advantage of the fact that only discrete (topological) representations of entity locations are needed for decision-making, to make the tracking and decision making more efficient. Our experiments show that policy search leads to faster task completion times as well as higher total reward compared to a manually crafted policy. Smoothed Sarsa learns a policy orders of magnitude faster than previous policy search algorithms. We demonstrate our results on the Player/Stage simulator and on the Pioneer robot.
Keywords :
Decision making; Delay estimation; Learning; Mobile robots; Navigation; Orbital robotics; Particle filters; Robotics and automation; State estimation; Uncertainty;
Conference_Titel :
Robotics and Automation, 2009. ICRA '09. IEEE International Conference on
Conference_Location :
Kobe
Print_ISBN :
978-1-4244-2788-8
Electronic_ISBN :
1050-4729
DOI :
10.1109/ROBOT.2009.5152707