Title :
Autonomous helicopter control using reinforcement learning policy search methods
Author :
Bagnell, J. Andrew ; Hneider, Jeff G Sc
Author_Institution :
Robotics Inst., Carnegie Mellon Univ., Pittsburgh, PA, USA
Abstract :
Many control problems in the robotics field can be cast as partially observed Markovian decision problems (POMDPs), an optimal control formalism. Finding optimal solutions to such problems in general, however is known to be intractable. It has often been observed that in practice, simple structured controllers suffice for good sub-optimal control, and recent research in the artificial intelligence community has focused on policy search methods as techniques for finding sub-optimal controllers when such structured controllers do exist. Traditional model-based reinforcement learning algorithms make a certainty equivalence assumption on their learned models and calculate optimal policies for a maximum-likelihood Markovian model. We consider algorithms that evaluate and synthesize controllers under distributions of Markovian models. Previous work has demonstrated that algorithms that maximize mean reward with respect to model uncertainty leads to safer and more robust controllers. We consider briefly other performance criterion that emphasize robustness and exploration in the search for controllers, and note the relation with experiment design and active learning. To validate the power of the approach on a robotic application we demonstrate the presented learning control algorithm by flying an autonomous helicopter. We show that the controller learned is robust and delivers good performance in this real-world domain.
Keywords :
Bayes methods; Markov processes; aircraft control; computational complexity; control system synthesis; decision theory; helicopters; learning (artificial intelligence); remotely operated vehicles; search problems; suboptimal control; active learning; autonomous helicopter control; partially observed Markovian decision problems; policy search methods; reinforcement learning; robustness; Artificial intelligence; Computer crashes; Control system synthesis; Helicopters; Learning systems; Optimal control; Robots; Robust control; Search methods; Uncertainty;
Conference_Titel :
Robotics and Automation, 2001. Proceedings 2001 ICRA. IEEE International Conference on
Print_ISBN :
0-7803-6576-3
DOI :
10.1109/ROBOT.2001.932842