DocumentCode :
2385462
Title :
Adaptive autonomous control using online value iteration with gaussian processes
Author :
Rottmann, Axel ; Burgard, Wolfram
Author_Institution :
Dept. of Comput. Sci., Univ. of Freiburg, Freiburg, Germany
fYear :
2009
fDate :
12-17 May 2009
Firstpage :
2106
Lastpage :
2111
Abstract :
In this paper, we present a novel approach to controlling a robotic system online from scratch based on the reinforcement learning principle. In contrast to other approaches, our method learns the system dynamics and the value function separately, which permits to identify the individual characteristics and is, therefore, easily adaptable to changing conditions. The major problem in the context of learning control policies lies in high-dimensional state and action spaces, that needs to be explored in order to identify the optimal policy. In this paper, we propose an approach that learns the system dynamics and the value function in an alternating fashion based on Gaussian process models. Additionally, to reduce computation time and to make the system applicable to online learning, we present an efficient sparsification method. In experiments carried out with a real miniature blimp we demonstrate that our approach can learn height control online. Further results obtained with an inverted pendulum show that our method requires less data to achieve the same performance as an off-line learning approach.
Keywords :
Gaussian processes; adaptive control; intelligent robots; learning (artificial intelligence); learning systems; optimal control; robot dynamics; Gaussian process; adaptive autonomous control; computation time reduction; learning control policy; online learning; online value iteration; reinforcement learning principle; robotic system control; system dynamics; value function; Adaptive control; Automatic control; Control systems; Dynamic programming; Gaussian processes; Learning systems; Optimal control; Programmable control; Robotics and automation; Runtime;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Robotics and Automation, 2009. ICRA '09. IEEE International Conference on
Conference_Location :
Kobe
ISSN :
1050-4729
Print_ISBN :
978-1-4244-2788-8
Electronic_ISBN :
1050-4729
Type :
conf
DOI :
10.1109/ROBOT.2009.5152660
Filename :
5152660
Link To Document :
بازگشت