DocumentCode :
2490753
Title :
Reinforcement learning accelerated by using state transition model with robotic applications
Author :
Senda, Kei ; Fujii, Shinji ; Mano, Syusuke
Author_Institution :
Kanazawa Univ., Ishikawa, Japan
Volume :
4
fYear :
2004
fDate :
28 Sept.-2 Oct. 2004
Firstpage :
3732
Abstract :
This paper discusses a method to accelerate reinforcement learning. Firstly defined is a concept that reduces the state space conserving policy. An algorithm is then given that calculates the optimal cost-to-go and the optimal policy in the reduced space from those in the original space. Using the reduced state space, learning convergence is accelerated. Its usefulness for both DP (dynamic programming) iteration and Q-learning are compared through a maze example. The convergence of the optimal cost-to-go in the original state space needs approximately N or more times as long as that in the reduced state space, where N is a ratio of the state number of the original space to the reduced space. The acceleration effect for Q-learning is more remarkable than that for the DP iteration. The proposed technique is also applied to a robot manipulator working for a peg-in-hole task with geometric constraints. The state space reduction can be considered as a model of the change of observation, i.e., one of cognitive actions. The obtained results explain that the change of observation is reasonable in terms of learning efficiency.
Keywords :
convergence; dynamic programming; learning (artificial intelligence); manipulators; state-space methods; dynamic programming iteration; learning convergence; reinforcement learning; robot manipulator; state transition model; Acceleration; Cognitive robotics; Convergence; Costs; Intelligent robots; Learning; Orbital robotics; Satellites; Solar energy; State-space methods;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Intelligent Robots and Systems, 2004. (IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference on
Print_ISBN :
0-7803-8463-6
Type :
conf
DOI :
10.1109/IROS.2004.1389995
Filename :
1389995
Link To Document :
بازگشت