Title :
Emergent proximo-distal maturation through adaptive exploration
Author :
Stulp, Freek ; Oudeyer, Pierre-Yves
Author_Institution :
Cognitive Robot., ENSTA-ParisTech, Paris, France
Abstract :
Life-long robot learning in the high-dimensional real world requires guided and structured exploration mechanisms. In this developmental context, we investigate here the use of the recently proposed PICMAES2 episodic reinforcement learning algorithm, which is able to learn high-dimensional motor tasks through adaptive control of exploration. By studying PICMAES2 in a reaching task on a simulated arm, we observe two developmental properties. First, we show how PICMAES2 autonomously and continuously tunes the global exploration/exploitation tradeoff, allowing it to re-adapt to changing tasks. Second, we show how PICMAES2 spontaneously self-organizes a maturational structure whilst exploring the degrees-of-freedom (DOFs) of the motor space. In particular, it automatically demonstrates the so-called proximo-distal maturation observed in humans: after first freezing distal DOFs while exploring predominantly the most proximal DOF, it progressively frees exploration in DOFs along the proximo-distal body axis. These emergent properties suggest the use of PICMAES2 as a general tool for studying reinforcement learning of skills in life-long developmental learning contexts.
Keywords :
adaptive control; learning (artificial intelligence); robots; DOF; adaptive exploration; degrees-of-freedom; emergent proximo distal maturation; motor space; reinforcement learning algorithm; robot learning; structured exploration mechanisms; Acceleration; Covariance matrix; Eigenvalues and eigenfunctions; Joints; Robots; Trajectory; Vectors;
Conference_Titel :
Development and Learning and Epigenetic Robotics (ICDL), 2012 IEEE International Conference on
Conference_Location :
San Diego, CA
Print_ISBN :
978-1-4673-4964-2
Electronic_ISBN :
978-1-4673-4963-5
DOI :
10.1109/DevLrn.2012.6400586