Reinforcement Learning with Multiple Heterogeneous Modules: A Framework for Developmental Robot Learning

Developmental learning approach by changing the internal state representation from simple to complex is promising in order for a robot to learn behaviors efficiently. We have proposed a reinforcement learning (RL) method for multiple learning modules with different state representations and algorithms. One of interesting results we showed is that a complex RL system can learn faster with the help of simpler RL systems that can not obtain the best performance. However, it did not consider the difference in sampling rates of learning modules. This paper discusses how the interaction among multiple learning modules with different sampling rates affects the robot learning. Experimental results in navigation task show that developmental learning described above is not always good strategy