A study of reinforcement learning for the robot with many degrees of freedom - acquisition of locomotion patterns for multi-legged robot

Reinforcement learning has recently been receiving much attention as a learning method for not only toy problems but also complicated systems such as robot systems. It does not need priori knowledge and has higher capability of reactive and adaptive behaviors. However, increasing of action-state space makes it difficult to accomplish the learning process. In most of the previous works, the application of the learning is restricted to simple tasks with a small action-state space. Considering this point, we present a new reinforcement learning algorithm: Q-learning with dynamic structuring of exploration space based on genetic algorithm. The algorithm is applicable to systems with high dimensional action and interior state spaces, for example, a robot with many redundant degrees of freedom. To demonstrate the effectiveness of the proposed algorithm simulations of locomotion patterns for a 12-leged robot were carried out. As the result, an effective behavior was obtained by using our proposed algorithm.