Control parallel double inverted pendulum by hierarchical reinforcement learning

Realizing balance control of single inverted pendulum without mathematic model by reinforcement learning has gained great success. However, further applying it to the multilevel inverted pendulum faces problems such as curse of dimensionality and difficulty in convergence. In this paper, we propose a hierarchical reinforcement learning algorithm to control parallel double inverted pendulum. Firstly, control of single inverted pendulum is learned by Q-learning algorithm. Then the learned control states of single inverted pendulum are used to direct control process of parallel double inverted pendulum so that the double one is controlled.