Accelerated reinforcement learning control using modified CMAC neural networks

Reinforcement learning is a class of model-free learning control methods that can solve Markov decision problems. One difficulty for the application of reinforcement learning control is its slow convergence, especially in MDPs with continuous state space. In this paper, a modified structure of CMAC neural networks is proposed to accelerate reinforcement learning control. The modified structure is designed by incorporating a priori knowledge of learning control problems so that the efficiency and generalization ability of reinforcement learning can be improved. Simulation results on the cart-pole balancing problem illustrate the effectiveness of the proposed method.