Balance control of robot with CMAC based Q-learning

Self-balancing two-wheel robot is a high order, multi-variable, nonlinear, strong-coupling and absolutely unstable system. A reinforcement learning algorithm based on many parallel Cerebellar Model Articulation Controller (CMAC) neural networks is proposed for the balance-control problem of self-balancing two-wheel robot. In the method, the outputs of CMAC are used to approximate the Q-functions of the input state variables. The input state variables are divided to decrease the grades of quantization. Therefore, the storage spaces of CMAC are reduced effectively, and the learning rate and control precision of Q-algorithm are improved. At the same time, the generalization of continuous state variables is realized too. The method is applied to solve the balance control problem of self-balancing two-wheel robot, and the simulation results show its correctness and efficiency.