Actor-critic neural network reinforcement learning for walking control of a 5-link bipedal robot

Today, researches on adaptive control have focused on bio-inspired learning techniques to deal with real-life applications. Reinforcement Learning (RL) is one of these major techniques, which has been widely used in robot control tasks recently. On the other hand, artificial neural networks are an accurate approximation tool in nonlinear robotic dynamic control tasks. In this paper, our main goal was to combine the advantages of the artificial neural networks and the RL to reduce the learning time length and enhance the control accuracy. Therefore, we have implemented one of the promising RL approaches, actor-critic RL to control the actuation torques of a planar five-link bipedal robot and retain the passive torso in the vertical position. Our control agent consists of two three-layered neural network units, known as the critic and the actor for learning prediction and learning control tasks. These units are synchronized by the temporal difference error, which implements the eligibility trace vector to assign credit or blame for the error. Moreover, since the neural networks are implemented in both of the actor and the critic sections, we have added a learning database to reduce the probability of inaccurate approximation of the nonlinear functions. Results of our presented control method reveal its perfect performance in stable walking control of the bipedal robot.