Reinforcement learning based neuro-control systems for an unmanned helicopter

This paper concerns with the autonomous flight control system of an unmanned helicopter, which is combined with reinforcement learning based neuro-controller. We assume that PID (proportional-integral-derivative) type, linear feedback controller is predesigned and it can stabilize the system with limited performance. The conservative control behavior is improved by the synthesis of the poor feedback controller and the neuro-controller. Actor-critic learning architecture is adopted as a learning agent. Actor network consists of feed-forward neural network and critic network is approximated with a tabular function approximator. The Q-value based critic network is trained via SARSA algorithm which is a variant of reinforcement learning. Several demonstrations are performed with a simple first-order system. Furthermore, the proposed neuro-control system is applied to an unmanned helicopter known as a highly nonlinear and complex system and the simulation results are presented.