A stable Lyapunov constrained reinforcement learning based neural controller for non linear systems

This paper proposes a Lyapunov constrained neural network based reinforcement learning (RL) controller with guaranteed stability for non linear systems. Neural networks have been used as universal function approximators to deal with one of the core problem faced in RL commonly known as `The Curse of Dimensionality´. We propose to constrain controller action set to the one dictated by the Lyapunov stability theory to produce a controller with guaranteed stability. We prove that when the controller action set is constrained the cost function turns out to be a Lyapunov candidate function thereby guaranteeing stability of the controller. Proposed methodology has been applied to the benchmark inverted pendulum (IP) balancing problem to validate its effectiveness. Simulation results and comparison against baseline neural Q learning control brings out the effectiveness and viability of the proposed control scheme.