Constrained online optimal control for continuous-time nonlinear systems using neuro-dynamic programming

This paper develops an online adaptive optimal control scheme to solve the infinite-horizon optimal control problem of continuous-time nonlinear systems with control constraints. A novel architecture is presented to approximate the Hamilton-Jacobi-Bellman equation. That is, only a critic neural network is used to derive the optimal control instead of typical action-critic dual networks employed in neuro-dynamic programming methods. Meanwhile, unlike existing tuning laws for the critic, the newly developed critic update rule not only ensures convergence of the critic to the optimal control but also guarantees the closed-loop system to be uniformly ultimately bounded. In addition, no initial stabilizing control is required. Finally, an example is provided to verify the effectiveness of the present approach.