Finite-horizon input-constrained nonlinear optimal control using single network adaptive critics

A single neural network based controller called the Finite-SNAC is developed in this study to synthesize finite-horizon optimal controllers for nonlinear control-affine systems. For satisfying the constraint on the input, a non-quadratic cost function is used. Inputs to the neural network are the current system states and the time-to-go and the network outputs are the costates which are used to compute the feedback control. Convergence of the reinforcement learning based training method to the optimal solution, the training error and the network weights are provided. The resulting controller is shown to solve the associated time-varying Hamilton-Jacobi-Bellman (HJB) equation and provide the fixed-final-time optimal solution. Performance of the new synthesis technique is demonstrated through an attitude control problem wherein a rigid spacecraft performs a finite time attitude maneuver subject to control bounds. The new formulation has a great potential for implementation since it consists of only one neural network with single set of weights and it provides comprehensive feedback solutions online though it is trained offline.