Optimal Controller Design Algorithm For Non-Affine in Input Discrete-Time Nonlinear System

Convergence is proven of the value-iteration-based algorithm to find the optimal controller in the case of general non-affine in input nonlinear systems. That is, it is shown that algorithm converges to the optimal control and the optimal value function. It is assumed that at each iteration the value and action update equations can be exactly solved. Then two standard neural networks (NN) are used: a critic NN is used to approximate the value function while an action network is used to approximate the optimal control policy.