Adaptive dynamic programming with stable value iteration algorithm for discrete-time nonlinear systems

In this paper, a new stable value iteration adaptive dynamic programming (ADP) algorithm, named “θ-ADP” algorithm, is proposed for solving the optimal control problems of infinite horizon discrete-time nonlinear systems. By introducing a parameter θ in the iterative ADP algorithm, it is proved that any of iterative control obtained in the proposed algorithm can stabilize the nonlinear system which overcomes the disadvantage of traditional value iteration algorithms. Neural networks are used to approximate the performance index function and compute the optimal control policy, respectively, for facilitating the implementation of the iterative θ-ADP algorithm. Finally, a simulation example is given to illustrate the performance of the proposed method.