A Novel Iterative $\theta$ -Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems

This paper is concerned with a new iterative θ-adaptive dynamic programming (ADP) technique to solve optimal control problems of infinite horizon discrete-time nonlinear systems. The idea is to use an iterative ADP algorithm to obtain the iterative control law which optimizes the iterative performance index function. In the present iterative θ-ADP algorithm, the condition of initial admissible control in policy iteration algorithm is avoided. It is proved that all the iterative controls obtained in the iterative θ-ADP algorithm can stabilize the nonlinear system which means that the iterative θ-ADP algorithm is feasible for implementations both online and offline. Convergence analysis of the performance index function is presented to guarantee that the iterative performance index function will converge to the optimum monotonically. Neural networks are used to approximate the performance index function and compute the optimal control policy, respectively, for facilitating the implementation of the iterative θ-ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the established method.