Generalized adaptive dynamic programming algorithm for discrete-time nonlinear systems: Convergence and stability analysis

This paper is concerned with a generalized iterative adaptive dynamic programming (ADP) algorithm for discrete-time nonlinear systems. The idea is to use iterative ADP algorithm to obtain iterative control laws which make the iterative performance index function reach the optimum. It is proved that for an arbitrary positive semi-definite function, the iterative performance index functions will converge to the optimum. For different initial functions, it shows that the convergence procedures of the iterative performance index functions are different. Stability properties of the system are analyzed in this paper to show that the iterative control can stabilize the system for suitable initial performance index functions. Admissible control properties of the iterative control laws obtained by the present generalized iterative ADP algorithm are also given in this paper.