Adaptive dynamic programming for a class of discrete-time non-affine nonlinear systems with time-delays

In this paper, an optimal control scheme for a class of non-affine nonlinear systems with time-delays in state and control variables is developed using a new iterative adaptive dynamic programming (ADP) algorithm. By introducing delay matrix functions, the explicit expression of optimal control is solved using the dynamic programming theory and the optimal control can iteratively be solved using the present technique. Convergence analysis is presented to show the performance index function to reach the optimum by the present method. Neural networks are used to approximate the performance index function, compute the optimal control policy, solve delay matrix functions and model the nonlinear system, respectively, for facilitating the implementation of the iterative ADP algorithm. A simulation example is given to demonstrate the validity of the present optimal control scheme.