Adaptive dynamic programming algorithm for discrete-time systems with ∈-error bound and discount factor in the performance cost

In this paper, we present our work on dynamic programming, which is referred as epsi-adaptive dynamic programming, for discrete-time system with discount factor 0< gamma <1 in the performance cost. A single controller, epsi-optimal controller , which is determined from an optimal cost V_epsi*, is obtained to approximate the optimal controller. The optimal controller mu_epsi* can always control the state to approach the equilibrium state, while the performance cost is close to the biggest lower bound of all performance costs within an error according to epsi. An algorithm for finding the optimal controller is developed and numerical experiments are given to illustrate the performance of the algorithm.