∈-Adaptive Dynamic Programming for discrete-time systems

Dynamic programming for discrete-time systems is difficult due to the ldquocurse of dimensionalityrdquo: one has to find a series of control actions that must be taken in sequence. This sequence will lead to the optimal performance cost, but the total cost of those actions will be unknown until the end of that sequence. In this paper, we present our work on dynamic programming for discrete-time system, which is referred as epsiv-adaptive dynamic programming. A single controller, epsiv-optimal controller u*_(epsiv) is determined from an epsiv-optimal cost J*_(epsiv) is given to approximate the optimal controller. The epsiv-optimal controller u*_(epsiv) can always control the state to approach to the equilibrium state, while the performance cost is close to the biggest lower bound of all performance costs within an error according to epsiv.