Adaptive critic designs

Author

Prokhorov, Danil V. ; Wunsch, Donald C., II

Author_Institution

Dept. of Electr. Eng., Texas Tech. Univ., Lubbock, TX, USA

Volume

8

Issue

5

fYear

1997

fDate

9/1/1997 12:00:00 AM

Firstpage

997

Lastpage

1007

Abstract

We discuss a variety of adaptive critic designs (ACDs) for neurocontrol. These are suitable for learning in noisy, nonlinear, and nonstationary environments. They have common roots as generalizations of dynamic programming for neural reinforcement learning approaches. Our discussion of these origins leads to an explanation of three design families: heuristic dynamic programming, dual heuristic programming, and globalized dual heuristic programming (GDHP). The main emphasis is on DHP and GDHP as advanced ACDs. We suggest two new modifications of the original GDHP design that are currently the only working implementations of GDHP. They promise to be useful for many engineering applications in the areas of optimization and optimal control. Based on one of these modifications, we present a unified approach to all ACDs. This leads to a generalized training procedure for ACDs

Keywords

adaptive control; backpropagation; duality (mathematics); dynamic programming; generalisation (artificial intelligence); heuristic programming; neurocontrollers; adaptive critic designs; backpropagation; dynamic programming; generalizations; generalized training procedure; globalized dual heuristic programming; neural nets; neurocontrol; optimal control; optimization; reinforcement learning; Adaptive control; Backpropagation algorithms; Dynamic programming; Hebbian theory; Neural engineering; Neural networks; Optimal control; Programmable control; Supervised learning; Working environment noise;

fLanguage

English

Journal_Title

Neural Networks, IEEE Transactions on

Publisher

ieee

ISSN

1045-9227

Type

jour

DOI

10.1109/72.623201

Filename

623201