New directions in ACDs: keys to intelligent control and understanding the brain

Adaptive critic designs (ADC) are sometimes called reinforcement learning systems, approximate dynamic programming or neurodynamic programming. Applications to cars and missiles confirm their advantages over older methods and their ability to overcome the “curse of dimensionality” in dynamic programming. Some ACDs can be formulated as new designs for “adaptive control” which, unlike classical neurocontrol designs, offer universal stability and fast response to transient disturbances. The author has argued that functional understanding of intelligence in mammal brains requires a model of the brain as an ACD. This paper reviews this field, and the new research needed to replicate mammal-like intelligence and beyond