DocumentCode :
3059284
Title :
An optimization-oriented approach to the adaptive control of Markov chains
Author :
Milito, R.A. ; Cruz, J.
Author_Institution :
University of Illinois, Urbana, Illinois
fYear :
1984
fDate :
12-14 Dec. 1984
Firstpage :
837
Lastpage :
838
Abstract :
We consider the control of a dynamic system modeled as a Markov chain. The transition probability matrix of the Markov chain depends on the control u and also on an unknown parameter α. The unknown parameter belongs to a given finite set A. The long run average cost depends on the control policy and the unknown parameter. Thus a direct approach to the optimization of the performance is not feasible. A common procedure calls for an on-line estimation of the unknown parameter and the minimization of the cost functional using the estimate in lieu of the true parameter. It is well-known that this "certainty equivalence" (CE) solution may fail to yield the optimal performance. This motivates the presentation of a new optimiza tion-oriented approach to adaptive control. We consider a composite functional which simultaneously take care of theestimation and control needs. The global minimum of this composite functional coincides with the minimum of the original cost functional. Thus its joint minimization with respect to control and parameter estimates would yield the optimal control policy This joint minimization is not feasible, but it suggests an algorithm that asymptotically achieves the desired goal. Due to space constraints we omit a review of the literature as well as the proofs of our claims. They will be presented elsewhere.
Keywords :
Adaptive control; Markov processes;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Decision and Control, 1984. The 23rd IEEE Conference on
Conference_Location :
Las Vegas, Nevada, USA
Type :
conf
DOI :
10.1109/CDC.1984.272125
Filename :
4048001
Link To Document :
بازگشت