Title :
The policy iteration algorithm for average continuous control of piecewise deterministic Markov processes
Author :
Costa, O.L.V. ; Dufour, F.
Author_Institution :
Dept. de Eng. de Telecomun. e Controle, Escola Politec. da Univ. de Sao Paulo, Sao Paulo, Brazil
Abstract :
The main goal of this paper is to apply the so-called policy iteration algorithm (PIA) for the long run average continuous control problem of piecewise deterministic Markov processes (PDMP´s) taking values in a general Borel space and with compact action space depending on the state variable. In order to do that we first derive some important properties for a pseudo-Poisson equation associated to the problem. In the sequence it is shown that the convergence of the PIA to a solution satisfying the optimality equation holds under some classical hypotheses and that this optimal solution yields to an optimal control strategy for the average control problem for the continuous-time PDMP in a feedback form.
Keywords :
Markov processes; Poisson equation; continuous time systems; feedback; iterative methods; optimal control; average continuous control; continuous-time PDMP; feedback form; general Borel space; optimal control strategy; optimality equation; piecewise deterministic Markov processes; policy iteration algorithm; pseudo-Poisson equation; Brazil Council; Costs; Differential equations; Feedback; Markov processes; Motion control; Optimal control; Poisson equations; Q measurement; Stochastic processes;
Conference_Titel :
Decision and Control, 2009 held jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009. Proceedings of the 48th IEEE Conference on
Conference_Location :
Shanghai
Print_ISBN :
978-1-4244-3871-6
Electronic_ISBN :
0191-2216
DOI :
10.1109/CDC.2009.5400773