DocumentCode :
3068928
Title :
Discrete-time equivalence for constrained semi-Markov decision processes
Author :
Beutler, F.J. ; Ross, K.W.
Author_Institution :
The University of Michigan
fYear :
1985
fDate :
11-13 Dec. 1985
Firstpage :
1122
Lastpage :
1123
Abstract :
A continuous-time average reward Markov decision process problem is most easily solved in terms of an equivalent discrete-time Markov decision process (DMDP); customary hypotheses include that the process is a Markov jump process with denumerable state space and bounded transition rates, that actions are chosen at the jump points of the process, and that the policies considered are deterministic. We derive an analogous uniformization result applicable to semi-Markov decision processes (SMDP) under a (possibly) randomized stationary policy. For each stationary policy governing an SMDP meeting certain hypotheses, we specify a past-dependent policy on a suitably constructed DMDP; the new policy carries the same average reward on the DMDP as the original policy on the SMDP. Discrete time reduction is applied to optimization on a SMDP subject to a hard constraint, for which the optimal policy has been shown to be stationary and possibly randomized at no more than a single state. Under some convexity conditions on the reward, cost, and action space, it is shown that a non-randomized policy is optimal for the constrained problem.
Keywords :
Constraint optimization; Cost function; Dynamic programming; Equations; Extraterrestrial measurements; Markov processes; Optimal control; Random variables; State-space methods; Time measurement;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Decision and Control, 1985 24th IEEE Conference on
Conference_Location :
Fort Lauderdale, FL, USA
Type :
conf
DOI :
10.1109/CDC.1985.268676
Filename :
4048476
Link To Document :
بازگشت