Title :
Stability and singular perturbations in constrained Markov decision problems
Author :
Altman, Eitan ; Gaitsgory, Vladimir A.
Author_Institution :
INRIA, Valbonne, France
fDate :
6/1/1993 12:00:00 AM
Abstract :
Constrained Markov decision problems (CMDPs) with the average cost criterion and a single ergodic chain, or the discounted cost with a general multichain structure, are considered. Conditions for stability of the optimal value and control to changes of the parameters of the problem, such as immediate costs, transition probabilities, and the discount factor, are established. Singular constrained problems, for which the optimal value and controls exhibit discontinuities, are studied
Keywords :
Markov processes; decision theory; optimisation; perturbation techniques; probability; stability; average cost criterion; constrained Markov decision problems; discount factor; discounted cost; immediate costs; multichain structure; optimisation; single ergodic chain; singular perturbations; stability; transition probabilities; Adaptive control; Constraint theory; Cost function; Linear programming; Optimal control; Parameter estimation; Stability criteria; State-space methods; Sufficient conditions; Telecommunication control;
Journal_Title :
Automatic Control, IEEE Transactions on
