Robust Optimality for Discounted Infinite-Horizon Markov Decision Processes With Uncertain Transition Matrices

Author

Li, Baohua ; Si, Jennie

Author_Institution

Dept. of Electr. Eng., Arizona State Univ., Tempe, AZ

Volume

53

Issue

9

fYear

2008

Firstpage

2112

Lastpage

2116

Abstract

We study finite-state, finite-action, discounted infinite-horizon Markov decision processes with uncertain transition matrices in the deterministic policy space. The transition matrices are classified as either independent or correlated. A generalized robust optimality criterion which can be degenerated to some popular optimality criteria is proposed, under which an optimal or near-optimal policy exists for any uncertain transition matrix. Theorems are developed to guarantee a stationary policy being optimal or near-optimal in the deterministic policy space.

Keywords

Markov processes; decision theory; infinite horizon; matrix algebra; uncertain systems; deterministic policy space; discounted infinite-horizon Markov decision process; finite-state finite-action MDP; generalized robust optimality criterion; uncertain transition matrix; Cost function; Estimation error; Inventory control; Quality control; Robust control; Robustness; Space stations; Markov decision processes; robust optimality criterion; uncertain transition matrix;

fLanguage

English

Journal_Title

Automatic Control, IEEE Transactions on

Publisher

ieee

ISSN

0018-9286

Type

jour

DOI

10.1109/TAC.2008.930182

Filename

4639451