Title :
Risk-constrained Markov decision processes
Author :
Borkar, Vivek ; Jain, Rahul
Author_Institution :
Sch. of Technol. & Comput. Sci., Tata Inst. of Fundamental Res. (TIFR) Mumbai, Mumbai, India
Abstract :
We propose a new constrained Markov decision process framework with risk-type constraints. The risk metric we use is Conditional Value-at-Risk (CVaR), which is gaining popularity in finance. It is a conditional expectation but the conditioning is defined in terms of the level of the tail probability. We propose an iterative offline algorithm to find the risk-contrained optimal control policy. A stochastic approximation-inspired `learning´ variant is also sketched.
Keywords :
Markov processes; approximation theory; iterative methods; optimal control; stochastic programming; Markov decision process; conditional value-at-risk; iterative offline algorithm; optimal control; risk-type constraints; stochastic approximation-inspired learning variant; tail probability; Approximation algorithms; Approximation methods; Convergence; Markov processes; Optimization; Yttrium; Constrained Markov decision processes; Risk measures; Stochastic Approximations;
Conference_Titel :
Decision and Control (CDC), 2010 49th IEEE Conference on
Conference_Location :
Atlanta, GA
Print_ISBN :
978-1-4244-7745-6
DOI :
10.1109/CDC.2010.5718076