Robust Markov Decision Processes using Sigma Point sampling

This paper presents a new robust decision making algorithm that accounts for model uncertainty in finite state/action, Markov Decision Processes (MDPs). In particular we generate robust and optimal control policies using Sigma Point sampling methods for dynamic multi-stage problems where the probabilistic transition model of the MDP may be fixed, but uncertain. In the case of poorly known transition model governing a MDP, this paper shows that the total number of scenarios in a scenario-based robust optimization may be decreased by generating a small number of appropriately chosen samples of the model. The robust policy for the worst- case instance of the data can be approximated by identifying the minimum objective function obtained from these realizations. This paper compares the proposed approach to more direct sampling-based approaches in a machine repair problem. The numerical examples show reduction in the total number of simulations required to obtain robust solutions while achieving optimal results.