Robust control policy for semi-Markov decision processes with dependent uncertain parameters

Recent researches by Cao indicate that the concept of Markov performance potential plays an important role in the study of Markov or semi-Markov systems. If system parameters are known with certainty, many effective potential-based optimization methods may be developed for semi-Markov decision processes (SMDPs) by using the equivalent infinitesimal generator, which is defined by Cao for the first time and by Yin later in a different way. Unfortunately, some of the parameters are often difficult to derive or even slowly time-varying, which leads to the uncertainty of the equivalent infinitesimal transition rates. Under these cases, we focus on the solution of the optimal robust control policy for both average- and discounted-cost SMDPs with dependent parameters, which are represented as compact sets. Potential-based solution techniques such as gradient methods are discussed for designing robust decision schemes.