Performance sensitivity analysis and optimization for a class of countable semi-Markov decision processes

Markov process and semi-Markov process are widely used in many practical engineering problems. Based on the potential theory, the problems of sensitivity analysis of a class of countable Markov decision processes is given in this paper. Based on this, the corresponding problems are investigated for a class of semi-Markov decision processes. The existence of the optimality equations are proved for both countable Markov decision process and countable semi-Markov decision process respectively.