On State Space Reduction in Sequential Decision Processes

This paper provides sufficient conditions when certain information about the past of a stochastic sequential decision processes can be ignored by a controller and when the problem state space can be reduced to a smaller set. We illustrate the results with particular applications to queueing control, control of semi-Markov decision processes with iid sojourn times, and uniformization of continuous-time Markov decision processes.