A general model for non-Markovian stochastic decision discrete-event systems

This paper extends previous work on modeling stochastic decision discrete-event systems (DDES) through a generalized semi-Markov decision process (GSMDP), which discards any restrictive unrealistic assumptions and can be applied to complex cases including non-Markovian environment. Moreover, as a typical example, we develop a GSMDP model for the optimal call admission control (CAC) problem in an integrated voice/data wireless network supporting multiple traffic types with different resource requirements. In contrast to existing methods, this approach can better model the real world of the next generation wireless network behaviors. Besides, through a form of reinforcement learning algorithm known as Q-learning, we can solve the Bellman optimality equation with requiring neither explicit state transition probabilities nor any assumptions behind the network model.