Solving sequential decision-making problems under virtual reality simulation system

A large class of problems of sequential decision-making can be modeled as Markov or semi-Markov decision problems, which can be solved by classical methods of dynamic programming. However, the computational complexity of classical MDP algorithms, such as value iteration and policy iteration, is prohibitive and will grow intractably with problem size. Furthermore, they require for each action the one step transition probability and reward matrices, which is often unrealistic for large and complex systems. We provide the decision-maker with a sequential decision-making environment by establishing a virtual reality simulation system, where the uncertainty property of the system can also be shown. In order to obtain the optimal or near optimal policy of the sequential decision problem, simulation optimization algorithms as infinitesimal perturbation analysis are applied to complex queuing systems. We present a detailed study of this method in the sequential decision-making problem in the Boeing-737 assembling process