The optimizing-simulator: Merging simulation and optimization using approximate dynamic programming

There is a wide range of simulation problems that involve making decisions during the simulation, where we would like to make the best decisions possible, taking into account not only what we know when we make the decision, but also the impact of the decision on the future. Such problems can be formulated as dynamic programs, stochastic programs and optimal control problems, but these techniques rarely produce computationally tractable algorithms. We demonstrate how the framework of approximate dynamic programming can produce near-optimal (in some cases) or at least high quality solutions using techniques that are very familiar to the simulation community. The price of this challenge is that the simulation has to be run iteratively, using statistical learning techniques to produce the desired intelligence. The benefit is a reduced dependence on more traditional rule-based logic.