Simulation to discover structure in optimal dynamic control policies

Simulation is known to be a powerful technique to analyze the dynamic behavior of a variety of systems. Dynamic programming is a technique to solve sequential decision making problems. The numerical solution of a dynamic program is a table with an optimal decision for all feasible states of the system. Such a table is usually complex as it lists all states, including those that are not so likely to be visited. In the paper we show how simulation helps in solving the problem efficiently and effectively. For cases on fishery management and inventory control, we show that the combination of simulation and dynamic programming results in a better understanding of the structure of an optimal policy. The insights obtained from simulation help in bounding the state space to speed up the solution process, and have resulted in the discovery a new class of ordering policies.