Numerical Algorithms for Optimal Controls for Nonlinear Stochastic Systems With Delays

The Markov chain approximation numerical methods were extended to general controlled nonlinear (and reflected) diffusions with delays in a recent monograph, where many types of ideal algorithms were developed and convergence proved. Any combination of the path, control, and reflection terms can be delayed. If the control and/or reflection terms are delayed, then the memory requirements with naive algorithms can be huge. Recasting the problem in terms of a “wave equation” yields algorithms with considerably reduced memory needs. The reference concentrated on theoretical issues. Here, we are concerned with adapting the ideal to applications, and present data showing that the method can work well.