Implicit sampling for path integral control

The applicability and usefulness of implicit sampling in stochastic optimal control is explored. The basic idea is to solve the stochastic Hamilton-Jacobi-Bellman equation with a Monte Carlo solver. This approach avoids the need for a grid of the domain (which is infeasible for problems of moderate dimension), however the sampling must be done carefully or else the Monte Carlo approach also becomes prohibitively expensive. Implicit sampling is a recently-developed variationally-enhanced Monte Carlo sampling method which is shown here to be efficient for a class of stochastic control problems. The theory is illustrated with examples.