Idempotent expansions for continuous-time stochastic control: compact control space

It is now well-known that many classes of deterministic control problems may be solved by max-plus or min plus (more generally, idempotent) numerical methods. The first such methods for stochastic control were developed only for discrete-time problems. The key tools enabling their development were the idempotent distributive property and the fact that certain solution forms are retained through application of the semigroup operator In particular, under certain conditions, pointwise minima of affine and quadratic forms pass through this operator. As the operator contains an expectation component, this requires application of the idempotent distributive property. In the case of finite sums and products, this property looks like our standard algebra distributive property; in the infinitesimal case, it is familiar to control theorists through notions of strategies, non-anticipative mappings and/or progressively measurable controls. Using this technology, the value function can be propagated backwards with a representation as a pointwise minimum of quadratic or affine forms. Here, we will remove the severe restriction to discrete-time problems. This extension requires overcoming significant technical hurdles. First, note that as these methods are related to the max-plus curse-of-dimensionality-free methods of deterministic control, there will be a discretization over time, but not over space. We will first define a parameterized set of operators, approximating the dynamic programming operator. We obtain the solutions to the problem of backward propagation by repeated application of the approximating operators. These solutions are parameterized by the time discretization step size. Using techniques from the theory of viscosity solutions, we show that the solutions converge to the viscosity solution of the Hamilton-Jacobi-Bellman partial differential equation (HJB PDE) associated with the original problem.