Eigenfunction approximation methods for linearly-solvable optimal control problems

We have identified a general class of nonlinear stochastic optimal control problems which can be reduced to computing the principal eigenfunction of a linear operator. Here we develop function approximation methods exploiting this inherent linearity. First we discretize the time axis in a novel way, yielding an integral operator that approximates not only our control problems but also more general elliptic PDEs. The eigenfunction problem is then approximated with a finite-dimensional eigenvector problem - by discretizing the state space, or by projecting on a set of adaptive bases evaluated at a set of collocation states. Solving the resulting eigenvector problem is faster than applying policy or value iteration. The bases are adapted via Levenberg-Marquardt minimization with guaranteed convergence. The collocation set can also be adapted so as to focus the approximation on a region of interest. Numerical results on test problems are provided.