A forward method for optimal stochastic nonlinear and adaptive control

A computational approach is taken to solve the optimal partially observed nonlinear stochastic control problem. The approach is to systematically solve the stochastic dynamic programming equations forward in time, using a nested stochastic approximation technique. Although computationally intensive, this provides a straightforward numerical solution for this class of problems and provides an alternative to the usual `curse of dimensionality´ associated with solving the dynamic programming equation backwards in time. In particular, the `curse´ is seen to take a new form, where the amount of computation depends on the amount of uncertainty in the problem and the length of the horizon. As a matter of more practical interest, it is shown that the cost degrades monotonically as the complexity of the algorithm is reduced. This provides a strategy for suboptimal control with clear performance/computation trade-offs. A numerical study focusing on a generic optimal stochastic adaptive control example is included to demonstrate the feasibility of the method