Neural approximations for infinite-horizon optimal control of nonlinear stochastic systems

A feedback control law is proposed that drives the controlled vector v_t of a discrete-time dynamic system to track a reference v_t* over an infinite time horizon, while minimizing a given cost function. The behavior of v_t* over time is completely unpredictable, Random noises act on the dynamic system and the state observation channel, which may be nonlinear. It is assumed that all such random vectors are mutually independent, and that their probability density functions are known. So general a non-LQG optimal control problem is very difficult to solve. The proposed solution is based on three main approximating assumptions: 1) the problem is stated in a receding-horizon framework where v_t* is assumed to remain constant within a shifting-time window; 2) the control law is assigned a given structure (that of a multilayer feedforward neural net) in which a finite number of parameters have to be determined so as to minimize the cost function; and 3) the control law is given a limited memory, which prevents the amount of data to be stored from increasing over time. Errors resulting from the second and third assumptions are discussed, Due to the very general assumptions under which the control law is derived, we are not able to report stability results. However, simulation results show that the proposed method may constitute an effective tool for solving, to a sufficient degree of accuracy, a wide class of control problems traditionally regarded as difficult ones. An example of freeway traffic optimal control is given