Potential based policy gradient optimization algorithm for a class of stochastic nonlinear systems

In this paper, a potential based policy gradient algorithm is presented for infinite horizon optimal control of a class of stochastic nonlinear systems, which is with continuous state space and unknown stochastic noise. First, it is shown that the optimal control problem can be transformed into a Markov decision process. Then, the potential based performance derivative formula is developed for continuous state space. For estimating potential function and state transition density function, RBF neural network and kn-Nearest Neighbor technique are applied. Thus, the system performance gradient with respect to the control parameters can be estimated from a sample path. Finally, the simulation shows the effectiveness of the algorithm.