Adaptive critic neural network-based controller for nonlinear systems

A novel multilayer neural network (NN) controller in discrete-time is designed to deliver a desired tracking performance for a class of nonlinear systems. A reinforcement learning scheme in discrete-time is proposed for the NN controller, where the learning is performed based on a certain performance measure, which is supplied from a critic. In other words, the critic conveys much less information than the desired output required in supervisory learning. Nevertheless, their ability to generate correct control actions makes adaptive critics prime candidates. The adaptive generating NN in the adaptive critic NN controller approximates the dynamics of the nonlinear system whereas a second NN based critic generates a signal, which is used to tune the weights of the action generating NN. Using the Lyapunov approach, the uniform ultimate boundedness. (UUB) of the closed-loop tracking error and weight estimates are shown by using novel weight updates. The adaptive critic NN does not require an offline learning phase and the weights can be initialized at zero or randomly. Simulation results verify the theoretical conclusions.