Stochastic Optimal Controller Design for Uncertain Nonlinear Networked Control System via Neuro Dynamic Programming

The stochastic optimal controller design for the nonlinear networked control system (NNCS) with uncertain system dynamics is a challenging problem due to the presence of both system nonlinearities and communication network imperfections, such as random delays and packet losses, which are not unknown a priori. In the recent literature, neuro dynamic programming (NDP) techniques, based on value and policy iterations, have been widely reported to solve the optimal control of general affine nonlinear systems. However, for realtime control, value and policy iterations-based methodology are not suitable and time-based NDP techniques are preferred. In addition, output feedback-based controller designs are preferred for implementation. Therefore, in this paper, a novel NNCS representation incorporating the system uncertainties and network imperfections is introduced first by using input and output measurements for facilitating output feedback. Then, an online neural network (NN) identifier is introduced to estimate the control coefficient matrix, which is subsequently utilized for the controller design. Subsequently, the critic and action NNs are employed along with the NN identifier to determine the forward-in-time, time-based stochastic optimal control of NNCS without using value and policy iterations. Here, the value function and control inputs are updated once a sampling instant. By using novel NN weight update laws, Lyapunov theory is used to show that all the closed-loop signals and NN weights are uniformly ultimately bounded in the mean while the approximated control input converges close to its target value with time. Simulation results are included to show the effectiveness of the proposed scheme.