Stochastic optimal control design for nonlinear networked control system via neuro dynamic programming using input-output measurements

Neuro dynamic programming (NDP) techniques for optimal control of nonlinear network control system (NNCS) are not addressed in the literature. Therefore, in this paper, a novel NNCS representation incorporating the unknown system uncertainties and network imperfections is introduced first by using input and output measurements. Then, an online neural network (NN) identifier is introduced to estimate the control coefficient matrix. Subsequently, the critic NN and action NN are employed along with the NN identifier to determine the forward-in-time, time-based stochastic optimal control of NNCS without using value and policy iterations. Instead, value function and control inputs are updated at every sampling instant. Lyapunov theory is used to show that all the closed-loop signals and NN weights are uniformly ultimately bounded (UUB) while the approximated control input converges close to its target value with time.