Model-free H∞ stochastic optimal design for unknown linear networked control system zero-sum games via Q-learning

In this paper, stochastic optimal strategy for unknown linear networked control system (NCS) quadratic zero-sum games related to H_∞ optimal control in the presence of random delays and packet losses is solved in forward-in-time manner. This approach does not require the knowledge of the system matrices since it uses Q-learning. The proposed stochastic optimal control approach, referred as adaptive dynamic programming (ADP), involves solving the action dependent Q-function Q(z, u, d) of the zero-sum game instead of solving the state dependent value function J(z) which satisfies a corresponding Game Theoretic Riccati equation (GRE). An adaptive estimator (AE) is proposed to learn the Q-function online and value and policy iterations are not needed unlike in traditional ADP schemes. Update laws for tuning the unknown parameters of adaptive estimator (AE) are derived. Lyapunov theory is used to show that all signals are asymptotic stable (AS) and that the approximated control and disturbance signals converge to optimal control and disturbance inputs. Simulation results are included to show the effectiveness of the scheme.