Model-free adaptive learning solutions for discrete-time dynamic graphical games

This paper introduces novel model-free adaptive learning algorithm to solve the dynamic graphical games in real-time. It allows online model-free tuning of the controller and critic networks. This algorithm solves the dynamic graphical game in a distributed fashion. Novel coupled Bellman equations and Hamiltonian functions are developed for the dynamic graphical games. Nash solution for the dynamic graphical game is given in terms of the solution to a set of coupled Hamilton-Jacobi-Bellman equations developed herein. An online model-free policy iteration algorithm is developed to learn the Nash solution for the dynamic graphical game in real-time. A proof of convergence for this algorithm is given under mild assumptions about the inter-connectivity properties of the graph.