Game Theoretic continuous time Differential Dynamic Programming

In this work, we derive a Game Theoretic Differential Dynamic Programming (GT-DDP) algorithm in continuous time. We provide a set of backward differential equations for the value function expansion without assuming closeness of the initial nominal control to the optimal control solution, and derive the update law for the controls. We introduce the GT-DDP algorithm and analyze the effect of the game theoretic formulation in the feed-forward and feedback parts of the control policies. Furthermore, we investigate the performance of GT-DDP through simulations on the inverted pendulum with conflicting controls and we apply the control gains on a stochastic system to demonstrate the effect of the design of the cost function to the feed-forward and feedback parts of the control policies. Finally, we conclude with some possible future directions.