Adaptive optimal control for linear discrete time-varying systems

In this paper, adaptive optimal control is proposed for linear discrete time-varying (LDTV) systems subject to unknown system dynamics. The idea of the method is a direct application of the Q-learning adaptive dynamic programming for time-varying systems. In order to derive the optimal control policy, an actor-critic structure is constructed and the time-varying least square method is adopted for parameter adaptation. The derived control policy robustly stabilizes the time-varying system and guarantees an optimal control performance. As no particular system information is required throughout the process, the proposed method provides a feasible solution to a large variety of applications. The validity of the proposed method is verified through simulation studies.