Online and model-free supplementary learning control based on approximate dynamic programming

An approximate dynamic programming (ADP) based supplementary learning control method is developed to online improve the performance of existing controllers. The proposed supplementary learning structure can make full use of the prior knowledge of the pre-designed controller and endow the controller with learning ability. Moreover, by introducing the action dependent value function for policy evaluation, the supplementary learning control can work in a model-free manner. The policy iteration algorithm is employed to train the actor-critic structure of the ADP supplementary controller. Simulation studies are carried out on the cart-pole system to validate the optimization and the adaptation capability of the proposed methodology.