Online adaptation of controller parameters based on approximate dynamic programming

Controller parameter tuning is an integral part of control engineering practice. Existing tuning methods usually start with an accurate mathematical model of the controlled system, which may pose some challenges for practicing engineers dealing with real systems. As such, parameter optimization and adaptation are treated as two independent steps during tuning. To address these issues, we propose a new, online parameterized controller tuning method for a general nonlinear dynamic system. This tuning method is based on direct heuristic dynamic programming (direct HDP), a model-free algorithm in the approximated dynamic programming (ADP) family. By using a Lyapunov stability approach, we provide uniformly ultimately bounded (UUB) results under some mild conditions for controller parameters, the critic neural network weights, and the action neural network weights. Simulation studies based on the benchmark cart-pole system demonstrate adaptability and optimization capabilities of the proposed controller parameter tuning method.