Iterative Learning Heuristic Dynamic Programming (ILHDP) design of a Steam Power Plant Controller

This paper presents a new dynamic programming method called the Iterative Learning Heuristic Dynamic Programming (ILHDP). The ILHDP is an Iterative Learning Control (ILC) based Neural Dynamic Programming (NDP) algorithm. The NDP aspect of the ILHDP algorithm is borrowed from traditional Adaptive Critic Design (ACD) algorithms. Typical NDP algorithms in the ACD class of algorithms train a Model Network beforehand and use a Critic Network, as the gradient approximator, trained back-and-forth with the Action Network in each iteration to converge the Action Network towards the optimal control policy. The proposed ILHDP algorithm updates the Model Network continually based on newly obtained data sampled during each Action Network optimization step on the same experiment. This process of Model Network updation ensures better gradient approximation presented by the Model Network itself. The presented ILHDP is used for the design of a Steam Power Plant controller with respect to the Active-Power-to-Frequency droop characteristics. Test results indicated that the ILHDP designed controller was capable of stabilizing the output power of the Steam Power Plant to track the load with a maximum tracking error of 0.011 for abrupt load changes as fast as 15s. The Steam Power Plant was also subjected to large transient spikes for which the designed controller proved to recover the system back to stability.