Data-driven design of two-degree-of-freedom controllers using reinforcement learning techniques

Motivated by the successful application for feedback control, this study extends the study of reinforcement learning techniques to the design of two-degree-of-freedom controllers in the data-driven environment. Based on the residual generator based form of Youla parameterisation, all stabilising controllers are first interpreted in the feedback-feedforward situation with a Kalman filter-based residual generator acting as the core part. For the reference tracking problem, further discussions are conducted from the regulatory perspective and using the Q learning, recursive least squares methods and the policy iteration algorithm. The entire design is carried out as a two-stage process that separately achieves the optimal feedback and feedforward controllers. Finally, the effectiveness of the proposed approach is demonstrated with its application in the laboratory continuous stirred tank heater process.