An actively adaptive control for linear systems with random parameters via the dual control approach

A new method is presented for controlling a discrete-time linear system with possibly time-varying random parameters in the presence of input and output noise. The cost is assumed to be quadratic in the state and control. Previous algorithms for the above problem when the system had both zeros and poles unknown were of the open-loop feedback type, i.e., they did not take into account that future observations will be made. Therefore, even though these schemes were adaptive, their learning was "accidental." In contrast to this, the new approach uses an expression of the optimal cost-to-go that exhibits the dual purpose of the control, i.e., learning and control. The effect of the present control on the future estimation ("learning") appears explicitly in the cost used in the stochastic dynamic programming equation. The resulting sequence of controls, which is of the closed-loop type, is shown via simulations to appropriately divide its energy between the learning and the control purposes. Therefore, this control is called actively adaptive because it regulates the speed and amount of learning as required by the performance index. The simulations on a third-order system with six unknown parameters also demonstrate the computational feasibility of the proposed algorithm.