The application of Bayesian statistics to the control of noisy extremum systems

The Bayesian estimation scheme has proved widely useful in providing estimates of inexactly observable parameters and state variables of noisy linear discrete-time systems for the derivation of an optimal control policy. Controllers may be derived which are adaptive and relatively insensitive to system parameter changes. In this paper dual control schemes are developed for a class of discrete-time noisy extremum systems given by the state equations x(t+1) = ??x(t) + Bu(t) + w(t) (1) z1 (t) = - + v1(t) (2) The optimal control policy is defined as that which minimizes ??t=0 T-1 E {x(t+1)tx(t+1)} The control scheme is intentionally sub-optimal to reduce computation to a feasible level. The technique is to reduce Equation (2) by a series of approximations to a set of linear equations. A set of control schemes may be set up based on the Bayesian estimation approach and the set of linearized equations. An algorithm for choosing at each time interval which of the control schemes is to be used to calculate the control is developed such that it satisfies certain performance criteria. The results of extensive computer studies of one-and two-dimensional extremum systems are presented.