Extremum or hill-climbing regulation: a statistical theory involving lags, disturbances and noise

Extremum or `hill-climbing¿ regulation is the technique of optimising the performance of a continuous process by trial-and-error adjustment of the controlled variables. A highly simplified model of a process is considered here. It contains a convex quadratic characteristic or `hill¿, which represents the graph of the controlled variable versus the performance. This is disturbed both vertically and horizontally by Brownian motions, and the process also involves dynamic lags and measurement noise. Under certain conditions which are discussed, the problem of designing the best regulator is related to a nonstationary filtering problem, in which the time-varying parameters are periodic or random, according to the nature of the searching strategy. This related problem has been examined by analogue computation and theoretical analysis in various asymptotic conditions. The results are interpreted to show how the best possible extremum regulator is constructed.