Actively adaptive methods for stochastic systems

An important element of adaptive control is learning of the drifting parameters. As the process unfolds, additional information becomes available, which will provide learning for the purpose of control. This information may come about accidentally through past control actions or as a result of active probing, which itself is a possible control policy. Thus learning is present, where it is accidental or deliberate. Since more learning may improve overall control performance, the probing signal may indirectly help in controlling the stochastic system. On the other hand, excessive probing should not be allowed even though it may promote learning because it is expensive in the sense that it will, in general, increase the expected cost performance of the system. A good control law must then regulate its adaptation (learning) in an optimal manner. An adaptive control method is called passively adaptive if learning is not planned in the manner described above; it is called actively adaptive if learning is planned and regulated for the purpose of final control. This paper gives an overview of adaptive control methods which were developed based on the concept of active learning for control purposes. Some comments on their practicality are also given.