Title :
Adaptive control and parameter estimation in Markov chains: A linear case
Author_Institution :
Universidad Simón Bolívar, Caracas, Venezuela
fDate :
4/1/1982 12:00:00 AM
Abstract :
We consider a controlled Markov chain with finite state space, whose transition probabilities are assumed to depend linearly upon an unknown real parameter α. In particular, we study the asymptotic behavior of the maximum likelihood estimate of α under an arbitrary realizable control when α is known to belong to a given bounded interval on the line. We show that the estimate converges with probability one and characterize those realizations for which convergence does not lead to the true value. We also suggest corrections to the control policy which guarantee almost sure convergence to the true value. For the adaptive situation, where the control depends only on the current estimate of α and the present state, we show that the maximum likelihood estimate converges to a value α*indistinguishable from the true one under the feedback law induced by α*.
Keywords :
Adaptive control, linear systems; Linear systems, stochastic; Markov processes; Parameter estimation, linear systems; Stochastic systems, linear; Adaptive control; Discrete wavelet transforms; Feedback control; Linear systems; Parameter estimation; State estimation; Testing; Uncertainty;
Journal_Title :
Automatic Control, IEEE Transactions on
DOI :
10.1109/TAC.1982.1102921