DocumentCode
1658293
Title
A consistent algorithm for derivative estimation of Markov chains
Author
Dai, Liyi
Author_Institution
Dept. of Syst. Sci. & Math., Washington Univ., St. Louis, MO, USA
Volume
2
fYear
1994
Firstpage
1990
Abstract
A consistent algorithm for derivative estimation of finite-state, discrete-time Markov chains is presented. The basic idea is to simulate original Markov chains with modified performance measures that can be estimated by extra simulation. The computational load of the extra simulation at each step is bounded. The algorithm attains the best possible rate of convergence as the simulation time goes to infinity. A connection between the algorithm and solutions to Poisson equations is also revealed
Keywords
Markov processes; differentiation; discrete time systems; finite automata; identification; simulation; stochastic processes; Poisson equations; computational load; derivative estimation; finite-state discrete-time Markov chains; modified performance measures; simulation; Computational modeling; Computer simulation; Cost function; Discrete event simulation; H infinity control; Poisson equations; Sensitivity analysis; State estimation; State-space methods; Stationary state;
fLanguage
English
Publisher
ieee
Conference_Titel
Decision and Control, 1994., Proceedings of the 33rd IEEE Conference on
Conference_Location
Lake Buena Vista, FL
Print_ISBN
0-7803-1968-0
Type
conf
DOI
10.1109/CDC.1994.411084
Filename
411084
Link To Document