Title :
A learning scheme for stationary probabilities of large markov chains with examples
Author :
Borkar, V.S. ; Das, D.J. ; Banik, A. Datta ; Manjunath, D.
Author_Institution :
Sch. of Technol. & Comput. Sci., Tata Inst. of Fundamental Res., Mumbai
Abstract :
We describe a reinforcement learning based scheme to estimate the stationary distribution of subsets of states of large Markov chains. dasiaSplit samplingpsila ensures that the algorithm needs to just encode the state transitions and will not need to know any other property of the Markov chain. (An earlier scheme required knowledge of the column sums of the transition probability matrix.) This algorithm is applied to analyze the stationary distribution of the states of a node in an 802.11 network.
Keywords :
Markov processes; learning (artificial intelligence); 802.11 network; Markov chains; reinforcement learning; stationary probabilities; transition probability matrix; Algorithm design and analysis; Approximation algorithms; Computer science; Eigenvalues and eigenfunctions; Function approximation; Learning; Sampling methods; State estimation; Stochastic processes; Zinc;
Conference_Titel :
Communication, Control, and Computing, 2008 46th Annual Allerton Conference on
Conference_Location :
Urbana-Champaign, IL
Print_ISBN :
978-1-4244-2925-7
Electronic_ISBN :
978-1-4244-2926-4
DOI :
10.1109/ALLERTON.2008.4797682
