Title :
Indexability and whittle index for restless bandit problems involving reset processes
Author :
Liu, Keqin ; Weber, Richard ; Zhao, Qing
Author_Institution :
Dept. of Electr. & Comput. Eng., Univ. of California, Davis, CA, USA
Abstract :
We consider a class of restless multi-armed bandit (RMAB) problems, in which the active action resets the stochastic evolution of the system. We obtain the Whittle index in closed-form, showing that it induces a policy that is equivalent to the myopic policy, and that it is optimal for stochastically identical arms. These results find applications in opportunistic spectrum access and supervisory control systems such as anomaly detection and control.
Keywords :
decision making; evolutionary computation; infinite horizon; probability; stochastic processes; Whittle index; active action reset; anomaly detection; decision making; indexability; infinite horizon; myopic policy; opportunistic spectrum access; probability; restless multiarmed bandit problem; stochastically identical arms; supervisory control system; system stochastic evolution; Equations; Indexes; Markov processes; Monitoring; Process control; Random processes; Transient analysis; Reset processes; Whittle index policy; myopic policy; restless multi-armed bandit;
Conference_Titel :
Decision and Control and European Control Conference (CDC-ECC), 2011 50th IEEE Conference on
Conference_Location :
Orlando, FL
Print_ISBN :
978-1-61284-800-6
Electronic_ISBN :
0743-1546
DOI :
10.1109/CDC.2011.6160533