Indexability and whittle index for restless bandit problems involving reset processes

Author

Liu, Keqin ; Weber, Richard ; Zhao, Qing

Author_Institution

Dept. of Electr. & Comput. Eng., Univ. of California, Davis, CA, USA

fYear

2011

fDate

12-15 Dec. 2011

Firstpage

7690

Lastpage

7696

Abstract

We consider a class of restless multi-armed bandit (RMAB) problems, in which the active action resets the stochastic evolution of the system. We obtain the Whittle index in closed-form, showing that it induces a policy that is equivalent to the myopic policy, and that it is optimal for stochastically identical arms. These results find applications in opportunistic spectrum access and supervisory control systems such as anomaly detection and control.

Keywords

decision making; evolutionary computation; infinite horizon; probability; stochastic processes; Whittle index; active action reset; anomaly detection; decision making; indexability; infinite horizon; myopic policy; opportunistic spectrum access; probability; restless multiarmed bandit problem; stochastically identical arms; supervisory control system; system stochastic evolution; Equations; Indexes; Markov processes; Monitoring; Process control; Random processes; Transient analysis; Reset processes; Whittle index policy; myopic policy; restless multi-armed bandit;

fLanguage

English

Publisher

ieee

Conference_Titel

Decision and Control and European Control Conference (CDC-ECC), 2011 50th IEEE Conference on

Conference_Location

Orlando, FL

ISSN

0743-1546

Print_ISBN

978-1-61284-800-6

Electronic_ISBN

0743-1546

Type

conf

DOI

10.1109/CDC.2011.6160533

Filename

6160533