Introducing strategic measure actions in multi-armed bandits

Multi-armed bandits may be used for modelling the process of selecting one among different wireless networks, given a set of system constraints typically formed by user-perceived network quality indicators. This work proposes a novel multi-armed bandit, that is made appropriate to the above context by introducing a distinction between two actions, to measure and to use, in order to better reflect real communication application scenarios. The impact of this introduction is analysed through simulations by comparing a traditional multi-armed bandit algorithm against methods that integrate the new concept of measuring vs. using. Results show that performance in terms of regret can be significantly improved using the proposed algorithms if the period needed for measuring is at least 3 times shorter than the one for the using action. The classical method would require a significantly shorter measuring period to reach the same regret, i.e. much stricter constraints on the allowed measure action duration.