Online learning in channel sensing order for cognitive radios

The information about channel statistics is sometimes not available for cognitive users a priori. The need to learn this information online creates a fundamental trade-off between exploitation and exploration. This can be formulated as a Multi-Armed Bandit (MAB) problem. In this work the channel sensing-order setting problem is studied under this scenario. It is shown that the regret of classic UCB1 policy will exponentially increase with the number of channels. Thus two new methods are proposed: UCB1 with virtual probing and UCB1 index based greedy search algorithm. The UCB1 with virtual probing method fully explores the dependencies between different sensing orders, which increases the learning efficiency of the user. The UCB1 index based greedy search method modifies the potential function of classic greedy search algorithm with the UCB1 index, which makes the decision converge to the optimal channel sensing order rapidly. The simulation results show that both the UCB1 with virtual probing and UCB1 index based greedy search methods outperform traditional ones.