Model free dynamic sensing order selection for imperfect sensing multichannel cognitive radio networks: A Q-learning approach

In multichannel cognitive radio networks (CRN), the sequence of sensing order is essential to determine the efficiency of detecting an idle channel and the effective time for spectrum access. Current works on the optimal channel sensing order have to know priori knowledge such as the channel idle probabilities, channel gains and achievable rates of different channels, which dynamically evolve and are difficult to obtain in practice. We are thus motivated to study a model-free sensing order selection scheme for multichannel CRN. In particular, we consider a time-slotted imperfect sensing CRN, where sensing errors impose more challenges to the selection of sensing order. The sensing order selection is modeled as a Q-Learning problem, where secondary users (SU) make intelligent decision on sensing order selection by learning from historical sensing and transmissions. Simulation results show that theq proposed scheme is robust in dynamic environment and can achieve significant throughput improvement compared with random sequential sensing.