Periodic collection of spectrum occupancy data by energy constrained cognitive IoT devices

The Internet of things (IoT) sets new bandwidth requirements for devices´ seamless connectivity and calls for new communication paradigms such as opportunistic spectrum access. However, realization of opportunistic spectrum access without causing interference to primary users poses significant challenges that are further exacerbated by the constrained nature of IoT devices both in terms of computational resources and battery capacity. In this work we consider the problem of energy efficient collection of primary users´ spectrum occupancy data that may be used to facilitate the prediction of future primary users´ states. We propose a process for a cognitive IoT device to decide the optimal spectrum sensing period for each state of a primary user. The decision process considers both energy consumption and accuracy in the tracking of a primary user´s state within the period. We model the problem at hand as a semi-Markov decision process embedded within a base Markov decision process using the framework of options. Finally, we utilize two variants of the Q-learning algorithm that are known to converge to optimal policies and we propose enhancements that significantly speed up the learning process. Numerical results verify the effectiveness of the proposed enhanced learning strategies.