Title :
Multiagent Reinforcement Learning Based Spectrum Sensing Policies for Cognitive Radio Networks
Author :
Lunden, Jarmo ; Kulkarni, Sanjeev R. ; Koivunen, Visa ; Poor, H. Vincent
Author_Institution :
Dept. of Signal Process. & Acoust., Aalto Univ., Espoo, Finland
Abstract :
This paper proposes distributed multiuser multiband spectrum sensing policies for cognitive radio networks based on multiagent reinforcement learning. The spectrum sensing problem is formulated as a partially observable stochastic game and multiagent reinforcement learning is employed to find a solution. In the proposed reinforcement learning based sensing policies the secondary users (SUs) collaborate to improve the sensing reliability and to distribute the sensing tasks among the network nodes. The SU collaboration is carried out through local interactions in which the SUs share their local test statistics or decisions as well as information on the frequency bands sensed with their neighbors. As a result, a map of spectrum occupancy in a local neighborhood is created. The goal of the proposed sensing policies is to maximize the amount of free spectrum found given a constraint on the probability of missed detection. This is addressed by obtaining a balance between sensing more spectrum and the reliability of sensing results. Simulation results show that the proposed sensing policies provide an efficient way to find available spectrum in multiuser multiband cognitive radio scenarios.
Keywords :
cognitive radio; learning (artificial intelligence); multi-agent systems; radio spectrum management; signal detection; stochastic games; telecommunication computing; telecommunication network reliability; SU collaboration; cognitive radio networks; distributed multiuser multiband spectrum sensing policies; frequency bands; local interactions; local neighborhood; local test statistics; missed detection probability; multiagent reinforcement learning; network nodes; partially observable stochastic game; secondary users; sensing reliability; spectrum occupancy; Ad hoc networks; Cognitive radio; Collaboration; Games; Learning (artificial intelligence); Sensors; Time-frequency analysis; Cognitive radio networks; collaborative spectrum sensing; multiagent reinforcement learning; multiuser multiband spectrum sensing policy; partially observable stochastic game;
Journal_Title :
Selected Topics in Signal Processing, IEEE Journal of
DOI :
10.1109/JSTSP.2013.2259797