Title :
Cell selection in two-tier femtocell networks using Q-learning algorithm
Author :
Xu Tan ; Xi Luan ; Yuxin Cheng ; Aimin Liu ; Jianjun Wu
Author_Institution :
Sch. of Electron. Eng. & Comput. Sci., Peking Univ., Beijing, China
Abstract :
Next-generation wireless networks will generate a heterogeneous network with micro base station (MBS) and femtocells where cell selection becomes crucial for balancing the utilization of the whole network. In this paper, we investigate cell selection problem in a two-tier femtocell network that contains a MBS and several femtocells with open/closed access methods and coverage areas. The selection process among groups of users in different service areas is formulated as a dynamic evolutionary game. In order to achieve an equilibrium, we present the Q-learning algorithm that can help distributed individual users adapt the situation and make cell selection decisions independently. With their own knowledge of the past, the users can learn to achieve the equilibrium without a centralized controller to gather other users information. Finally, simulation results present the convergence and effectiveness of the proposed algorithm.
Keywords :
femtocellular radio; game theory; learning (artificial intelligence); MBS; Q-learning algorithm; cell selection; centralized controller; dynamic evolutionary game; heterogeneous network; microbase station; open/closed access methods; reinforcement learning; two tier femtocell networks; users information; wireless networks; Base stations; Computer architecture; Educational institutions; Femtocell networks; Games; Heuristic algorithms; Microprocessors; Cell selection; Q-learning; evolutionary game; femtocell networks; pure greedy algorithm;
Conference_Titel :
Advanced Communication Technology (ICACT), 2014 16th International Conference on
Conference_Location :
Pyeongchang
Print_ISBN :
978-89-968650-2-5
DOI :
10.1109/ICACT.2014.6779115