Cell selection in two-tier femtocell networks using Q-learning algorithm

Next-generation wireless networks will generate a heterogeneous network with micro base station (MBS) and femtocells where cell selection becomes crucial for balancing the utilization of the whole network. In this paper, we investigate cell selection problem in a two-tier femtocell network that contains a MBS and several femtocells with open/closed access methods and coverage areas. The selection process among groups of users in different service areas is formulated as a dynamic evolutionary game. In order to achieve an equilibrium, we present the Q-learning algorithm that can help distributed individual users adapt the situation and make cell selection decisions independently. With their own knowledge of the past, the users can learn to achieve the equilibrium without a centralized controller to gather other users information. Finally, simulation results present the convergence and effectiveness of the proposed algorithm.