On implementation requirements and performances of Q-Learning for self-organized femtocells

In this paper we propose two Reinforcement Learning (RL) algorithms as a solution for the aggregated interference management, in realistic femto networks characterized by high dynamism due to, e.g., mobility of users, lognormal shadowing, fast fading, random activity patterns of femto nodes, etc. We discuss the Q-Learning (QL) algorithm, presented in previous works, which allows to learn online the most appropriate resource allocation policy, by continuous interactions with the environment. We improve it by fuzzy logic, in the form of Fuzzy Q-Learning (FQL), which allows a continuous state and action representation and a faster learning process. This approach overcomes the subjectivity in QL state and action space design, and allows femtocells to improve precision in the decision making process. However, these gains come at the expense of improved computational costs. This is why we focus this paper on the study of the feasibility of the proposed approach in 3rd Generation Partnership Project (3GPP) systems, and in state of the art processors.