Accelerated learning in machine learning-based resource allocation methods for Heterogenous Networks

Heterogeneous Networks, such as those with Femtocells and Macrocell Basestations, face the task of resource allocation to ensure all users, both primary (mobile user) and secondary (femtocell user), receive assurances of quality of service. One method of performing this allocation, Q-learning, involves the use of a reward function (defining objectives) and a Q-table (storing policy information). This Q-table can be shared between users to speed up convergence on a policy ensuring a desired quality of service. In this paper, a reward function and state structure are presented and compared to another Q-learning reward function. The designed RF is shown to increase the sum femtocell user capacity in most scenarios while maintaining the desired quality of service for the mobile user. The sharing of Q-tables formed using th e designed reward function and state structure with nodes entering the network is shown to significantly speed up convergence in most scenarios when compared to convergence without sharing Q-tables.