Joint Resource Management with Reinforcement Learning in Heterogeneous Networks

In heterogeneous networks where multiple access networks are used, the resource management will be an important role to utilize wireless resources more effectively. We consider a base station that provides multiple access networks with different carrier frequencies and uses Adaptive Modulation and Coding scheme (AMC) based on the channel quality to communicate with the users at each access network. In order to maximize the wireless resource utilization with the guarantee of Quality of Service (QoS) for each user, the resource management in the base station needs to make various decisions taking into account of the state of wireless resource utilization and the users´ behaviors. In this paper, we introduce a joint resource management which performs admission control, access network selection and vertical handover decision jointly and propose its optimal algorithm. The problem is naturally formulated as a Semi-Markov Decision Process (SMDP) and the optimal policy at each state is derived via reinforcement learning. The performance evaluation shows that the proposed algorithm outperforms other three algorithms in terms of the blocking probability and the system throughput.