A Learning-Based Network Selection Method in Heterogeneous Wireless Systems

With the coexistence of various wireless technologies, next generation wireless communications will likely consist of an integrated system of networks, where the Access Points (APs) and Base Stations (BSs) work together to maximize the mobile-user Quality of Service (QoS). In such heterogeneous environment where handheld devices with different access technologies are not uncommon, it should be possible to select networks and seamlessly switch from one AP/BS to another in order to elevate user performance. In this paper, this type of network selection and handover mechanism with the goal of maximizing QoS is formulated as a Markov Decision Process (MDP). An algorithm based on Reinforcement Learning (RL) is then obtained that selects the best network based not only on the current network load but also the potential future network states. This algorithm aims at balancing the number of handovers and the achievable QoS. The results illustrate that while the QoS performance of the proposed algorithm is comparable to the performance of the optimum opportunistic selection algorithm, fewer number of network handovers (on average) are required. In addition, compared to the existing predefined network selection strategies with no handover, the MDP-based algorithm offers significantly better QoS.