Reinforcement Learning-based Handover in Millimeter-wave Networks

Sammanfattning: Millimeter Wave (mmWave) is a key technology to meet the challenge of data rates and the lack of bandwidth in sub-6GHz networks. Due to a high operation frequency, the mmWave network has unique channel characteristics and a relatively high pathloss. Therefore, a dense deployment of Base Station (BS) is necessary, leading to a more frequent handover, which may cause a degradation of User Equipment (UE) experience. Furthermore, a massive number of devices cause an interference issue and a high dropping probability. In this project, we propose a handover method based on Reinforcement Learning (RL). This handover method provides a seamless connection and considers the load balancing. To verify the proposed method, Q-learning is selected to solve this RL problem and a simulation environment of mmWave is set up, including the pathloss model, system model, and beamforming. The average data rate, number of handovers, and number of available resources are evaluated during the movement of UEs. The results are compared with rate-max method and random backup method in different interference scenarios. Our proposed method shows a notable performance in terms of data rate, for example, while doubling the interference, the data rate decreases 8.6% with our method while it decreases 20% with the random-backup method. Moreover, our method has the minimum number of handovers in the trajectory. The performance in multiple trajectories is also illustrated and it performs as expected.