Using search based methods for beamforming

Sammanfattning: In accommodating the growing global demand for wireless, Multi-User Multiple-Input and Multiple-Output (MU-MIMO) systems have been identified as the key technology. In such systems, a transmitting basestation serves several users simultaneously, increasing the network capacity. However, sharing the same time-frequency physical resources can cause interference for the simultaneously scheduled users if not moderated properly. One way to mitigate this interference is by directing radio power through the radio channel in specific directions, a method which is called beamforming. Following the successful implementation of the AlphaZero algorithm in another radio resource management technique, scheduling, this thesis explores the potential of using a similar search-based method for the beamforming problem, striving towards the ultimate objective of making decisions for scheduling and beamforming jointly. However, as AlphaZero only supports discrete action spaces and the action space of the beamforming problem is continuous, a modification of the algorithm is required. The proposed course of action is to extend AlphaZero into Sampled AlphaZero, using sample-based policy improvement to create an algorithm that is both more scalable for large discrete action spaces and able to handle high dimensional continuous action spaces. To evaluate the performance of the models, test environments were simulated and solved using increasingly larger so-called codebooks, containing predefined beamforming solutions. The results of the Sampled AlphaZero model demonstrated promising performance even for very large codebook sizes, indicating the model's suitability for addressing the beamforming problem in a non-codebook-based context. Furthermore, this thesis explores how states in the search can be represented and preprocessed for the neural network to learn efficiently, demonstrating clear benefits of using a singular value decomposition-based state preprocessing over raw states as input to the neural network.