Learning Multi-Agent Combat Scenarios in StarCraft II with League Training : an exploration of advanced learning techniques on a smaller scale

Sammanfattning: Google DeepMind trained their state-of-the-art StarCraft II agent AlphaStar using leaguetraining with massive computational power. In this thesis we explore league training onsmall-scale combat scenarios in StarCraft II, using limited computational resources, to an-swer whether this approach is suitable for smaller problems. We present two types ofagents: one trained using league training, and one trained against StarCraft II’s built-in AIusing traditional reinforcement learning. These agents are evaluated against each other, aswell as versus the built-in AI and human players. The results of these evaluations show thatthe agents trained with traditional reinforcement learning outperform the league-trainedagents, winning 520 out of 900 games played between them, with 21 ties. They also showbetter performance against the built-in AI and human players. The league agents still per-form well, with their skill being rated at the level of a Diamond or low Master player by ahuman Grandmaster ranked player. This shows that, while producing good results, leaguetraining does not reach the performance of the less computationally dependent traditionalreinforcement learning for problems of this scale. This might be due to the problem’s lowcomplexity in comparison to the full game of StarCraft II. A core aspect of league trainingis to find new strategies that target agents’ weaknesses in order to let agents learn to dealwith them, thus decreasing their exploitability. In smaller scenarios, there are not manydiverse strategies to apply, so this aspect cannot be fully utilised.