Exploring Deep Reinforcement Learning Algorithms for Homogeneous Multi-Agent Systems

Sammanfattning: Despite advances in Deep Reinforcement Learning, multi-agent systems remain somewhat unexplored, in comparison to single-agent systems, with few clear conclusions. In order to investigate this, two algorithms have been implemented and tested on a simple multi-agent system: Deep Q Learning with several improvements (EDQN) and Asynchronous Advantage ActorCritic (A3C). The result shows that with an increasing number of agents, learning a well performing policy takes more time. When only a few agents are used, the performance of both algorithms when fully trained is similar and could be viewed as satisfactory. With more than 3-4 agents the performance of the A3C algorithm decreases while EDQN maintains its good performance. Certain hyperparameters of these algorithms have been investigated and the results have been presented. In conclusion EDQN performs better than A3C with multiple agents. For both algorithms, there is a strong sensitivity with regards to the hyperparameters.