Insights into Model-Agnostic Meta-Learning on Reinforcement Learning Tasks

Sammanfattning: Meta-learning has been gaining traction in the Deep Learning field as an approach to build models that are able to efficiently adapt to new tasks after deployment. Contrary to conventional Machine Learning approaches, which are trained on a specific task (e.g image classification on a set of labels), meta-learning methods are meta-trained across multiple tasks (e.g image classification across multiple sets of labels). Their end objective is to learn how to solve unseen tasks with just a few samples. One of the most renowned methods of the field is Model-Agnostic Meta-Learning (MAML). The objective of this thesis is to supplement the latest relevant research with novel observations regarding the capabilities, limitations and network dynamics of MAML. For this end, experiments were performed on the meta-reinforcement learning benchmark Meta-World. Additionally, a comparison with a recent variation of MAML, called Almost No Inner Loop (ANIL) was conducted, providing insights on the changes of the network’s representation during adaptation (meta-testing). The results of this study indicate that MAML is able to outperform the baselines on the challenging Meta-World benchmark but shows little signs actual ”rapid learning” during meta-testing thus supporting the hypothesis that it reuses features learnt during meta-training.