Risk-Sensitive Decision-Making for Autonomous-Driving

Sammanfattning: A natural aspect of the real world is that one can face uncertain situations on a daily basis. Depending on one's experience, we humans behave and respond differently to uncertainty. However, when designing intelligent agents, one needs to pay attention to the uncertainty inlearning tasks to design risk-sensitive algorithms. This thesis contributes to this line of work by considering uncertainty in reinforcement learning. This project investigated the question whether the method of conformal prediction, which is a paradigm for uncertainty quantification, can help obtaining confident predictions of state-action values when utilised by reinforcement learning agents. For this purpose, we proposed a novel algorithm that combined the distributional C51 agent with the Classification with Valid and Adaptive Coverage algorithm. Moreover, a driving environment for evaluation of the algorithm was implemented using the autonomous driving simulator Carla. Our risk-sensitive agent and our own implementation of the risk-neutral C51 agent were trained on various self-driving tasks including multiple spawn points and non-player characters, in addition to other learning environments such as the Cliff Walking, Cart Pole, Lunar Lander and Acrobot. We used metrics such as the number of collisions and survival rate for self-driving tasks and number of falls in the Cliff Walking to evaluate the agents. Our results showed that both agents perform similarly on some tasks and may have difficulty in solving some other tasks properly. We conclude that our algorithm does not give rise to a robust agent, but it can be used as a starting point for further research in the area.