The curious case of artificial intelligence : An analysis of the relationship between the EU medical device regulations and algorithmic decision systems used within the medical domain

Sammanfattning: The healthcare sector has become a key area for the development and application of new technology and, not least, Artificial Intelligence (AI). New reports are constantly being published about how this algorithm-based technology supports or performs various medical tasks. These illustrates the rapid development of AI that is taking place within healthcare and how algorithms are increasingly involved in systems and medical devices designed to support medical decision-making.  The digital revolution and the advancement of AI technologies represent a step change in the way healthcare may be delivered, medical services coordinated and well-being supported. It could allow for easier and faster communication, earlier and more accurate diagnosing and better healthcare at lower costs. However, systems and devices relying on AI differs significantly from other, traditional, medical devices. AI algorithms are – by nature – complex and partly unpredictable. Additionally, varying levels of opacity has made it hard, sometimes impossible, to interpret and explain recommendations or decisions made by or with support from algorithmic decision systems. These characteristics of AI technology raise important technological, practical, ethical and regulatory issues. The objective of this thesis is to analyse the relationship between the EU regulation on medical devices (MDR) and algorithmic decision systems (ADS) used within the medical domain. The principal question is whether the MDR is enough to guarantee safe and robust ADS within the European healthcare sector or if complementary (or completely different) regulation is necessary. In essence, it will be argued that (i) while ADS are heavily reliant on the quality and representativeness of underlying datasets, there are no requirements with regard to the quality or composition of these datasets in the MDR, (ii) while it is believed that ADS will lead to historically unprecedented changes in healthcare , the regulation lacks guidance on how to manage novel risks and hazards, unique to ADS, and that (iii) as increasingly autonomous systems continue to challenge the existing perceptions of how safety and performance is best maintained, new mechanisms (for transparency, human control and accountability) must be incorporated in the systems. It will also be found that the ability of ADS to change after market certification, will eventually necessitate radical changes in the current regulation and a new regulatory paradigm might be needed.