An evaluation of GPU virtualization

Sammanfattning: There has been extensive research and progress on virtualization on CPUs for a while. More recently the focus on GPU virtualization has increased as processing power doubles roughly every 2.5 years. Coupled with advances in memory management and the PCIe standard the first hardware assisted virtual solutions became available in the 2010s. Very recently, a new virtualization mode called Multi-Instance GPU (MIG) makes it possible to isolate partitions with memory in hardware rather than just software. This thesis is focused on virtual GPU performance and capabilities for AI training in a multi tenant situation. It explores the technologies currently used for GPU virtualization,including Single Root IO Virtualization (SR-IOV) and mediated devices. It also covers a proposed new standard for IO virtualization called SIOV that addresses some of the limitations in the SR-IOV standard. The limitations of time sliced virtualization are mainly the lack of customization for a partition compared to CPU virtualization and the problem of overhead. MIG virtualization is more customisable in how compute power and memory can be allocated, the biggest limitation is that fast intercommunication is not currently possible between partitions, making MIG more suited for applications that can run on just one partition. It is also not suited for graphical applications as it currently does not support any graphical APIs. The experimental results showed that in compute situations the overhead of time sliced virtualization is around 5% while the maximum intercommunication bandwidth is lowered by 11% and latency increased by 25%. Time slice windows of 4ms compared to 2ms can decrease scheduling overhead to nearly 0.5% at the cost of increased latency for the end user, this can be beneficial for applications where user interactivity is not of importance.