A performance comparison between CPU and GPU in TensorFlow

Sammanfattning: The fast-growing field of Machine Learning has in the later years become more common, as it has gone from a restricted research area to actually be in general use. Frameworks such as TensorFlow have been developed to scale and analyze artificial neural networks, which are used in one of the areas in Machine Learning called Deep Learning. This paper will study how well the framework TensorFlow performs in regard to time and memory allocation on the processor units CPU and GPU since these are the factors that are often the restraining resources. Three neural networks have been used to measure how TensorFlow allocates the resources and computes operations used to process the neural network during the training phase. By using TensorFlows profiler we could trace how each operation was executed in the CPU and GPU, from the gathered data we could analyse how the operations allocated memory and time. Our results show that the training of a more complex neural network benefits from being executed on the GPU, while a simple neural network has no or an insignificant profit from being executed on the GPU over the CPU. The result also indicates possible findings for further research such as processor utilisation as the gaps in the scheduling has not been studied in this paper.