Reduction of Common Operations in a Neural Network

Sammanfattning: Machine learning models are becoming increasingly complex, and particularly artificial neural networks. Meanwhile, solutions are moving closer towards the edge with implementations on devices such as smartphones, TVs and cameras. This creates a demand for efficient models that perform well despite restricted computational resources. One particularly successful convolutional neural network that was first introduced in 2014 is the Inception network, which in its first edition was named GoogLeNet. The Inception network’s structure was highly successful for complex image classification and the model has since then been developed iteratively, resulting in several new versions. The latest version, Inception-V4, was presented in 2016 and the model structure is still highly utilized, and many modern neural networks draw inspiration from it. The aim of this thesis is to develop and evaluate a more lightweight and potentially more efficient version of the Inception network, and discuss its opportunities and challenges. The proposed lighter version, named LightInception, is based on the original Inception-V4 model, where a process called reduction of common operations was implemented to lower the network’s complexity. Reduction of common operations is a method developed in this study for simplifying networks with parallel structures, such as the Inception network. The process draws inspiration from multiple modern network structures and best practices. In practice, its implementation on Inception-V4 lowered the redundancy in the network and reduced the total number of parameters by 33%. LightInception was evaluated on six diverse data sets with regard to inference time, accuracy, loss, convergence, training volatility, and weight utilization. The model showed promising results with higher or equal accuracy for at least half of the evaluated data sets. This indicates that reduction of common operations may be an efficient means to reduce model complexity without losing representative power, and the process is suggested for further investigation.