Generating lightning bolt videos perceived as real in images using machine learning

Sammanfattning: Background. Weather and weather effects are important features when trying to immerse the viewer into a virtual world. Lightning and thunder is one of those effects when attempting to create rough weather, realistic lightning however requires heavy computations, using physics, weather systems, and knowledge of the 3d world. Objectives. This thesis investigates the possibility of leveraging the predictive power of machine learning to generate animated lightning bolts inside of images, and then investigates the possibility to generate the animated lightning bolts in real time.   Methods. A new data-set for training will be created consisting of videos of lightning bolts. Four image to video machine learning architectures will be investigated and two will be tested in an attempt to find a suitable model for generating the animated lightning bolts. The selected model will be used to generate videos for a questionnaire to collect qualititive data regarding the perceived realism of the animated lightning bolts. To figure out if it is possible to generate the animated lightning bolts in real time the final model will be performance measured and compared to real time requirements of video games and video editing software. Results. For the training data-set 106 curated and pre-processed videos were collected. By gathering four and testing two different machine learning architectures it was found that the architecture based on stochastic Image-to-Video Synthesis using conditional invertible neural networks were the most suited for generating animated lightning bolts. The questionnaire received a 77% positive rating for the generated lightning bolts, with a 1% statistical significance a p-value of 0.00005 was obtained. The performance of the selected machine learning model were measured to be inadequate for real time applications like video games but more than enough for video editing software. Conclusions. The goal of generating animated lightning bolts percieved as real were achieved by creating a new data-set and investigating multiple machine learning architectures. Real time generation is achievable for video editing applications, but real time generation for video games is not yet possible unless the background is static.