System for augmented video with Unreal Engine

Sammanfattning: Video and body cameras are increasingly used by operational personnel, such as first responders, to improve situational awareness and safety of operations. However, to attain the full potential of video applications in this domain, operators need support to find the relevant information in the multiple video streams sent from the accident site. Needed are flexible applications that can handle multiple video streams and augment relevant parts of the video to support the operator. In this thesis, we present a video application, with virtual object augmentation and overlay functionality. The system handles a set of video streams and augments them using Unreal Engine and Gstreamer. Current modern software frameworks for video streaming and augmentation were examined as part of the development.  The performance of the application was evaluated using a simulated set of video streams and augmentation requests. Frames per second were measured to ensure a reliable and functional application. The study suggests that Unreal Engine, together with Gstreamer, is a suitable framework combination for the development of this application. Unreal Engine provides nDisplay, which is a powerful feature for multi-display setups. It supports synchronized presentation on displays, even in a cluster of multiple computers, with easy setup in a nDisplay editor. Furthermore, extending Unreal using C++ facilitates the integration with Gstreamer that enables integration with other applications with its native C++ support and external APIs for accessing media data. The performance measurements of the final application show adequate performance with respect to the defined use cases.