Cloud-based video stabilization processing in real-time

Sammanfattning: On-device real-time video stabilization has become a common feature of modernsmartphones. This processing technique improves quality but is an expensive processthat requires significant resources from the devices themselves. By offloading thisprocess to the cloud, much better hardware could be utilized to perform these tasks.A solution is proposed in which a video recording application and newly introducedprocessing server securely exchange video and metadata in real-time so that thestabilized version gets saved on the client device. The prototypes are built on top ofImint's reference Android application and C++ core processing library and use acommunication protocol based on TLS over TCP. The impact on latency, betweenstopping a recording and the recorded video being saved, was measured and dependson several factors but falls within the range of 100 to 200ms on top of round-triptimes of the network between the client and server. Hardware usage increased by13.5% compared to the local solution, battery consumption by 30%, while picturequality wasn't affected as much. The remote processing solution opens up a widerange of applications, in which more expensive stabilization techniques or otheruse-cases like stabilized live-streaming could be performed.