Real-time Embedded Panoramic Imaging for Spherical Camera System

Sammanfattning: Panoramas or stitched images are used in topographical mapping, panoramic 3D reconstruction, deep space exploration image processing, medical image processing, multimedia broadcasting, system automation, photography and other numerous fields. Generating real-time panoramic images in small embedded computer is of particular importance being lighter, smaller and mobile imaging system. Moreover, this type of lightweight panoramic imaging system is used for different types of industrial or home inspection. A real-time handheld panorama imaging system is developed using embedded real-time Linux as software module and Gumstix Overo and PandaBoard ES as hardware module. The proposed algorithm takes 62.6602 milliseconds to generate a panorama frame from three images using a homography matrix. Hence, the proposed algorithm is capable of generating panorama video with 15.95909365 frames per second. However, the algorithm is capable to be much speedier with more optimal homography matrix. During the development, Ångström Linux and Ubuntu Linux are used as the operating system with Gumstix Overo and PandaBoard ES respectively. The real-time kernel patch is used to configure the non-real-time Linux distribution for real-time operation. The serial communication software tools C-Kermit, Minicom are used for terminal emulation between development computer and small embedded computer. The software framework of the system consist UVC driver, V4L/V4L2 API, OpenCV API, FFMPEG API, GStreamer, x264, Cmake, Make software packages. The software framework of the system also consist stitching algorithm that has been adopted from available stitching methods with necessary modification. Our proposed stitching process automatically finds out motion model of the Spherical camera system and saves the matrix in a look file. The extracted homography matrix is then read from look file and used to generate real-time panorama image. The developed system generates real-time 180° view panorama image from a spherical camera system. Beside, a test environment is also developed to experiment calibration and real-time stitching with different image parameters. It is able to take images with different resolutions as input and produce high quality real-time panorama image. The QT framework is used to develop a multifunctional standalone software that has functions for displaying real-time process algorithm performance in real-time through data visualization, camera system calibration and other stitching options. The software runs both in Linux and Windows. Moreover, the system has been also realized as a prototype to develop a chimney inspection system for a local company.