Skyline Delineation for Localization in Occluded Environments : Improved Skyline Delineation using Environmental Context from Deep Learning-based Semantic Segmentation

Sammanfattning: This thesis addresses the problem of improving the delineation of skylines, also referred to as skyline detection, in occluded and challenging environments where existing skyline delineation methods may struggle or fail. Delineated skylines can be used in monocular camera localization methods by comparing delineated skylines to digital elevation model data to estimate a position based on known terrain. This is particularly useful in GPS-denied environments in which active sensing is either impractical or undesirable for various reasons, so that passive sensing using monocular cameras is necessary and/or strategically advantageous. This thesis presents a novel method of skyline delineation using deep learning-based semantic segmentation of monocular camera images to detect natural skylines of distant landscapes in the presence of occlusions. Skylines are extracted from semantic segmentation predictions as the boundary between pixel clusters labeled as terrain to those labeled as sky, with additional segmentation classes representing the known set of potential occlusions in a given environment. Additionally, each pixel in the detected skyline contours are assigned a confidence score based on local intensity gradients to reduce the potential impacts of erroneous skyline contours on position estimation. The utility of these delineated skylines is demonstrated by obtaining orientation and position estimates using existing methods of skyline-based localization. In these methods, the delineated natural skyline is compared to rendered skylines using digital elevation model data and the position estimate is obtained by finding the closest match. Results from the proposed skyline delineation method using semantic segmentation, with accompanying localization demonstration, is presented on two distinct data sets. The first is obtained from the Perseverance Rover operating in the Jezero Crater region of Mars, and the second is obtained from an uncrewed surface vessel operating in the Gulf of Koper, Slovenia.