Smart Scooter : Solving e-scooter safety problems with multi-modal, privacy-preserving sensor technology and machine learning

Sammanfattning: Micromobility ride-share scooters (e-scooters) have become a popular mode of transport in several major cities around the world, yet several safety and accessibility issues stem from how these scooters are operated, including sidewalk riding, unsafe parking and wrong-way riding. This thesis tackles these issues through a novel, privacy-preserving, end-to-end sensor system that employs lightweight machine learning models to provide real-time feedback to users to present unsafe scooter operation. Though this problem has been widely studied in technology startups and most of the existing solutions entail using cameras, there is an interest in academia to propose solutions that do not use cameras, as they cause privacy concerns when used in urban environments and are known to fail at night or in low-visibility conditions. Considering this, we propose a portable, cheap, and robust system that preserves privacy through the use of an Inertial Measurement Unit and radar sensors, which give only low-resolution heatmaps as outputs. Furthermore, unlike cameras, radars function even in low-visibility conditions. Using data from Inertial Measurement Unit sensors, a 1D Convolutional Neural Network is used to classify whether a scooter is being ridden on a street or on a sidewalk. Radar heatmaps are used to train a lightweight Convolutional Neural Network to classify common objects in urban environments (Lampposts, Walls, Trees, Humans, Fire Hydrants and Middle of Sidewalk). For the purpose of this research; Lamppost, Wall, Tree are considered safe parking scenarios whereas Fire Hydrant, Human and Middle of Sidewalk is considered unsafe. No public datasets exist on this topic, so two novel datasets are created. The models trained are lightweight in terms of memory and have fast inference time and are therefore suitable for edge computing, and for being deployed on a sensor system mounted on the scooter. The results achieved demonstrate the potential of this approach, though further work is needed to ensure performance for deployment in the real world.