Object Detection in Augmented Reality

Sammanfattning: Augmented reality is a field which is getting increased funding every year, as more businesses are realizing the potential of rendering virtual objects in the real world. As the equipment gets more commercialized, the costs will get lowered while performance also goes up. As of now, augmented reality mostly makes use of plane detection and marker detection to find and locate objects. We want to incorporate machine learning using deep neural networks to be able to find objects in an augmented reality scene. In this report, we go through the process of developing an iOS application, which incorporates the use of object detection, object recognition and augmented reality. Our goal is to identify if the combination of these fields is both feasible and desirable with modern tools available. The application is supposed to work as an alternative to conventional assembly manuals for furniture. The project is about showing a user how to, step by step, put together furniture using an iPhone X and give instructions in an augmented reality experience. The machine learning model is taught to recognize all the different parts of furniture and where they are in the image. Different methods were tried to make this possible, such as image segmentation, edge detection and YOLO. The final product makes use of ARKit, developed by Apple, to render the augmented reality scene. It also uses Turi Create to train a neural network for object detection and classifier. We then evaluate the application with user tests. We further conclude that modern technology available makes our idea possible and the user tests show that the concept has great potential and is desirable.