Dynamic Mixed Reality AssemblyGuidance Using Optical Recognition Methods

Sammanfattning: Mixed Reality (MR) is an emerging paradigm in industry. While MR equipment and software have taken great technological strides in past years, standardized methods and workflows for developing MR systems for industry have not been widely adopted for many tasks. This thesis proposes a dynamic MR system for an assembly process. Optical recognition methods are explored to drive the application logic. The systemis developed using the Unity platform for the HoloLens 2. The software tools Vuforia Engine and Mixed Reality Toolkit (MRTK) are utilized. The project work concludes with an application capable of guiding users using graphics and audio. Successful methods are realized for calibrating the application logic for dynamic object positions,as well as for validating user actions. Experiments are conducted to validate the system. Subjects complete a different assembly process using paper instructions as guidance before using the MR application. Qualitative results regarding the MR experience are obtained through a questionnaire subjects answer, where the experience using paper instructions serves as a benchmark. Data obtained from an experienced user completing the assembly process is used as a quantitative benchmark for system performance measures. All subjects were able to complete the assembly tasks correctly using the MR application. Results show significantly better system performance for the experienced user compared to subjects unfamiliar with the MR system. Vuforia Engine recognition tools successfully tracked individual components that meet a specific criterion. Methods for validating user actions using Vuforia Engine software tools and the HoloLens’s internal hand tracking capabilities resulted in a high validation success rate. The thesis concludes effective training methods for the specific assembly scenario, although not robust for general implementation.