Sim2Real: Generating synthetic images from industry CAD models with domain randomization

Sammanfattning: Deep learning methods for computer vision applications require massive visual data for model training. Although it is possible to utilize public datasets such as ImageNet, MS COCO, and CIFAR-100, it becomes problematic when there is a need for more task-specific data when new training data collection typically is needed. Manually collecting and labeling image data is a difficult job. Because it is costly and labor-intensive, ground-truth labeling may introduce human errors. In manufacturing, the problem becomes even more critical. Industrial parts are often unique and patent-protected. Acquiring the image data for industrial parts is only possible after manufacturing starts. Those concerns are obstacles to applying deep learning methods for computer vision applications in manufacturing workplaces such as Scania.Therefore, a low-cost, fast, and reliable solution for the visual data collection problem is favored. Synthetic data generation could be one of the possible solutions. The domain randomization technique has recently been promising in bridging the simulation-reality domain gap between synthetic and real-world visual data. We recognized the prior research outcomes and implemented domain randomization adopted synthetic image generation pipeline on top of Blender Python API. We proposed a group of configurable parameters to instruct how the pipeline constructs the 3D virtual scene, 3D objects, illuminations, and camera viewport. The pipeline triggers image rendering and ground-truth annotation for every constructed virtual scene. Ultimately, the pipeline applies configurable image noise and blur as extra domain randomization. The pipeline is a scalable and automatic solution for generating synthetic images for deep-learning model training.We generated synthetic training images for two manufacturing use cases. The model testing results show promising precision, recall, and [email protected] performance for the industrial parts detection use case where each category is separated. For the wheel-assembly inspection use case, further configurable parameters tunning may be needed for better model testing performance. Our performed ablation study provided insights that choosing textures based on domain knowledge could significantly bridge the virtual-reality domain gap. Further research could focus on testing configurable parameters on more real-world use cases and seeking ways to improve the recall scores for similar or small objects.Faculty of Science and Technology, Uppsala University. Place of publication Uppsala. Supervisor: Xiaomeng Zhu, Subject reader: Joakim Lindblad, Examiner: Nataša Sladoj