Synthetic Data Generation for AI-based Machine Vision Applications

TL;DR: A method for synthesizing sensor data for machine vision tasks is presented. It generates realistic images and annotations for object detection, segmentation, and pose estimation. The method uses physically based rendering techniques and incorporates material properties and lighting conditions. It also introduces synthetic defects for quality control applications.

Abstract: This paper presents a method for synthesizing 2D and 3D sensor data for various machine vision tasks.Depending on the task, different processing steps can be applied to a 3D model of an object.For object detection, segmentation and pose estimation, random object arrangements are generated automatically.In addition, objects can be virtually deformed in order to create realistic images of non-rigid objects.For automatic visual inspection, synthetic defects are introduced into the objects.Thus sensor-realistic datasets with typical object defects for quality control applications can be created, even in the absence of defective parts.The simulation of realistic images uses physically based rendering techniques.Material properties and different lighting situations are taken into account in the 3D models.The resulting tuples of 2D images and their ground truth annotations can be used to train a machine learning model, which is subsequently applied to real data.In order to minimize the reality gap, a random parameter set is selected for each image, resulting in images with high variety.Considering the use cases damage detection and object detection, it has been shown that a machine learning model trained only on synthetic data can also achieve very good results on real data.