Magic of Synthetic Data for Computer Vision Tasks

Using Blender and Unity3D to generate images for Computer Vision tasks

Introduction

Everything you see in the video below is synthetically generated, including the person.

The gap between synthetic and real images is slowly closing. Are we able to capitalize on this to help our computer vision projects?

This blog aims to provide a high level understanding on how we can leverage synthetic data to streamline computer vision projects.

In many projects we spend a significant amount of time searching for images that will eventually need to be labelled by humans.

Instead we will create a 3D model in Blender, and then use Unity Game Engine to simulate random images that are automatically labelled by the software.

Ultimately we are minimizing the data preparation stage (figure 1: collecting and labelling image data). This data will be used to train an object detector.

The diagram below shows the amount of time spent on different aspects of a object detection — computer vision project.

Figure 1: Approximate timeline to value for Computer Vision project

Object detection is a example of a computer vision solution used to classify objects and their location(s) in the image. Take a look at this COVID test detector that I created using synthetic images:

Figure 2: Performance on a batch of external images

The COVID test detector you saw above was trained using synthetic data and fine tuned with less than 100 real images.

The key objective is to detect a COVID test and remove the background. This reduces the chance of background objects/textures from influencing a separate classification model downstream.

Project Summary