Skip to content
NAMEFRAMECommercial PreviewApply for Pilot
Apply for Pilot
Guide

How to generate synthetic data in Unreal Engine

Seven steps from an existing level to a validated, exported dataset. The numbers in this guide are from the 40-frame Map_Airbase_Demo capture used across this site, so you can compare what you get against what it got. You can follow it in the Studio panel without writing anything, or from a terminal.

Engine
Unreal Engine 5.8
Time to first frames
under an hour
Code required
none, or all of it
Artifact
downloadable sample
  1. 01

    Pick a scene and decide what you are labelling

    Any UE 5.8 level works, including one you already have. What matters is that it contains the objects you care about, or that you can place them.

    Decide the classes before anything else. Six is plenty to start with. Every class needs actors in the level that belong to it, and every actor you want labelled needs to belong to a class, because anything unassigned renders into the picture but never into a label.

  2. 02

    Draw the zones

    Four kinds of volume drive everything. Spawn zones say where objects may be placed. Target zones say what the camera should look at. Exclusion zones carve out places nothing may go. Camera zones say where the camera may stand.

    This is authoring, not randomising. A spawn zone drawn across a runway will put crates on the runway, which may be exactly what you want or a mistake you will only notice in frame 400.

  3. 03

    Set the population rules

    How many of each class, and how they are distributed: scattered on surfaces, on a grid, along a spline, in clusters, relative to anchors, or restricted to navigable ground.

    Set minimum spacing and maximum slope here. The reference capture used 1 metre spacing and a 12 degree slope limit, and both are visible in the output afterwards.

  4. 04

    Validate before you render anything

    Validation resolves every asset the spawner will reach for and checks the scene against the recipe. On the reference run that was 38 assets across 4 pools.

    This step costs seconds and catches the mistakes that would otherwise cost you a run: a class with no actors, a zone with no navigable ground, an asset that no longer exists.

  5. 05

    Dry run, then smoke test

    A dry run plans every placement without rendering. The reference run planned 3,360 placements and refused 0.

    A smoke test renders a handful of frames end to end and puts them through the same gates a full run uses. Camera placements that cannot see enough get rejected and resampled: 181 placements were tried to accept 40 frames.

  6. 06

    Generate

    By this point nothing is new. The full run uses the configuration the three previous steps agreed on, which is why it should be the boring step.

    It writes a raw dump: rendered frames, the identity buffer, depth in metres, a per-frame record of every actor and camera, and a spawn manifest per frame.

  7. 07

    Label, verify and export

    Labels are derived from the dump afterwards rather than during capture, which is what lets you re-export a run into a different format without going back to the editor.

    Verification then checks the labels against the geometry, and the export writes YOLO and COCO. Read the report before you train on anything.

The same seven steps, from a terminal

The Studio writes the job file these commands read, so the two paths are the same pipeline with different front doors.

# 04  validate the scene against the recipe, refusing an incomplete one
nameframe scene-validate scene_manifest.json --generation-ready

# 05  plan every placement without rendering, then render a few frames end to end
nameframe studio-dry-run job.json _out/dry
nameframe live-smoke --remote-url http://127.0.0.1:30010 --frames 8 --out-root _out/smoke

# 06  the full run; the job file names the dump directory
nameframe capture-unreal job.json --remote-url http://127.0.0.1:30010

# 07  derive labels, verify them against the geometry, export and grade
nameframe label    _out/run1/dump _out/run1/labels
nameframe verify   _out/run1/dump
nameframe report   _out/run1/dump _out/run1/report
nameframe dataset  _out/run1/dump _out/run1/dataset --format yolo,coco
nameframe validate _out/run1/dataset --out _out/run1/validation

Steps six and seven collapse into one call if you prefer: capture-unreal takes --verify, --label-out, --report-out, --dataset-out and --validate-out, and runs the whole chain in order.

Read _out/run1/report/report.json before you train on the result. It holds the quality gates, the verification checks, and the class balance. If a check failed, it says so there.

What to check in your first run

Four things go wrong more often than everything else combined.

Objects floating or buried

Ground contact is traced and reported per actor. Some floating is legitimate, a crate on a roof, and the report names the surface each actor is resting on so you can tell which is which.

Frames that see nothing

If viewpoint validation is off or too permissive, you get frames of empty tarmac. Set a minimum number of visible targets and let rejected placements be resampled.

Everything at one distance

A single camera zone produces a single distance band. Check the distance distribution in the report before generating thousands of frames of the same shot.

Class balance you did not intend

Scene contents dominate whatever you spawn. The reference capture ended up with eighteen barrels per tank, which is fine if you know, and a problem if you find out after training.