The dataset pipeline

Everything between a raw dump and a graded dataset. All of it runs in Python, none of it needs Unreal, and any of it can be run again over a dump you already have.

The stages#

raw dump
   │
   ├─ verify      does the capture agree with itself?
   │
   ├─ label       derive boxes, masks and keypoints from the identity buffer
   │
   ├─ report      previews, distributions, quality gates
   │
   ├─ dataset     export to YOLO and COCO, with splits
   │
   └─ validate    grade the export and check it for leakage

From a terminal that is:

nameframe verify   _out/run1/dump
nameframe label    _out/run1/dump _out/run1/labels
nameframe report   _out/run1/dump _out/run1/report
nameframe dataset  _out/run1/dump _out/run1/dataset --format yolo,coco --val-frac 0.2
nameframe validate _out/run1/dataset --out _out/run1/validation

capture-unreal can run the whole chain in one call with --verify, --label-out, --report-out, --dataset-out and --validate-out. The Studio does the same thing when you press Generate.

Verify runs before labelling, on purpose#

It would be quicker to label first and check afterwards. It would also mean deriving thousands of annotations from a capture that is already wrong, and then discovering it at the point where the cost of being wrong is highest.

Verification asks whether the capture agrees with itself: whether identity colours collide, whether recorded actor positions land inside the blobs that claim them, whether the manifests reconcile against the run index, whether relabelling the same dump twice produces the same bytes.

What labelling does#

Reads the identity buffer, finds each instance's pixels, and turns them into annotations:

• Boxes from the extremes of the visible pixels, so occlusion and truncation are handled by construction rather than estimated.
• Masks from the pixels themselves, traced to polygons for COCO.
• Keypoints for pose-enabled classes, projected from native bones.

Fragments are reassembled here: an object seen through a railing arrives as several disconnected patches of one colour and becomes one instance rather than several.

Quality rules are applied at this stage too. A box below the minimum pixel size, or beyond the maximum aspect ratio, is rejected rather than exported as an unlearnable target. That is why the number of annotations a dataset publishes is smaller than the number of visible instances a scan of the buffer would count.

Export and splits#

--format takes yolo, coco, or both from one capture. Splitting is controlled by --val-frac and --test-frac, with --split-strategy choosing between hash, by_dump and sequential.

The strategy matters more than it looks. Consecutive frames of one capture are nearly identical, so a naive split puts near-duplicates on both sides of the boundary and produces a validation score that measures memorisation. The validator checks for exactly that afterwards.

Validation is a gate, not a report#

--fail-under makes the grade a build gate: below the threshold the command exits non-zero and a pipeline stops. See quality for what is checked and how the score is composed.

Re-deriving later#

Because every stage above reads the dump rather than the previous export, a capture from six months ago can become a dataset in a format that did not exist when it ran. Keep the dumps.