Colour collisions
Two objects assigned the same identity colour merge into one instance. Checked on every run with zero tolerance for near-matches. Reference capture: 289 colours across 289 classes, no collisions.
Ground truth
Ground truth is read out of the engine, not drawn on top of it.
“Ground truth” in a real dataset means a human’s best guess, agreed between two annotators and a review pass. In a rendered scene it means something stricter: the values the renderer used to draw the frame. This page is about where those values come from and how you check that they are right.
- Source
- Unreal Engine 5.8
- Coordinate frame
- ned-z-down
- Positions
- metres, per actor, per frame
- Verification
- pixels vs transforms
Three sources, one frame
Everything NameFrame writes comes from three places, and it is worth being precise about which is which, because they fail differently.
- The identity buffer. A second render of the same frame in which every object is painted a unique flat colour. This is what makes boxes and masks exact rather than approximate: a pixel either belongs to an object or it does not, and occlusion resolves itself because the renderer already resolved it.
- Actor transforms. The position, rotation and scale of every object in the scene at the moment of capture, in metres, recorded alongside the camera pose. This is the three-dimensional truth that the two-dimensional labels are a projection of.
- The depth buffer. Distance per pixel, in metres, as float rather than as an 8-bit picture of distance.
Why having both matters
Pixels and transforms are two independent descriptions of the same scene, which means they can be checked against each other. Project each recorded actor position through the camera and it should land inside the blob that the identity buffer says belongs to that actor. If it does not, something is wrong: the wrong colour, a stale transform, an object that moved between the render and the record.
That check runs on every capture. On the reference run its result was: 197 blobs match their recorded actor positions. A pipeline that only produced masks would have no way to run it.
What can still go wrong
Engine-derived does not mean infallible. These are the real failure modes, and what the pipeline does about each.
Cross-painting
A thin object in front of another can take pixels that belong to its neighbour, leaving the neighbour with no box at all. This is the check the reference capture failed, and it is reported rather than hidden.
Fragmented objects
An object seen through a railing arrives as several disconnected blobs. Those are reassembled into one instance rather than counted as several: 1,409 fragments merged across the reference run.
Identity withheld
Some actors deliberately carry no identity, so they render into the picture but never into a label. 14 in the reference capture. Distinguishing these from a bug matters.
The coordinate details people actually need
Most of the trouble in a synthetic pipeline is a sign convention nobody wrote down.
- Positions are in metres, not Unreal’s centimetres. The conversion happens once, at capture, so nothing downstream has to remember.
- The camera frame is ned-z-down, recorded per frame with position, quaternion and Euler angles, so you do not have to reconstruct one from the other.
- Horizontal field of view is 64 degrees on the reference capture, and camera intrinsics are exported with the frame.
- Depth is metres per pixel as float32. Anything you see on this site as a colourful depth image is a visualisation of that array, never the array itself.
All of it is in the frame records inside the downloadable sample packs, so you can check the conventions against real files rather than against this paragraph.