Depth estimation supervision
Dense, exact, and free with the frame. No sensor noise, no missing returns on dark or shiny surfaces, no range ceiling. Which is both the advantage and the catch below.
Depth
Depth in metres, not a picture of depth.
Most depth you find in a synthetic dataset is an 8-bit greyscale image, which means it has been quantised to 256 values and normalised against a range nobody wrote down. What NameFrame writes is the array the engine had: float32 metres, one value per pixel, unnormalised.
- Type
- float32, m
- Resolution
- 1920×1080
- Median depth
- 63.8 m
- 1st to 99th
- 24.8–194.7 m
The picture above is not the data
It cannot be. A float array spanning 24.8 to 194.7 metres has no visual form; putting it on a screen means choosing a range and a colour map, and both choices are display decisions rather than properties of the capture. The image is turbo-mapped between the frame’s own first and ninety-ninth percentile, because a linear grey ramp over the full range collapses an aerial frame into a pale wash.
The file beside it holds the numbers. That distinction matters more than it sounds: a pipeline that only ships the visualisation has thrown away the measurement and kept the picture of it.
| Property | Value |
|---|---|
| Array type | float32 |
| Shape | 1080 × 1920 |
| Values | 2,073,600 |
| Unit | metres from the camera |
| Finite values | 100% |
| 1st percentile | 24.8 m |
| Median | 63.8 m |
| 99th percentile | 194.7 m |
Read the percentiles together and the scene appears: half the frame is within 63.8 metres, which is the ground directly below the camera, and the last percent stretches to 194.7, which is the far end of the airfield. That spread is why the visualisation needs a perceptual colour map and why an 8-bit export would waste most of its range on ground the model does not care about.
Reading it
It is a NumPy array. There is no format to parse and no scale factor to look up.
import numpy as np
depth = np.load("depth/plugin_000004.npy") # (1080, 1920) float32
print(depth.dtype, depth.shape)
print("nearest", depth.min(), "m farthest", depth.max(), "m")
# distance to a specific instance: mask it from the identity buffer, then read
import cv2
seg = cv2.imread("segmentation/plugin_000004.png")[:, :, ::-1]
mask = np.all(seg == (231, 76, 60), axis=-1) # one instance colour
print("median distance to that object:", np.median(depth[mask]), "m")That last block is the whole reason depth and the identity buffer ship together. Either one alone answers half a question; together they say “this object, this far away”, which is what a depth-supervised model needs.
What depth is good for, and what it is not
Distance per instance
Combined with the identity buffer, every annotated object gets a real distance. The reference capture’s instances sit between 33.2 and 187.8 metres from the camera.
Not a substitute for a real sensor
A LiDAR or stereo rig has noise, dropouts and a range limit. A model trained only on perfect depth learns to trust it. If you deploy against a real sensor, degrade the synthetic depth deliberately.
Storage is the real constraint
2,073,600 float32 values is 8 MB per frame. A ten-thousand frame run is 80 GB of depth alone, which is why depth writing can be turned off for long soaks and why the sample packs ship only a few frames of it.