How to export a depth map from Unreal Engine

What a grey PNG loses

Search this question and you will be pointed at a post process material that reads SceneDepth, divides it by some number until the picture looks right, and writes a PNG. That works for a visualisation and fails for anything else, for three reasons that all bite later rather than immediately.

A PNG holds eight bits per channel, so 256 distinct values. A scene spanning 24.8 to 194.7 metres, like the frame below, is being crushed into steps of roughly 0.67 metres. The divisor is picked by eye and is not written down anywhere, so nobody downstream can turn the grey back into a distance. And because it is chosen per scene, two captures of the same world are not comparable.

What a model or a point cloud actually needs is the number the renderer already had: the distance from the camera to the surface, before anyone normalised it for a screen.

That picture is the display version, colour mapped so the falloff is visible on a screen. The file behind it is not that picture. It is an array of 2,073,600 floating point numbers, and the grey was rendered from it for this page rather than the other way round.

What the capture writes

Every frame gets a depth.npy beside its image: a plain NumPy array, float32, the same shape as the frame, holding metres. No divisor, no clamp, no gamma. Four lines read it.

Because the units are real, the array answers questions a normalised image cannot. How far away is that object. How much of the frame is within twenty metres. Which pixels belong to a surface closer than the one the label claims. That last one is how occlusion gets decided.

Two quantities both called depth

This is the part that quietly ruins reprojections. Unreal’s SceneDepth is distance along the camera’s forward axis, the Z of the point in view space. The distance from the camera to the point, which is what you want for a point cloud, is longer everywhere except dead centre, and the gap grows toward the corners of the frame.

At a 90 degree horizontal field of view the corner of the frame is off axis by enough that the two differ by more than a third. If you unproject a depth map without accounting for it, your point cloud comes out dished: correct in the middle, pulled toward the camera at the edges. It looks plausible, so it usually gets through review.

Converting between them needs the camera intrinsics. A depth map without the camera it was taken with is half an export. The capture writes both into the same frame record, and there is a separate guide on getting the intrinsics out.

Three checks on the numbers

In this order.

Put a ruler in the scene. Place an object at a distance you know, read the pixel, compare. If the number is out by a constant factor you have a units problem, centimetres against metres being the usual one. If it is out by a factor that changes across the frame, you have the scene depth against ray distance problem above.

Look at the histogram, not the picture. A depth map that has been through a normalisation step somewhere has a suspiciously flat distribution and hard edges at 0 and 1. A real one is lumpy, with a spike wherever a large flat surface faces the camera.

Check the sky. Pixels with nothing in them have to hold something, and what they hold tells you what the pipeline did. A far plane value, an infinity, and a zero all mean different things, and code that averages over them without checking produces a number that is quietly meaningless.

What this does not give you

A rendered depth map is geometrically exact. That is also why it looks nothing like a depth sensor. There is no shot noise, no missing returns on dark or specular surfaces, no flying pixels at edges, no baseline gap where a stereo rig would see nothing. Training a depth estimator only on this and testing it on a real sensor is a well known way to be disappointed.

Treat it as the geometry it is: ground truth for supervision, and a source of occlusion facts. If you need sensor behaviour, model it deliberately on top, where the model is written down and can be changed.