A temperature per class
A clothed person reads near 33 C, not 37: clothing insulates and the camera sees the coat. Bare skin runs warmer, which is why a face stands out from a body in a real image.
How to generate thermal synthetic data in Unreal Engine
Thermal training data is hard to collect, expensive to label and exactly the case a rendered scene should be good at. It is also the modality where synthetic work most often cheats: a grey-and-glow filter over a photograph looks like thermal and teaches a model nothing, because the filter never learns which pixels are warm.
- The wrong approach
- A colour filter over the visible frame
- The right one
- A temperature per object, then a palette
- The test
- Contrast inverts between noon and midnight
- Train on
- White hot, and one palette only
Why a thermal look is not thermal data
Most tools that offer thermal offer a lens preset: desaturate, raise contrast, add bloom, maybe tint. Applied to a daylight photograph it looks convincing in a thumbnail. Applied to a night scene it produces three dark smudges where three people are standing, because the people were dark in the photograph and a colour transform cannot know they were warm.
That failure is the whole difference. A thermal image is a map of surface temperature. A visible image is a map of reflected light. They agree by accident in daylight and disagree completely at night, which is when thermal is worth having at all.
Where a rendered scene gets its temperatures
The renderer does not compute heat, but it knows something almost as useful: which object every pixel belongs to. That is the same buffer instance segmentation comes from, and it turns temperature into a lookup rather than a simulation.
A workable model needs four things, and none of them require a physics solver:
Absolute or relative to air
A person is 33 whether the air is 5 or 30. A parked car is a few degrees over whatever the air is. That distinction is what makes contrast invert between day and night rather than staying fixed.
Attenuation with distance
Air absorbs and re-emits, so a warm target loses contrast against the background as it gets further away. Without this, a person at 300 metres reads as brightly as one at 30, and a detector learns that distance costs a target nothing.
Sensor noise
Real uncooled cores quote a noise-equivalent temperature difference around 40 to 60 mK. A noiseless thermal image is as unrealistic as a noiseless photograph and trains a model to expect clean edges.
Day and night are opposite problems
In daylight, surface temperature tracks insolation: a road in full sun can sit twenty degrees over air temperature, and a person in the same frame is cooler than the ground. Image brightness is a decent stand-in for background warmth here, because the same sun did both jobs.
After dark that proxy is worthless. A bright patch in a night photograph is a street lamp, not a warm surface. Background temperature should be flat ambient instead, with one physical effect layered on: surfaces that see open sky radiate to it and cool, so a clearing runs colder than ground under a canopy. That single term is most of the texture in a night thermal image, and without it the background is a flat grey field.
Choosing a palette, and sticking to it
White hot is what almost every published thermal dataset encodes, and what you should train on. Black hot is the same field inverted; some operators prefer it, and a model trained on one and shown the other fails completely, so never mix them in a corpus. Ironbow and the other colour ramps read beautifully to a person and are a poor training target, because they spend three channels encoding one number.
One more thing that costs a day if you get it wrong: fix the display range across the whole run. A real camera re-ranges every frame, and reproducing that faithfully means a wall changes shade the moment a hot object leaves the picture. Consecutive frames stop being comparable, and any temporal model sees flicker that is not in the scene.
Check it the way you would check any label
Two checks, both cheap. First, render one scene at a warm ambient and a cold one and confirm the contrast inverts. If your people stay bright in both, temperature is not following the objects and you have built a colour filter with extra steps.
Second, look at the histogram. A thermal frame of a night scene should be almost entirely cold with a very small warm tail, because that tail is three people in a field of trees. If your display range is set from an ordinary percentile, that tail gets clipped to background and the targets disappear from the image while remaining in the labels.
Say what the model does not do
This part matters more for thermal than for any other modality, because thermal looks authoritative. A class temperature table does not model emissivity, so polished metal will read far colder in reality than in your corpus. It does not model reflected sky, so glass and still water will surprise you. It does not model thermal history, so a car parked an hour ago and one parked a minute ago look the same. And it does not model the sensor’s spectral response, so a mid-wave core sees a different scene from a long-wave one.
Ship that list with the data. A synthetic thermal corpus is useful for geometry, occlusion, framing and labels, and it is not a substitute for radiometric truth. A reader who finds that out from you trusts the rest; one who finds out from their own results does not.
Doing it with NameFrame
Thermal is derived after the capture rather than rendered during it, so any existing run with depth can be turned into one without going back to the editor. The class temperatures, the ambient, the sky and the atmosphere are all arguments, and a manifest beside the frames records exactly which model produced them.
nameframe thermal D:/runs/wood D:/runs/wood/thermal \
--palette white_hot,ironbow --ambient 7 --sky -50 \
--background ambient --sky-view-cooling 3.5The measured behaviour, the palettes and the full list of what the model does not do are on the thermal page.