Material and animation randomization

Varying how objects look and move, natively, deterministically, and with the level restored afterwards. Two separate rule sets that behave the same way.

What it is for#

A model trained on one appearance learns that appearance. If every forklift in your data is the same shade of yellow, the model has learned yellow as part of what a forklift is, and will hesitate at an orange one.

Material randomization varies surface appearance across frames. Animation randomization varies pose and motion state for skeletal actors, so a crowd is not forty copies of the same idle.

Rules#

Both are expressed as rule lists on the scenario, compiled into a randomization plan:

material_randomization_rules
animation_randomization_rules

A rule selects targets the same way a class does — by label, class or explicit target — and offers pools of materials or animations, optionally weighted.

Four properties that matter#

These are what separate this from "randomly change things and hope".

• Native. Applied in the engine during capture, so what is rendered is what was chosen. It is not a post-process approximation of a different material.
• Deterministic. Driven by the seed streams, so the same seed produces the same assignment. A frame that looked wrong can be regenerated exactly.
• Preflighted. Assets referenced by a rule are resolved before the run starts. A pool pointing at a material that no longer exists fails at validation rather than a thousand frames in.
• Restored. The level is put back afterwards. Randomization is a capture behaviour, not an edit to your project, and a run that leaves your scene recoloured is a run nobody will trust twice.

Where to be careful#

Randomising appearance is not free of consequences for the label. If a rule can assign a material that makes two classes visually indistinguishable, the dataset now contains examples that a human could not classify either.

Vary within what the class actually means. A forklift can be any colour; a warning sign that is randomised into an unreadable colour has stopped being a warning sign.

Related#

Sensor and lens effects also vary appearance, but offline and for a different reason: they simulate the camera rather than the world. See the execution model for why that distinction is kept.