Keypoints and pose

Skeleton keypoints projected from native Unreal bones, for classes whose actors actually have skeletons. Opt-in per class, and worth being conservative about.

The keypoint order#

The current build defines 13 keypoints, in this order:

head
left_shoulder
right_shoulder
left_elbow
right_elbow
left_wrist
right_wrist
left_hip
right_hip
left_knee
right_knee
left_ankle
right_ankle

Order matters: it is the order they are written in, and the index a training format binds to. It comes from the code rather than from this page, so adding one updates the list here.

Where they come from#

Bones in the engine, projected into image space through the recorded camera. Not inferred from pixels, and not estimated by a model. That is the whole reason to generate pose data synthetically: keypoint annotation by hand is slow and inconsistent, and a pose model trained on another model's predictions inherits its errors.

Visibility is derived alongside the position, so a keypoint behind the body is marked rather than silently placed.

Enable it per class, deliberately#

Pose should only be enabled for classes whose native skeleton and bone evidence supports it. Enabling it on a class with no skeleton does not produce nothing; it produces confidently wrong keypoints, which is worse.

Classes that usually should not carry pose:

vehicles
props
food
tools
most animals

Export#

Pass --task pose to the dataset command. The COCO exporter writes person_keypoints alongside instances where configured; see outputs and formats.

Verification#

A dedicated check compares native bone evidence against the projected offsets. On a capture with no pose-enabled actors it reports that it was skipped rather than passing silently, which is the distinction between "checked and fine" and "not checked".