Smart Viewpoint validation

A camera placement is a candidate, not a decision. Validation is what decides whether the frame is worth rendering, and it is the difference between forty usable frames and forty pictures of tarmac.

The loop#

sample a camera position
      │
      ▼
project the targets, test the candidate
      │
      ├── fails → sample again, up to the retry budget
      │
      └── passes → render the frame

The reference capture tried 181 placements to accept 40 frames: a median of 4 or 5 attempts per frame, a maximum of 9. Those 141 rejections are the feature. Each one is a frame that would have contained nothing worth annotating.

What a candidate is tested against#

The full set of controls, as recorded in a capture manifest:

min_targets                        enough objects in frame
max_targets                        not too many
min_visible_percent                enough of the scene visible
min_bbox_width_px                  targets large enough to annotate
min_bbox_height_px
min_camera_distance_m              not inside the subject
max_camera_distance_m              not so far it is meaningless
min_segmentation_coverage_percent  the identity buffer is not near-empty
min_depth_valid_percent            depth resolved for most of the frame
min_focus_score                    not out of focus
min_mean_luma                      not black
max_mean_luma                      not blown out
require_fully_visible              targets not clipped
reject_camera_collision            camera not inside geometry
reject_empty_segmentation          something is actually labelled
reject_invalid_depth
reject_under_landscape             camera not beneath the terrain
reject_blur
reject_exposure
max_retries                        the budget
fallback                           what to do when it runs out

Not all of these are enabled on every run. The reference capture, for instance, required at least 18 visible targets and allowed up to 32 retries, but left blur and exposure rejection off because it was using fixed lighting.

When the budget runs out#

The fallback policy decides. The reference capture used relax_constraints: rather than failing the frame outright, the requirements are loosened and the placement is retried.

This is a real trade-off worth making deliberately. Relaxing keeps the run moving and produces a frame that is worse than requested; failing keeps the standard and produces fewer frames. Which you want depends on whether an imperfect frame is more useful to you than no frame, and the recorded validation result on each frame tells you afterwards which ones were accepted the easy way.

Calibrating the thresholds#

The gate runs natively, but the thresholds are worth calibrating offline against a small sample first. Set min_targets too high for your scene and every placement is rejected; set it at one and you get frames containing a single distant crate.

A dry run and a smoke test exist to find that number before a full run does. See the camera and weather tutorial.

It is recorded per frame#

"viewpoint_validation": {
  "accepted": true, "attempts": 8,
  "total_targets": 289, "projected_targets": 138,
  "visible_targets": 73, "valid_targets": 73,
  "fully_visible_targets": 69,
  "visible_percent": 25.26, "depth_valid_percent": 100,
  "segmentation_coverage_percent": 9.58, "mean_luma": 165.2
}

From frame 000004 of the reference capture. This is what lets a coverage question have a measured answer rather than an impression.