chore(training): add air_approach_weight and restart stage-5 intercepts

Stage 5 blocked all three attempts on productive_air_touch_fraction
stuck exactly at 0.0 across a continuous 180M-step lineage, while
goal_rate/upright_fraction/forward_motion_fraction kept improving on
the same budget. forward_velocity_to_ball_weight (the term that solved
Stage 4's ground pursuit) is hard-gated below GROUND_HANDLING_HEIGHT
and does nothing in the air, so Stage 5's air_intercept_chance had no
matching aerial incentive to learn from. air_approach_weight adds the
airborne mirror (nose-first 3D closing speed, no uprightness
multiplier) and folds into HANDLING_REWARD_FLAGS so Stage 6 inherits
it too. Deleted the three blocked attempts and reset state to resume
Stage 5 from the Stage-4 checkpoint with the new term.
This commit is contained in:
Josh Creek
2026-08-18 16:03:56 +01:00
parent 0e685edcf8
commit 88591e031f
15 changed files with 93 additions and 254 deletions
+23
View File
@@ -639,6 +639,29 @@ marking anything as passing, so the run dies immediately with `RuntimeError:
No passing generation-5 stage index 0`. Note that retry2 already clears Stage
5's own 0.75 goal-rate floor; the 0.80 Stage-4 figure was always the stricter
of the two.
**Stage 5 (`intercepts`) blocked after its own three attempts on 2026-08-18**,
all on the same single floor: `rollout/productive_air_touch_fraction` stayed
exactly 0.0 across a continuous 180M-step lineage (retries resume the
previous attempt's checkpoint, so this is one training run, not three), while
`air_touch_fraction` sat at noise level (0.00008 → 0.00006 → 0.00006) and
`goal_rate`/`upright_fraction`/`forward_motion_fraction` all kept improving on
the same budget — a dead-flat metric next to ones that keep moving, the same
missing-mechanism signature as Stage 4's original plateau, not a slow-learning
one. The cause: `forward_velocity_to_ball_weight`, the term that actually
taught ground pursuit, is hard-gated below `GROUND_HANDLING_HEIGHT` and does
nothing in the air, so Stage 5's `air_intercept_chance` was asking for aerial
pursuit with only the generic, orientation-agnostic `velocity_to_ball_weight`
(0.04) to learn it from. `air_approach_weight` (`ship_ai_controller.gd`) adds
the airborne mirror — nose-first 3D closing speed on the ball, active above
`GROUND_HANDLING_HEIGHT`, no uprightness multiplier since a real aerial
requires pitching away from level — set to 0.15 to match
`forward_velocity_to_ball_weight`'s proven magnitude, and folded into
`HANDLING_REWARD_FLAGS` so Stage 6 inherits it too. The three blocked attempts
were deleted and Stage 5 restarts from Stage 4's checkpoint with the new term,
same reasoning as every previous mechanism change: don't resume a policy
shaped by an absent term into one where it now exists.
Stage 6's `league` opponent mode samples a historical exported policy at each
episode reset. Each later stage preserves the preceding shaping and adds one
new difficulty.