fix(training): make the stage-5 air-intercept drill physically solvable

productive_air_touch_fraction sat at exactly 0.0 across nine Stage-5
attempts and 540M timesteps. Two rounds of reward shaping were aimed at
it (air_approach_weight, then air_touch_bonus_weight); both worked --
airborne_fraction 0.223->0.258, mean_altitude 2.59->3.25,
vertical_thrust_mean 0.004->0.063 -- and the ship now visibly plays the
ball in the air. The metric could not see it because it counts only
touches with the ball above AIR_TOUCH_HEIGHT (5m), and
_place_air_intercept never produced a reachable one.

Simulating the spawn distribution against the ship's flight envelope
(vertical_thrust 120 / mass 5 = 24 m/s^2 less gravity, drag capping
climb near 12 m/s): a ball spawned 6-12m up at 6-11 m/s is above 5m for
a median of 0.80s, while the ship spawned 7-13m behind, 3-10m below, and
at a dead stop. An ideal interceptor -- point mass, instant attitude, no
righting torque, zero reaction delay -- makes that touch in 0.00% of
episodes and reaches the ball at all in 0.5%.

Retune the drill instead of the reward: ball higher (8-14m) and slower
(4-8 m/s), ship closer (4-9m behind), narrower lateral spread, and a
6-14 m/s planar run-up rather than a standing start -- the dead stop was
the largest single factor. Ideal interceptor now reaches the ball in
~98% of episodes and above 5m in ~37%, so the 0.005 floor has headroom.
AIR_TOUCH_HEIGHT stays 5.0 so the metric remains comparable with earlier
generations.

Resume from retry2 rather than restarting from Stage 4: that rule guards
against a changed reward function invalidating the value function, and
the reward function is untouched here -- only the state distribution
moved, so the policy that already learned to fly is what should be
pointed at a reachable target. Adds a one-shot resume_override to
generation5_state.json, consumed on first use.
This commit is contained in:
Josh Creek
2026-08-21 15:14:36 +01:00
parent 23e3dd18f9
commit 818f8e89cd
4 changed files with 146 additions and 13 deletions
+8 -3
View File
@@ -1,7 +1,7 @@
{
"stage_index": 1,
"attempt": 2,
"status": "blocked",
"attempt": 0,
"status": "in_progress",
"log": [
{
"stage_index": 0,
@@ -506,5 +506,10 @@
],
"decision": "fail"
}
]
],
"resume_override": {
"stage_index": 1,
"experiment": "20260821-0056-gen5-s5-intercepts-retry2",
"reason": "Stage 5 attempts 1-3 blocked on productive_air_touch_fraction=0.0, but the cause was _place_air_intercept's unreachable spawn geometry, not the policy or the reward function (see the Round 9 note in generation5.py). The drill was fixed; the reward function is unchanged, so this lineage's learned flight behaviour is kept rather than restarted from Stage 4."
}
}