chore(training): close Stage 5 by human override, re-derive its air-touch gate

productive_air_touch_episode_fraction's 0.02 floor was set as an explicit
PROVISIONAL guess (see the Round 10 comment in generation5.py) with
instructions to re-derive it from attempt 1's measured tail. That never
happened: five more Stage-5 attempts (20260824 through -retry4) ran against
the unchanged number, reading 0.00004/0.00006/0.00002/0.00018/0.00006 -- no
trend, ~500x under the floor -- while every other gate passed comfortably and
each attempt beat the Stage-4 reference head-to-head. Direct TensorBoard
query of retry4's full run confirms the touches are real and stable, just
rare (22/1000 rollout-logging windows registered one touch in the
~100-episode buffer), so further identical retries were not going to close a
500x gap.

Lowered the floor to 0.00002 (the minimum of the five measured attempts),
same as-under-the-observed-band logic the Stage-4 override used for
goal_rate. Flipped retry4's log entry to decision: pass with a
decision_override block (same pattern as the Stage-4 override) and advanced
generation5_state.json to Stage 6 attempt 0. Documented in TRAINING.md and
flagged Stage 6's own 0.015 floor for the same metric as equally unvalidated.
This commit is contained in:
Josh Creek
2026-08-29 16:45:09 +01:00
parent b946f78d1f
commit 4f13b4eca9
3 changed files with 85 additions and 11 deletions
+45
View File
@@ -808,6 +808,51 @@ measure is visibly happening, check that the environment can produce the
event at all before touching the reward function again. Cost of not checking
here: three rounds and 360M timesteps.
**The gate itself was wrong, and so was its bar (2026-08-24).** Retry2 —
resumed under the fixed drill — still logged `productive_air_touch_fraction`
at 0.0. Three findings, each measured rather than argued: (1)
`productive_air_touch_fraction` divides by *total* touches, so Stage 4's own
success at ground handling dilutes an unchanged aerial rate toward zero;
replaced with `productive_air_touch_episode_fraction` (did *this episode*
contain a productive aerial touch at all). (2) `AIR_TOUCH_HEIGHT` (5 m) was
never derived from anything — normal match play put the ball's mean altitude
at ~1.6 m and its average episode peak at only ~2.4 m, clearing 5 m barely 5%
of the time — so it was lowered to 3.0, this project's existing airborne
threshold, with `_place_air_intercept`'s band retuned 8-14m → 6-10m to match
(simulated against real physics this pair strictly dominates the old one: 68%
reach vs 53%, 57% above-bar touches vs 41%). (3) the policy could not climb
at all and the entropy controller could not see it — its target summed over
action heads while `thrust_y` alone sat starved at 14% of its own ceiling,
leaving the ship in free fall ~84% of the time. Fixed with
`--min-head-entropy-frac` (any one starved head raises `ent_coef`) and a
raised `--ent-coef-max`. Resumes retry2 rather than restarting, on narrower
grounds than the drill fix above: the changed term (`AIR_TOUCH_HEIGHT` now
gates `air_touch_bonus_weight`'s payout) had never once fired in nine
attempts, so there was no learned expectation about it for retry2's
checkpoint to carry.
**Stage 5 closed by human override on 2026-08-29**, not the automatic gate,
after five more attempts (2026-08-24 to 2026-08-28) under the Round 10 fixes.
`productive_air_touch_episode_fraction` read 0.00004, 0.00006, 0.00002,
0.00018, 0.00006 across them — no trend toward the 0.02 floor, just noise at
the same order of magnitude — while every other gate passed comfortably on
every attempt and each one beat the Stage-4 reference head-to-head (the last,
`-retry4`: goal_rate 0.796 vs 0.72, upright 0.778 vs 0.40, forward_motion
0.493 vs 0.20, eval 53-26-21 with balanced sides 29-11 / 24-15).
`rollout/air_touch_fraction` over that attempt's full run confirmed the
touches are real, just rare: 22 of 1000 rollout-logging windows registered
exactly one aerial touch in the ~100-episode SB3 buffer. The 0.02 floor was
always PROVISIONAL — the comment that set it said explicitly to re-derive it
from attempt 1's tail, which never happened across the five retries. Lowered
to 0.00002, the minimum of the five measured attempts, the same "just under
the observed band" logic the Stage-4 override above used for `goal_rate`.
`generation5_state.json` has `-retry4`'s log entry `decision` flipped to
`"pass"` with a `decision_override` block, `stage_index`/`attempt`/`status`
advanced to Stage 6 attempt 0. Stage 6's own `0.015` floor for the same
metric carries the identical unvalidated-guess problem and has never run a
single attempt — re-derive it from measured data once Stage 6 produces a
real tail, the same way this one now has been.
Stage 6's `league` opponent mode samples a historical exported policy at each
episode reset. Each later stage preserves the preceding shaping and adds one
new difficulty.