Hard has been a label-only duplicate of medium.json since medium was promoted
on 2026-08-17. Promote 20260823-1734-gen5-s5-intercepts-retry2 into
hard.json so the tier is a genuinely distinct policy, and so the strongest bot
the curriculum has produced survives the next round's checkpoint pruning —
promoted files are never touched by training scripts.
Stage 5 blocked after three attempts, so like medium.json this comes from a run
recorded as decision: "fail". Both failing floors are covered in TRAINING.md:
goal_rate 0.7369 vs 0.75 is marginal, and productive_air_touch_fraction 0.0001
vs 0.005 is a bar no policy in the lineage has approached, against a metric
quantised at 0.01 per ~100-episode window. On every other axis it is the best
yet: upright_fraction 0.757 against a 0.40 floor that the pre-Round-6 lineage
never pushed past 0.331, and forward_motion_fraction 0.479 against 0.20.
Chosen over attempt 2 (retry1) on a tiebreak, not a margin. retry1 posts a much
wider indirect result against medium.json (63-23-14 vs 47-32-21), but a direct
100-episode head-to-head between the two finished 36-39 with 25 draws, so that
gap does not reflect a real strength difference. Attempt 3 is the later
checkpoint (it resumed from attempt 2) and edges every telemetry metric.
Verified: hard.json is byte-identical to its source export, matches easy/medium
on input_size 83, 3 layers and action space, and beats medium.json 19-7-4 in a
fresh 30-episode paired run. Tiers stay monotonic: hard > medium > easy.
That head-to-head also showed a 17% physical side imbalance (physical teams
0-1 = 29-46), reproduced at 13% in the 30-episode check. Inside the 20% bar
used elsewhere and equal across both models, but noted in TRAINING.md as worth
investigating rather than assuming variance.
20260816-2126-gen5-s4-handling-retry2 exhausted its three attempts and
missed only the 0.80 training goal-rate floor, at 0.7731. Every
evaluation gate passed: 65-22-13 versus promoted/easy.json, 87% non-draw
against an 80% floor, 12.6% physical-side imbalance against a 20%
ceiling, and both handling telemetry floors clear. The round improved the
goal rate monotonically across attempts (0.537 -> 0.683 -> 0.773) and the
checkpoint plays well by hand, so close Stage 4 by human override.
Promote it to Game/bots/promoted/medium.json. Medium and Hard both point
at the new policy: Hard stays a label-only duplicate until a stronger one
earns hard.json, which keeps the tiers monotonic rather than leaving Hard
weaker than Medium.
generation5_state.json flips that log entry to "pass" with a
decision_override block preserving the original verdict and reasoning,
and advances to Stage 5 attempt 1. This is what passing_entry() needs to
resolve Stage 5's resume checkpoint and evaluation reference, and what
league_pool() will need at Stage 6; --skip-to-next-stage would advance
the stage without marking anything as passing and die immediately.
generation5.sh now pulls before launching. Each stage ends in
commit_progress()'s push, which fails and kills the run hours in if the
box is behind origin.
Three curriculum generations (2026-07-21 through 2026-08-04) all tried
gating *when* the policy could use vertical thrust/pitch-roll on top of a
continuous Gaussian action space, and all three failed the same way: PPO's
action-distribution std collapsed within ~10% of steps and never recovered,
landing at a 15-32% win rate vs the grounded reference regardless of
mechanism (hard mask, then a gradual ramp). Generation 3's final attempt
just landed at 24% — the worst of the three.
Root cause, verified against this project's own physics: hovering this ship
requires *holding* thrust.y ~= 0.408 continuously (mass 5.0, vertical_thrust
120, gravity 9.8). A collapsed near-zero-mean Gaussian can brush that value
but never sustain it long enough to earn the reward gradient that would
move the mean — no amount of gating *when* the axis acts fixes a problem in
*how* the policy represents a decision on it. This also independently found
and fixes a real bug: godot_rl never marks an episode timeout as a
truncation, so PPO was bootstrapping V(s)=0 on every 30s draw in every
generation to date.
- Game/scripts/ship_action_codec.gd (new): single source of truth for a
per-axis MultiDiscrete action space (7 heads, nvec [5,5,5,5,5,5,2]) shared
by training and in-game inference, replacing the continuous Gaussian.
thrust_y's bins are deliberately asymmetric so a random policy drifts
through the volume instead of floor-pinning. Legacy continuous decode
(ai_ship_controller.gd's old logic) preserved verbatim so every
pre-generation-4 export (e.g. Game/bots/promoted/easy.json) keeps working
unchanged via an optional "action_space" JSON field.
- ship_observations.gd: append own contact state (SIZE 31 -> 35, append-only)
so the value function can see what wall_contact_penalty fires on.
- ship_ai_controller.gd: action space/decode via the codec; drop the
vertical_ramp/pitch_roll_ramp mechanism entirely; tilt_penalty default
lowered 4x (aerial approaches require pitching); flight telemetry
(airborne_fraction, mean_altitude, air_touch_fraction, vertical_thrust_mean)
and truncation-snapshot fields on get_info().
- training_mode.gd: new air_drill_chance state-setter branch (ball spawned
high, ships low, kept clear of walls) so aerial practice is forced by the
environment instead of relying on reward-driven exploration alone; snapshot
terminal observations before a timeout reset for the truncation fix.
- cosmic_env.py: remap ShipAIController's truncated/terminal_obs info into
SB3's TimeLimit.truncated/terminal_observation keys.
- train.py: --reset-logits (+ --reset-logits-heads) replaces the
now-meaningless --reset-std; new EntropyFloorCallback (a persistent
per-rollout ent_coef controller replacing the one-shot std-reset shock)
and per-head entropy logging; FlightTelemetryCallback; --air-drill-chance/
--tilt-penalty flags; optional AbortIfCallback kill-criterion.
- export_policy.py: writes the action_space block for MultiDiscrete models;
index-level parity check (argmax per head) instead of comparing floats.
- curriculum.py: full rewrite — 3 stages (bootstrap/selfplay/gauntlet), no
grounded stage, full action space live from step 1; deletes generation
1-3's checkpoint-lineage machinery (nothing to resume from); final report
evaluates against both promoted/easy.json and the new
promoted/reference-grounded.json (a copy of curric-s5-aggression, the
strongest grounded-era artifact, kept as a fixed yardstick).
- run_training.sh/.gitignore: commit only final.zip, not the ~2400
intermediate checkpoint files a single stage was writing (~500MB ->
~0.2MB per run); requirements.txt pinned (behaviour here now depends on
specific library internals, not just public APIs).
- test_action_space.py (new): offline rung-0 check catching a head-order
mismatch before it silently corrupts 24h of training.
Validated: GDScript compiles clean (Godot --headless --import + script
validation), free_play.tscn and training.tscn both boot headless without
errors, offline action-space assertions pass. Not yet run: the actual
smoke-training/A-B validation ladder steps in TRAINING.md's "Generation 4"
section, before committing to the full ~32h curriculum.
See TRAINING.md's "Generation 4" section for the full design writeup.
Josh Creek