mirror of
https://github.com/jcreek/CosmicClash.git
synced 2026-09-17 12:52:05 +00:00
feat(bots): promote gen5 stage-5 policy to the Hard tier
Hard has been a label-only duplicate of medium.json since medium was promoted on 2026-08-17. Promote 20260823-1734-gen5-s5-intercepts-retry2 into hard.json so the tier is a genuinely distinct policy, and so the strongest bot the curriculum has produced survives the next round's checkpoint pruning — promoted files are never touched by training scripts. Stage 5 blocked after three attempts, so like medium.json this comes from a run recorded as decision: "fail". Both failing floors are covered in TRAINING.md: goal_rate 0.7369 vs 0.75 is marginal, and productive_air_touch_fraction 0.0001 vs 0.005 is a bar no policy in the lineage has approached, against a metric quantised at 0.01 per ~100-episode window. On every other axis it is the best yet: upright_fraction 0.757 against a 0.40 floor that the pre-Round-6 lineage never pushed past 0.331, and forward_motion_fraction 0.479 against 0.20. Chosen over attempt 2 (retry1) on a tiebreak, not a margin. retry1 posts a much wider indirect result against medium.json (63-23-14 vs 47-32-21), but a direct 100-episode head-to-head between the two finished 36-39 with 25 draws, so that gap does not reflect a real strength difference. Attempt 3 is the later checkpoint (it resumed from attempt 2) and edges every telemetry metric. Verified: hard.json is byte-identical to its source export, matches easy/medium on input_size 83, 3 layers and action space, and beats medium.json 19-7-4 in a fresh 30-episode paired run. Tiers stay monotonic: hard > medium > easy. That head-to-head also showed a 17% physical side imbalance (physical teams 0-1 = 29-46), reproduced at 13% in the 30-episode check. Inside the 20% bar used elsewhere and equal across both models, but noted in TRAINING.md as worth investigating rather than assuming variance.
This commit is contained in:
+30
-6
@@ -181,12 +181,36 @@ promoted file is never touched by training scripts, never overwritten by a
|
||||
same-named future export, and never disturbed by pruning old experiment
|
||||
files from the flat dump.
|
||||
|
||||
Easy and Medium are now genuinely different policies. Hard still points at
|
||||
`medium.json` and remains a label-only duplicate until a stronger policy earns
|
||||
`hard.json`. Every tier runs at full trained cadence (`reaction_ticks=8`,
|
||||
`action_noise=0`) — the game does not manufacture difficulty gaps by
|
||||
handicapping a model. When promoting, keep the tiers monotonic: a lower tier
|
||||
must never point at a policy that beats the tier above it.
|
||||
`hard.json` was promoted 2026-08-24 from generation 5's
|
||||
`20260823-1734-gen5-s5-intercepts-retry2` — Stage 5 attempt 3, and like
|
||||
`medium.json` above it is recorded as a *fail* in `generation5_state.json`.
|
||||
Stage 5 blocked after three attempts on two telemetry floors: `goal_rate`
|
||||
0.7369 against a 0.75 bar (marginal), and `productive_air_touch_fraction`
|
||||
0.0001 against 0.005. The second is not a judgement about this policy — see
|
||||
"Generation 5" below — the metric is quantised at 0.01 (one touch per ~100
|
||||
episode logging window), so a 0.005 floor demands a productive air touch in
|
||||
half of all windows, and no policy in the lineage has ever come close. On
|
||||
everything else it is the strongest bot produced so far: `upright_fraction`
|
||||
0.757 against a 0.40 floor (the pre-Round-6 lineage never exceeded 0.331),
|
||||
`forward_motion_fraction` 0.479 against 0.20, and it beats `medium.json`
|
||||
47-32-21 over 100 paired episodes.
|
||||
|
||||
Attempt 2 (`20260823-0258-gen5-s5-intercepts-retry1`) posts a much wider
|
||||
margin against `medium.json` (63-23-14) and was the obvious alternative, but a
|
||||
direct 100-episode head-to-head between the two finished 36-39 with 25 draws —
|
||||
a dead heat, so the wider indirect margin does not reflect a real strength
|
||||
difference. Attempt 3 was taken on the tiebreakers: it is the later checkpoint
|
||||
(it resumed from attempt 2) and edges every telemetry metric. That head-to-head
|
||||
also measured a 17% physical side imbalance (physical teams 0-1 = 29-46, with
|
||||
`retry1` going 15-25 as team 0 but 21-14 as team 1) — inside the 20% bar used
|
||||
elsewhere, but large enough to be worth understanding rather than assuming it
|
||||
is noise, since it affects both models equally and may point at an asymmetry in
|
||||
the arena or in `ship_observations.gd`'s team-1 mirroring.
|
||||
|
||||
Every tier runs at full trained cadence (`reaction_ticks=8`, `action_noise=0`)
|
||||
— the game does not manufacture difficulty gaps by handicapping a model. When
|
||||
promoting, keep the tiers monotonic: a lower tier must never point at a policy
|
||||
that beats the tier above it.
|
||||
|
||||
To promote a new bot into a tier: copy the chosen `Game/bots/<experiment>.json`
|
||||
to `Game/bots/promoted/<tier>.json` (overwriting the old one), and note the
|
||||
|
||||
Reference in New Issue
Block a user