mirror of
https://github.com/jcreek/CosmicClash.git
synced 2026-09-10 16:04:04 +00:00
feat(training): add generation 5 curriculum
This commit is contained in:
+84
-9
@@ -162,11 +162,9 @@ flat files there, never touching subdirectories.
|
||||
|
||||
`Game/bots/promoted/<tier>.json` is the small, curated, hand-maintained set
|
||||
actually referenced by the shipped game — currently `easy.json` (promoted
|
||||
2026-07-24 from `curric-s6-unmask`, the strongest checkpoint at the
|
||||
time — note `curric-s6-unmask` was itself generation 1's *failed* unmask
|
||||
stage, so `easy.json` is weaker than `reference-grounded.json` below; a
|
||||
strong generation 4 result should promote a real replacement, plus
|
||||
`medium.json`/`hard.json`) and `reference-grounded.json` (added for
|
||||
2026-08-08 from generation 4's `20260806-1939-curric-s3-gauntlet`; this is
|
||||
the 320M-step MultiDiscrete policy and the foundation for the planned
|
||||
generation-5 curriculum below) and `reference-grounded.json` (added for
|
||||
generation 4 — a copy of generation 3's `curric-s5-aggression`, made before
|
||||
the flat `Game/bots/` dump was scrapped for the redesign, kept as the
|
||||
strongest grounded-era artifact and the fixed yardstick generations 1-3 were
|
||||
@@ -176,6 +174,11 @@ promoted file is never touched by training scripts, never overwritten by a
|
||||
same-named future export, and never disturbed by pruning old experiment
|
||||
files from the flat dump.
|
||||
|
||||
Until distinct `medium.json` and `hard.json` policies earn promotion, the
|
||||
three menu tiers all run this same `easy.json` policy at its full trained
|
||||
cadence (`reaction_ticks=8`, `action_noise=0`). The tiers are labels only;
|
||||
the game does not manufacture difficulty gaps by handicapping this model.
|
||||
|
||||
To promote a new bot into a tier: copy the chosen `Game/bots/<experiment>.json`
|
||||
to `Game/bots/promoted/<tier>.json` (overwriting the old one), and note the
|
||||
source experiment + date in this section. Do this for `medium.json`/
|
||||
@@ -493,10 +496,13 @@ alone (`train.py` only forwards a flag when you pass it), so ordinary runs
|
||||
are unaffected. Full flag list: `--opponent-mode {self_play,inert,frozen}`,
|
||||
`--opponent-model <path>` (for `frozen`), `--draw-penalty`,
|
||||
`--attack-goal-bias`, `--kickoff-chance`, `--near-goal-chance`,
|
||||
`--air-drill-chance` (generation 4's state-setter aerial curriculum),
|
||||
`--velocity-to-ball-weight`, `--ball-distance-penalty`, `--ball-touch-reward`,
|
||||
`--airborne-penalty`, `--tilt-penalty`, `--ball-velocity-to-goal-weight`,
|
||||
`--goal-reward`. (`--vertical-ramp`/`--pitch-roll-ramp` are gone — generation
|
||||
`--air-drill-chance`, `--air-intercept-chance`, `--team-size`,
|
||||
`--velocity-to-ball-weight`, `--forward-velocity-to-ball-weight`,
|
||||
`--ball-distance-penalty`, `--ball-touch-reward`, `--airborne-penalty`,
|
||||
`--tilt-penalty`, `--ground-tilt-penalty`, `--speed-reward-weight`,
|
||||
`--ball-velocity-to-goal-weight`, `--goal-reward`, and
|
||||
`--opponent-pool` with `--opponent-mode=league`.
|
||||
(`--vertical-ramp`/`--pitch-roll-ramp` are gone — generation
|
||||
4 has no locomotion mask/ramp to control.)
|
||||
|
||||
### Running it automatically
|
||||
@@ -546,6 +552,75 @@ Running a stage by hand (e.g. to experiment with flags before trusting the
|
||||
orchestrator) still works exactly as the table above describes — just call
|
||||
`next_run.sh`/`run_training.sh` directly with that stage's flags.
|
||||
|
||||
### Generation 5 follow-on
|
||||
|
||||
Generation 4's stage-3 export is the foundation rather than a throwaway
|
||||
baseline: all generation-5 stages resume from
|
||||
`checkpoints/20260806-1939-curric-s3-gauntlet/final.zip`. Its match results
|
||||
are strong, but playtesting and its final telemetry expose the next learning
|
||||
targets: it spends about 39% of play above the airborne threshold while only
|
||||
about 0.04% of episode-level touches are aerial, and it often travels on its
|
||||
side and strikes the ball with its roof. This is a successful scoring policy
|
||||
that now needs control quality and a more productive use of flight.
|
||||
|
||||
Turbo remains forward-only for players and policies: it activates only with
|
||||
positive forward thrust and multiplies the resulting combined thrust vector.
|
||||
Generation 5 preserves the same control contract Stage 3 was trained under.
|
||||
|
||||
Generation 5 adds three episode telemetry signals to TensorBoard:
|
||||
`upright_fraction` (low-altitude ticks with the
|
||||
ship's up vector substantially upright), `forward_motion_fraction`
|
||||
(low-altitude moving ticks whose planar velocity points broadly along the
|
||||
nose), and `productive_air_touch_fraction` (touches above the aerial height
|
||||
that send the ball toward the attack goal). The automatic gates are
|
||||
deliberately conservative catastrophe floors; every stage records its final
|
||||
500-rollout tail means in `generation5_state.json` so later threshold changes
|
||||
can be based on evidence instead of a single watched match.
|
||||
|
||||
| Stage | Regime | Budget | Learning target | Advancement gate |
|
||||
|---|---|---:|---|---|
|
||||
| 4 — `handling` | Self-play, current balanced start mix | 40M (~4h) | Prefer upright, nose-led travel near the floor. Replace the orientation-agnostic speed bonus with low-altitude forward-motion shaping, and apply the stronger tilt cost only near the floor so pitch/roll remain free in genuine aerial play. | Before Stage 5: at least 80% training goal rate, at least 80% non-draw rate in the paired evaluation versus promoted Stage 3, no clear head-to-head regression, no more than 20% physical-side win imbalance, and the upright/forward-motion telemetry floors. |
|
||||
| 5 — `intercepts` | Self-play with 40–50% improved air-intercept starts | 60M (~6h) | Convert existing vertical movement into useful aerial touches. Spawn a moving high ball on reachable attacking and defensive trajectories, away from walls, so contact is instrumental to scoring or saving rather than independently rewarded. | No clear regression versus Stage 4; productive aerial-touch telemetry must improve materially without reducing upright/forward-motion telemetry back to the Stage-3 baseline. |
|
||||
| 6 — `league` | Live policy against a frozen opponent sampled per episode from Stage 3, Stage 4, and Stage 5 | 100M (~10h) | Prevent a narrow self-play equilibrium and consolidate ground handling, aerial interception, attack, and defence against distinct styles. | No clear head-to-head regression against any pool member plus conservative handling/aerial telemetry floors. Promote the passing result to `medium.json` after these recorded evaluations support it. |
|
||||
|
||||
Stage 7 teamplay remains deliberately unconfigured. The fixed roster
|
||||
observation and `team_size` plumbing can run 2v2, but there is no paired 2v2
|
||||
evaluation or team-credit reward yet; spending 120M steps without those gates
|
||||
would make a pass meaningless.
|
||||
|
||||
`training/generation5.py` implements Stages 4–6 separately from the completed
|
||||
generation-4 orchestrator and state. It always begins Stage 4 from
|
||||
`checkpoints/20260806-1939-curric-s3-gauntlet/final.zip`, then resumes each
|
||||
later stage from its passing predecessor. `generation5.sh` runs it detached,
|
||||
and retries/blocks use the same restart-safe pattern as the earlier
|
||||
curriculum:
|
||||
|
||||
```bash
|
||||
cd training
|
||||
.venv/bin/python generation5.py --dry-run # print and validate the next command only
|
||||
./generation5.sh # run/resume in tmux
|
||||
tmux attach -t cosmic-generation5
|
||||
cat generation5_state.json
|
||||
```
|
||||
|
||||
Stage 4 removes the generic speed bonus, halves the old orientation-agnostic
|
||||
closing reward, and adds a nose-led planar approach reward plus a tilt cost
|
||||
that fades to zero by 3m altitude. Its scoring gates deliberately run before
|
||||
Stage 5: becoming upright is not progress if the resulting policy stops
|
||||
finishing goals. Stage 5 adds moving high-ball intercept
|
||||
starts aimed toward real goals rather than a standalone air-touch reward.
|
||||
Stage 6's `league` opponent mode samples a historical exported policy at each
|
||||
episode reset. Each later stage preserves the preceding shaping and adds one
|
||||
new difficulty.
|
||||
|
||||
The physical-side gate is separate from the model-vs-model score. A paired
|
||||
side swap can make an identical policy appear perfectly balanced overall even
|
||||
when the Player 2 ship never functions. This caught the canonical action-frame
|
||||
bug exposed by Stage 3's pitch/roll use: team 1 observations are rotated 180°
|
||||
about Y, but rotation commands feed world-space torque, so team 1 pitch and
|
||||
roll must be rotated back (X/Z signs inverted). Thrust remains unchanged
|
||||
because it is applied through the ship's local basis.
|
||||
|
||||
## Self-play notes
|
||||
|
||||
By default both ships share the live policy (mirrored, team-relative
|
||||
|
||||
Reference in New Issue
Block a user