mirror of
https://github.com/jcreek/CosmicClash.git
synced 2026-09-16 00:02:07 +00:00
feat(training): Replace all-or-nothing unmask with a gradual ramp
Generation 2's single "unmask" stage (flip vertical/pitch-roll locomotion from grounded-only to full 3D in one step) failed 3 independent 240M-step attempts, landing at a stable 32% / 28% / 31% win rate vs curric-s5-aggression each time -- not noise, and not fixable by more training time (attempts 2-3 each continued the same checkpoint lineage for another full 240M steps with zero improvement). Every attempt shows train/std collapsing from ~0.30 to ~0.13-0.15 within the first ~10% of steps and never recovering: the policy locks the newly-opened axes back down before ever meaningfully exploring them. Replaces the boolean allow_vertical/allow_pitch_roll mask on ShipAIController with float vertical_ramp/pitch_roll_ramp multipliers (0.0-1.0), scaling axis effect in set_action() instead of gating it outright -- the action space never changes shape, so checkpoints stay resumable across ramp values. The single unmask stage in curriculum.py becomes 4: three ungated warmup stages (25%/50%/75% authority, airborne_penalty ramping in step) that train, checkpoint, and always advance with no eval gate, then the measured stage at full authority -- same reference, opponent mode, and 240M budget as the 3 failed attempts, for a direct comparison. Adds a "gated" flag/branch to main()'s loop for the ungated stages. This is generation 3 of the curriculum; generation 2's state is archived to curriculum_state_gen2.json (mirroring the earlier gen1 -> gen2 archival) and curriculum_state.json resets fresh, since its stage 0 no longer means what it used to. See TRAINING.md's "Generation 3" section for the full postmortem, stage table, and the open question about whether scaling action effect in Godot (which PPO's own entropy/exploration math never sees) actually addresses the collapse.
This commit is contained in:
@@ -8,7 +8,7 @@ The training pipeline is built — see `TRAINING.md` (self-play PPO via the vend
|
||||
|
||||
- [x] Promote a first tier: `curric-s6-unmask` copied into `Game/bots/promoted/easy.json` as the shipped "easy" bot (see TRAINING.md's "Promoted bots" section) — `match.tscn`/`spectate.tscn` now default there instead of `run05.json`.
|
||||
- [ ] Long training runs on the Linux/3090 box to produce actually-good bots; promote further checkpoints into `Game/bots/promoted/` as `medium`/`hard` tiers once they clear `easy.json` in `evaluate.py`.
|
||||
- [x] Staged curriculum (score → defend → avoid draws → full mechanics) via `train.py`'s `--opponent-mode`/`--draw-penalty`/`--attack-goal-bias`/`--allow-vertical`/`--allow-pitch-roll` flags — see TRAINING.md's "Curriculum training" section. `--opponent-mode=frozen` is a single-fixed-model slice of the league idea below, not the full sampled pool.
|
||||
- [x] Staged curriculum (score → defend → avoid draws → full mechanics) via `train.py`'s `--opponent-mode`/`--draw-penalty`/`--attack-goal-bias`/`--vertical-ramp`/`--pitch-roll-ramp` flags — see TRAINING.md's "Curriculum training" section. `--opponent-mode=frozen` is a single-fixed-model slice of the league idea below, not the full sampled pool.
|
||||
- [ ] Frozen-opponent league: train the live policy against a *pool* of past exported checkpoints, sampled per-episode (today's `--opponent-mode=frozen` only supports one fixed model per run) to prevent self-play strategy collapse on long runs.
|
||||
- [ ] Richer state setter / curriculum: aerial states, wall plays, rebound scenarios as skill grows (beyond the score/defend/draw staging already in place).
|
||||
- [ ] Main-menu difficulty picker (Match already takes `bot_model_path`/`bot_reaction_ticks`/`bot_action_noise` exports).
|
||||
|
||||
Reference in New Issue
Block a user