feat(*): add staged curriculum training with an automated stage-by-stage orchestrator

This commit is contained in:
Josh Creek
2026-07-21 12:38:51 +01:00
parent 1d539cc8c7
commit 8e3fafcc8b
7 changed files with 581 additions and 18 deletions
+3 -2
View File
@@ -7,8 +7,9 @@ Deferred work, in rough priority order. The current architecture (ShipAction/Shi
The training pipeline is built — see `TRAINING.md` (self-play PPO via the vendored godot_rl_agents bridge, JSON policy export, in-game GDScript inference, eval ladder). Remaining:
- [ ] Long training runs on the Linux/3090 box to produce actually-good bots; promote checkpoints into `Game/bots/` as `easy`/`medium`/`hard` tiers.
- [ ] Frozen-opponent league: train the live policy against a pool of past exported checkpoints (via `AIShipController` on the opponent ship in TrainingMode) to prevent self-play strategy collapse on long runs.
- [ ] Richer state setter / curriculum: aerial states, wall plays, rebound scenarios as skill grows.
- [x] Staged curriculum (score → defend → avoid draws → full mechanics) via `train.py`'s `--opponent-mode`/`--draw-penalty`/`--attack-goal-bias`/`--allow-vertical`/`--allow-pitch-roll` flags — see TRAINING.md's "Curriculum training" section. `--opponent-mode=frozen` is a single-fixed-model slice of the league idea below, not the full sampled pool.
- [ ] Frozen-opponent league: train the live policy against a *pool* of past exported checkpoints, sampled per-episode (today's `--opponent-mode=frozen` only supports one fixed model per run) to prevent self-play strategy collapse on long runs.
- [ ] Richer state setter / curriculum: aerial states, wall plays, rebound scenarios as skill grows (beyond the score/defend/draw staging already in place).
- [ ] Main-menu difficulty picker (Match already takes `bot_model_path`/`bot_reaction_ticks`/`bot_action_noise` exports).
- [ ] Optional: exported headless Linux build for faster parallel training instances (train.py currently runs the project from source, which is fine but re-parses scripts per instance).