mirror of
https://github.com/jcreek/CosmicClash.git
synced 2026-09-13 04:12:03 +00:00
feat(*): Add self-play RL training pipeline with PPO trainer, in-game GDScript policy inference, and bot opponent support in Match mode
This commit is contained in:
@@ -4,11 +4,13 @@ Deferred work, in rough priority order. The current architecture (ShipAction/Shi
|
||||
|
||||
## AI opponent (reinforcement learning)
|
||||
|
||||
- [ ] `AIShipController extends ShipController` — produces a `ShipAction` per physics tick from observations instead of keyboard input.
|
||||
- [ ] Observation builder: self ship state (position, orientation, velocities) + ball state (group `"ball"`) + goal positions/teams (group `"goal"`), normalized for the policy.
|
||||
- [ ] Reward shaping: goals scored/conceded, ball touches, ball-toward-opponent-goal velocity, etc.
|
||||
- [ ] Headless training scene: a `GameMode` subclass with no HUD/camera, run via `godot --headless`, stepping the sim for training (consider godot-rl-agents or a custom socket bridge).
|
||||
- [ ] Swap the inert placeholder opponent in Match mode for the trained `AIShipController`.
|
||||
The training pipeline is built — see `TRAINING.md` (self-play PPO via the vendored godot_rl_agents bridge, JSON policy export, in-game GDScript inference, eval ladder). Remaining:
|
||||
|
||||
- [ ] Long training runs on the Linux/3090 box to produce actually-good bots; promote checkpoints into `Game/bots/` as `easy`/`medium`/`hard` tiers.
|
||||
- [ ] Frozen-opponent league: train the live policy against a pool of past exported checkpoints (via `AIShipController` on the opponent ship in TrainingMode) to prevent self-play strategy collapse on long runs.
|
||||
- [ ] Richer state setter / curriculum: aerial states, wall plays, rebound scenarios as skill grows.
|
||||
- [ ] Main-menu difficulty picker (Match already takes `bot_model_path`/`bot_reaction_ticks`/`bot_action_noise` exports).
|
||||
- [ ] Optional: exported headless Linux build for faster parallel training instances (train.py currently runs the project from source, which is fine but re-parses scripts per instance).
|
||||
|
||||
## Match mode polish
|
||||
|
||||
@@ -28,3 +30,4 @@ Deferred work, in rough priority order. The current architecture (ShipAction/Shi
|
||||
|
||||
- [ ] No autoloads yet by design — add a singleton only when cross-scene state is actually needed (e.g. passing match settings/results between menu, match, and results screens).
|
||||
- [ ] More arenas: `arena_01.tscn` is the template — an arena is terrain + lighting + two team-tagged goals + spawn markers, with no rules or state.
|
||||
- [ ] Enclose the arena (walls/ceiling): ships and ball can currently fly out of the play volume. TrainingMode papers over this with an out-of-bounds episode-end + penalty; a real enclosed arena fixes it for players too and the training guard can then be removed.
|
||||
|
||||
Reference in New Issue
Block a user