jcreek/CosmicClash
1
0
Fork 0
mirror of https://github.com/jcreek/CosmicClash.git synced 2026-09-13 17:52:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(*): Add self-play RL training pipeline with PPO trainer, in-game GDScript policy inference, and bot opponent support in Match mode

Browse Source
This commit is contained in:
Josh Creek
2026-07-18 19:32:51 +01:00
parent 328831df1f
commit 85f96eb15e
81 changed files with 3934 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+10
View File
@@ -47,3 +47,13 @@ The structure was deliberately chosen so an RL-trained AI opponent and, later, m
- **HUD / telemetry pattern**: `Ship` emits flight data via signals only when values change past thresholds (`_last_*` fields, `*_THRESHOLD` constants). `HUDController` (`scripts/HUDController.gd` on `scenes/HUD.tscn`, instanced by each mode's scene) discovers the ship, camera rig, and game mode via groups (`"ship"`, `"ship_camera"`, `"game"`), connects to signals, and only updates labels — no polling. Follow this discovery-by-group + signal-push pattern for new instruments or cross-node communication, not hardcoded `get_node` paths or per-frame polling.
- **Input actions** are defined in `Game/project.godot` under `[input]` (`move_forward`, `turn_left`, `turbo`, `reset_ball`, etc.) and read only by `PlayerShipController` (plus mode-level `_unhandled_input` for `reset_ball`/`ui_cancel`) — add new controls there rather than hardcoding key checks.
- Physics engine is Jolt (`Game/project.godot`, `[physics] 3d/physics_engine="Jolt Physics"`).
## Reinforcement learning / AI bots
See `TRAINING.md` for the full workflow (training, exporting, evaluating, difficulty tiers). Architecture summary:
- `scenes/training.tscn` (`scripts/training_mode.gd`, extends `GameMode`) is the headless self-play environment: two ships driven by `RLShipController`s, with `ShipAIController` (extends the vendored plugin's `AIController3D`) as the only class touching godot_rl types. The plugin is vendored (not a submodule) at `Game/addons/godot_rl_agents` — see its `VENDORED.md`; its C#/ONNX files are unused.
- `scripts/ship_observations.gd` is the shared observation builder used by both training and in-game inference — never fork or diverge these two paths. Team 1's observations are mirrored (180° about Y) so one policy plays both sides.
- In-game bots: `scripts/ai_ship_controller.gd` (a `ShipController`) runs the exported policy JSON via `scripts/policy_network.gd` (pure-GDScript MLP) — no .NET/ONNX/Python at runtime. Models live in `Game/bots/`; Match mode's `bot_model_path`/`bot_reaction_ticks`/`bot_action_noise` exports configure the opponent.
- Python side lives in `training/` (venv, not committed): `train.py` (SB3 PPO, launches parallel headless Godot instances from source), `export_policy.py` (checkpoint → JSON with parity check), `evaluate.py` (head-to-head eval, appends `training/eval_history.json`).
- The flattened action space is Box(7): thrust xyz, rotation xyz, turbo (>0 = on) — this is `ShipAction` verbatim; change either only deliberately and together.