feat(*): Add self-play RL training pipeline with PPO trainer, in-game GDScript policy inference, and bot opponent support in Match mode

TRAINING.md

@@ -0,0 +1,140 @@
+# Training the AI bot
+
+Cosmic Clash bots are trained with reinforcement learning (self-play PPO): two
+ships in a headless arena share one policy that learns by playing against
+itself. Training runs in Python ([Godot RL Agents](https://github.com/edbeeching/godot_rl_agents)
+bridge + Stable-Baselines3); the trained policy is exported to a small JSON
+file and runs **inside the game** in pure GDScript — shipped bots need no
+Python, no .NET, no network.
+
+## How it fits together
+
+- `Game/scenes/training.tscn` + `scripts/training_mode.gd` — headless self-play
+  environment: two RL ships, randomized episode starts, goal rewards. Contains
+  the vendored godot_rl_agents `Sync` node that talks TCP to the trainer.
+- `scripts/ship_ai_controller.gd` — training-side bridge (observations,
+  rewards, action mapping). `scripts/ship_observations.gd` is the *shared*
+  observation builder — training and in-game inference must stay identical,
+  so never fork it.
+- `training/train.py` — PPO trainer; launches N parallel headless Godot
+  instances (2 agents each).
+- `training/export_policy.py` — SB3 checkpoint → JSON policy for the game.
+- `scripts/ai_ship_controller.gd` + `scripts/policy_network.gd` — in-game
+  inference (GDScript MLP forward pass).
+- `training/evaluate.py` — pits two exported policies against each other and
+  appends to `training/eval_history.json`.
+
+## Hardware
+
+The environment is our own headless Godot sim — fully cross-platform:
+
+- **Any machine (e.g. the M4 Mac mini)**: fine for pipeline development,
+  smoke runs, and short experiments. Env stepping is CPU-bound; the policy is
+  a small MLP, so even CPU-only PPO updates are cheap.
+- **Linux + NVIDIA GPU (e.g. the RTX 3090 box)**: recommended for real
+  multi-hour/overnight runs. PyTorch CUDA works out of the box; more CPU
+  cores also mean more parallel Godot instances (`--n-parallel`).
+
+There is no hard GPU requirement (unlike Rocket League tooling) — a GPU
+mainly speeds up learning updates on long runs.
+
+## Setup
+
+Needs Python 3.10+ and a Godot 4.7 binary.
+
+```bash
+cd training
+python3.12 -m venv .venv          # macOS: brew install python@3.12
+.venv/bin/pip install -r requirements.txt
+```
+
+On Linux, download the Godot 4.7 Linux binary and point at it:
+
+```bash
+export GODOT_BIN=~/godot/Godot_v4.7.1-stable_linux.x86_64
+```
+
+(macOS default is `/Applications/Godot.app/Contents/MacOS/Godot`; override
+with `GODOT_BIN` or `--godot_bin` if yours lives elsewhere.)
+
+## Run a training session
+
+```bash
+cd training
+.venv/bin/python train.py --experiment run01 --timesteps 20000000 --n-parallel 6 --speedup 16
+```
+
+- Checkpoints land in `training/checkpoints/run01/` every `--checkpoint-every`
+  steps (default 100k), plus `final.zip` on exit (also written on Ctrl-C).
+- Resume with `--resume checkpoints/run01/final.zip`.
+- `--n-parallel` = Godot instances (2 agents each). Scale with CPU cores.
+- `--speedup` = in-engine physics speedup. Raise until CPU saturates.
+- `--wandb` mirrors logs to Weights & Biases (`pip install wandb` first).
+
+Expect the smoke-run scale (~100k steps) to only learn crude ball-chasing;
+real behaviour needs tens of millions of steps (hours on the 3090 box).
+
+### Watch progress
+
+```bash
+.venv/bin/tensorboard --logdir training/logs
+```
+
+Key curves: `rollout/ep_rew_mean` (should trend up), `rollout/ep_len_mean`
+(should trend *down* from 225 as goals end episodes early — 225 action steps
+= the 30s episode timeout).
+
+### Reward/observation tuning
+
+Reward weights are exported vars on `ShipAIController` (goal reward on
+`TrainingMode`) — tune in `training.tscn`/scripts without touching the
+trainer. If you change the *observation* layout (`ship_observations.gd`),
+old checkpoints/exports become incompatible: retrain, and bump a note in
+your experiment name.
+
+## Export a checkpoint into the game
+
+```bash
+cd training
+.venv/bin/python export_policy.py checkpoints/run01/final.zip ../Game/bots/hard.json
+```
+
+The exporter runs a parity check (JSON forward pass vs SB3 prediction) before
+writing. Models live in `Game/bots/`.
+
+## Evaluate progress between checkpoints
+
+TensorBoard shows learning, but "is the new checkpoint actually *better*?"
+needs head-to-head play:
+
+```bash
+.venv/bin/python export_policy.py checkpoints/run01/ppo_5000000_steps.zip /tmp/candidate.json
+.venv/bin/python evaluate.py ../Game/bots/hard.json /tmp/candidate.json --episodes 40
+```
+
+Golden-goal episodes (first goal wins, timeout = draw), sides swapped halfway
+for fairness, using the exact inference path that ships in-game. Every run
+appends to `training/eval_history.json` — the long-term progress record.
+Evaluate each new candidate against the previous promoted bot and a fixed
+early reference to see absolute progress over time.
+
+## Difficulty tiers
+
+A bot is `(model, reaction_ticks, action_noise)` — configured on the Match
+mode (`bot_model_path`, `bot_reaction_ticks`, `bot_action_noise` in
+`match.tscn`) or any `AIShipController`:
+
+- **Model**: the main lever. An early checkpoint *is* an easy bot — promote
+  e.g. `easy.json` / `medium.json` / `hard.json` from different stages of one
+  training run (verify the gaps with `evaluate.py`).
+- **reaction_ticks** (default 8 = training cadence): higher = slower
+  reactions, easier.
+- **action_noise**: adds execution error, easier.
+
+## Self-play notes
+
+Both ships share the live policy (mirrored, team-relative observations — see
+`ship_observations.gd`), so training is always against the current self.
+Fixed-opponent training against frozen checkpoints (league play, to avoid
+strategy collapse on long runs) is deferred — see TODO.md; the pieces
+(exported JSON bots + `AIShipController`) already exist.