mirror of
https://github.com/jcreek/CosmicClash.git
synced 2026-09-11 17:23:44 +00:00
Josh Creek
3fd1c00895
Generation 2's single "unmask" stage (flip vertical/pitch-roll locomotion from grounded-only to full 3D in one step) failed 3 independent 240M-step attempts, landing at a stable 32% / 28% / 31% win rate vs curric-s5-aggression each time -- not noise, and not fixable by more training time (attempts 2-3 each continued the same checkpoint lineage for another full 240M steps with zero improvement). Every attempt shows train/std collapsing from ~0.30 to ~0.13-0.15 within the first ~10% of steps and never recovering: the policy locks the newly-opened axes back down before ever meaningfully exploring them. Replaces the boolean allow_vertical/allow_pitch_roll mask on ShipAIController with float vertical_ramp/pitch_roll_ramp multipliers (0.0-1.0), scaling axis effect in set_action() instead of gating it outright -- the action space never changes shape, so checkpoints stay resumable across ramp values. The single unmask stage in curriculum.py becomes 4: three ungated warmup stages (25%/50%/75% authority, airborne_penalty ramping in step) that train, checkpoint, and always advance with no eval gate, then the measured stage at full authority -- same reference, opponent mode, and 240M budget as the 3 failed attempts, for a direct comparison. Adds a "gated" flag/branch to main()'s loop for the ungated stages. This is generation 3 of the curriculum; generation 2's state is archived to curriculum_state_gen2.json (mirroring the earlier gen1 -> gen2 archival) and curriculum_state.json resets fresh, since its stage 0 no longer means what it used to. See TRAINING.md's "Generation 3" section for the full postmortem, stage table, and the open question about whether scaling action effect in Godot (which PPO's own entropy/exploration math never sees) actually addresses the collapse.
27 lines
2.8 KiB
Markdown
27 lines
2.8 KiB
Markdown
# TODO
|
|
|
|
Deferred work, in rough priority order. The current architecture (ShipAction/ShipController seam, Arena/GameMode split, code-driven spawning, group-tagged ball/goals) was chosen specifically so these bolt on without rework.
|
|
|
|
## AI opponent (reinforcement learning)
|
|
|
|
The training pipeline is built — see `TRAINING.md` (self-play PPO via the vendored godot_rl_agents bridge, JSON policy export, in-game GDScript inference, eval ladder). Remaining:
|
|
|
|
- [x] Promote a first tier: `curric-s6-unmask` copied into `Game/bots/promoted/easy.json` as the shipped "easy" bot (see TRAINING.md's "Promoted bots" section) — `match.tscn`/`spectate.tscn` now default there instead of `run05.json`.
|
|
- [ ] Long training runs on the Linux/3090 box to produce actually-good bots; promote further checkpoints into `Game/bots/promoted/` as `medium`/`hard` tiers once they clear `easy.json` in `evaluate.py`.
|
|
- [x] Staged curriculum (score → defend → avoid draws → full mechanics) via `train.py`'s `--opponent-mode`/`--draw-penalty`/`--attack-goal-bias`/`--vertical-ramp`/`--pitch-roll-ramp` flags — see TRAINING.md's "Curriculum training" section. `--opponent-mode=frozen` is a single-fixed-model slice of the league idea below, not the full sampled pool.
|
|
- [ ] Frozen-opponent league: train the live policy against a *pool* of past exported checkpoints, sampled per-episode (today's `--opponent-mode=frozen` only supports one fixed model per run) to prevent self-play strategy collapse on long runs.
|
|
- [ ] Richer state setter / curriculum: aerial states, wall plays, rebound scenarios as skill grows (beyond the score/defend/draw staging already in place).
|
|
- [ ] Main-menu difficulty picker (Match already takes `bot_model_path`/`bot_reaction_ticks`/`bot_action_noise` exports).
|
|
|
|
## Multiplayer (long term)
|
|
|
|
- [ ] `RemoteShipController extends ShipController` — feeds replicated `ShipAction`s from a network peer into the local ship simulation.
|
|
- [ ] Networked `GameMode` subclass: per-peer ship spawning (MultiplayerSpawner or custom), authoritative server for ball/score.
|
|
- [ ] C# backend / online servers per README roadmap (not started).
|
|
- [ ] Possible v0.2 split-screen: spawn one `ship_camera_rig` + viewport per local player (camera is already outside the ship scene to allow this).
|
|
|
|
## General
|
|
|
|
- [ ] No autoloads yet by design — add a singleton only when cross-scene state is actually needed (e.g. passing match settings/results between menu, match, and results screens).
|
|
- [ ] More arenas: `arena_01.tscn` is the template — an arena is a setting (sky, lighting, decoration) + two team-tagged goals on the standard goal lines + spawn markers + an instance of the shared `objects/arena_boundary.tscn`, with no rules or state. All arenas share the standard play volume defined by `ArenaBoundary`'s constants.
|