jcreek/CosmicClash
1
0
Fork 0
mirror of https://github.com/jcreek/CosmicClash.git synced 2026-09-10 16:04:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
9bdeb73fa7e39ce01e04091d5d4f60d3defed5fb
CosmicClash/training/eval_history.json
T
Josh Creek 1811e9333e feat(training): curriculum generation 4 — MultiDiscrete action space redesign 
Three curriculum generations (2026-07-21 through 2026-08-04) all tried
gating *when* the policy could use vertical thrust/pitch-roll on top of a
continuous Gaussian action space, and all three failed the same way: PPO's
action-distribution std collapsed within ~10% of steps and never recovered,
landing at a 15-32% win rate vs the grounded reference regardless of
mechanism (hard mask, then a gradual ramp). Generation 3's final attempt
just landed at 24% — the worst of the three.

Root cause, verified against this project's own physics: hovering this ship
requires *holding* thrust.y ~= 0.408 continuously (mass 5.0, vertical_thrust
120, gravity 9.8). A collapsed near-zero-mean Gaussian can brush that value
but never sustain it long enough to earn the reward gradient that would
move the mean — no amount of gating *when* the axis acts fixes a problem in
*how* the policy represents a decision on it. This also independently found
and fixes a real bug: godot_rl never marks an episode timeout as a
truncation, so PPO was bootstrapping V(s)=0 on every 30s draw in every
generation to date.

- Game/scripts/ship_action_codec.gd (new): single source of truth for a
  per-axis MultiDiscrete action space (7 heads, nvec [5,5,5,5,5,5,2]) shared
  by training and in-game inference, replacing the continuous Gaussian.
  thrust_y's bins are deliberately asymmetric so a random policy drifts
  through the volume instead of floor-pinning. Legacy continuous decode
  (ai_ship_controller.gd's old logic) preserved verbatim so every
  pre-generation-4 export (e.g. Game/bots/promoted/easy.json) keeps working
  unchanged via an optional "action_space" JSON field.
- ship_observations.gd: append own contact state (SIZE 31 -> 35, append-only)
  so the value function can see what wall_contact_penalty fires on.
- ship_ai_controller.gd: action space/decode via the codec; drop the
  vertical_ramp/pitch_roll_ramp mechanism entirely; tilt_penalty default
  lowered 4x (aerial approaches require pitching); flight telemetry
  (airborne_fraction, mean_altitude, air_touch_fraction, vertical_thrust_mean)
  and truncation-snapshot fields on get_info().
- training_mode.gd: new air_drill_chance state-setter branch (ball spawned
  high, ships low, kept clear of walls) so aerial practice is forced by the
  environment instead of relying on reward-driven exploration alone; snapshot
  terminal observations before a timeout reset for the truncation fix.
- cosmic_env.py: remap ShipAIController's truncated/terminal_obs info into
  SB3's TimeLimit.truncated/terminal_observation keys.
- train.py: --reset-logits (+ --reset-logits-heads) replaces the
  now-meaningless --reset-std; new EntropyFloorCallback (a persistent
  per-rollout ent_coef controller replacing the one-shot std-reset shock)
  and per-head entropy logging; FlightTelemetryCallback; --air-drill-chance/
  --tilt-penalty flags; optional AbortIfCallback kill-criterion.
- export_policy.py: writes the action_space block for MultiDiscrete models;
  index-level parity check (argmax per head) instead of comparing floats.
- curriculum.py: full rewrite — 3 stages (bootstrap/selfplay/gauntlet), no
  grounded stage, full action space live from step 1; deletes generation
  1-3's checkpoint-lineage machinery (nothing to resume from); final report
  evaluates against both promoted/easy.json and the new
  promoted/reference-grounded.json (a copy of curric-s5-aggression, the
  strongest grounded-era artifact, kept as a fixed yardstick).
- run_training.sh/.gitignore: commit only final.zip, not the ~2400
  intermediate checkpoint files a single stage was writing (~500MB ->
  ~0.2MB per run); requirements.txt pinned (behaviour here now depends on
  specific library internals, not just public APIs).
- test_action_space.py (new): offline rung-0 check catching a head-order
  mismatch before it silently corrupts 24h of training.

Validated: GDScript compiles clean (Godot --headless --import + script
validation), free_play.tscn and training.tscn both boot headless without
errors, offline action-space assertions pass. Not yet run: the actual
smoke-training/A-B validation ladder steps in TRAINING.md's "Generation 4"
section, before committing to the full ~32h curriculum.

See TRAINING.md's "Generation 4" section for the full design writeup.
2026-08-04 23:27:57 +01:00

193 lines
6.3 KiB
JSON
Raw Blame History

[
{
"timestamp": "2026-07-18T18:27:59+00:00",
"model_a": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/rookie.json",
"model_b": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/rookie.json",
"episodes": 6,
"wins_a": 1,
"wins_b": 0,
"draws": 5,
"win_rate_a": 0.167
},
{
"timestamp": "2026-07-19T12:11:30+00:00",
"model_a": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/run01.json",
"model_b": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/run02.json",
"episodes": 40,
"wins_a": 4,
"wins_b": 1,
"draws": 35,
"win_rate_a": 0.1
},
{
"timestamp": "2026-07-20T18:47:54+00:00",
"model_a": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/run09.json",
"model_b": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/run07.json",
"episodes": 40,
"wins_a": 6,
"wins_b": 9,
"draws": 25,
"win_rate_a": 0.15
},
{
"timestamp": "2026-07-21T15:52:20+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s1-score.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/rookie.json",
"episodes": 100,
"wins_a": 22,
"wins_b": 16,
"draws": 62,
"win_rate_a": 0.22
},
{
"timestamp": "2026-07-21T18:19:44+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s2-defend.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s1-score.json",
"episodes": 100,
"wins_a": 23,
"wins_b": 16,
"draws": 61,
"win_rate_a": 0.23
},
{
"timestamp": "2026-07-21T20:46:34+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s3-no_draws.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s2-defend.json",
"episodes": 100,
"wins_a": 44,
"wins_b": 19,
"draws": 37,
"win_rate_a": 0.44
},
{
"timestamp": "2026-07-22T11:36:42+00:00",
"model_a": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/curric-s3-no_draws.json",
"model_b": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/curric-s2-defend.json",
"episodes": 100,
"wins_a": 26,
"wins_b": 60,
"draws": 14,
"win_rate_a": 0.26
},
{
"timestamp": "2026-07-22T11:38:57+00:00",
"model_a": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/curric-s4-mechanics.json",
"model_b": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/curric-s3-no_draws.json",
"episodes": 100,
"wins_a": 26,
"wins_b": 33,
"draws": 41,
"win_rate_a": 0.26
},
{
"timestamp": "2026-07-22T11:40:07+00:00",
"model_a": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/curric-s4-mechanics.json",
"model_b": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/curric-s2-defend.json",
"episodes": 100,
"wins_a": 24,
"wins_b": 57,
"draws": 19,
"win_rate_a": 0.24
},
{
"timestamp": "2026-07-22T19:33:07+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s5-aggression.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s2-defend.json",
"episodes": 100,
"wins_a": 41,
"wins_b": 47,
"draws": 12,
"win_rate_a": 0.41
},
{
"timestamp": "2026-07-24T01:25:37+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s6-unmask.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s5-aggression.json",
"episodes": 100,
"wins_a": 25,
"wins_b": 50,
"draws": 25,
"win_rate_a": 0.25
},
{
"timestamp": "2026-07-25T06:41:43+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s6-unmask-retry1.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s5-aggression.json",
"episodes": 100,
"wins_a": 20,
"wins_b": 61,
"draws": 19,
"win_rate_a": 0.2
},
{
"timestamp": "2026-07-26T12:08:35+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s6-unmask-retry2.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s5-aggression.json",
"episodes": 100,
"wins_a": 15,
"wins_b": 68,
"draws": 17,
"win_rate_a": 0.15
},
{
"timestamp": "2026-07-27T23:30:59+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/20260726-1904-curric-s1-unmask.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s5-aggression.json",
"episodes": 100,
"wins_a": 32,
"wins_b": 50,
"draws": 18,
"win_rate_a": 0.32
},
{
"timestamp": "2026-07-30T11:24:45+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/20260729-0837-curric-s1-unmask-retry1.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s5-aggression.json",
"episodes": 100,
"wins_a": 28,
"wins_b": 57,
"draws": 15,
"win_rate_a": 0.28
},
{
"timestamp": "2026-07-31T15:07:16+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/20260730-1224-curric-s1-unmask-retry2.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s5-aggression.json",
"episodes": 100,
"wins_a": 31,
"wins_b": 56,
"draws": 13,
"win_rate_a": 0.31
},
{
"timestamp": "2026-08-02T13:58:25+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/20260801-1131-curric-s4-unmask.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s5-aggression.json",
"episodes": 100,
"wins_a": 29,
"wins_b": 55,
"draws": 16,
"win_rate_a": 0.29
},
{
"timestamp": "2026-08-03T17:29:13+00:00",
"model_a": "/home/jcreek/ai-training/CosmicClash/Game/bots/20260802-1458-curric-s4-unmask-retry1.json",
"model_b": "/home/jcreek/ai-training/CosmicClash/Game/bots/curric-s5-aggression.json",
"episodes": 100,
"wins_a": 30,
"wins_b": 56,
"draws": 14,
"win_rate_a": 0.3
},
{
"timestamp": "2026-08-04T21:26:41+00:00",
"model_a": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/20260803-1829-curric-s4-unmask-retry2.json",
"model_b": "/Users/jcreek/Documents/repos/GitHub/CosmicClash/Game/bots/curric-s5-aggression.json",
"episodes": 100,
"wins_a": 24,
"wins_b": 56,
"draws": 20,
"win_rate_a": 0.24
}
]
Reference in New Issue View Git Blame Copy Permalink