All checkpoints/logs/exported policies here are a continuous-Gaussian,
31-input action/observation shape that generation 4's MultiDiscrete
redesign is structurally incompatible with -- nothing to resume from (see
the prior commit and TRAINING.md's "Generation 4" section). Game/bots/promoted/
(easy.json, and the new reference-grounded.json copied from
curric-s5-aggression before this) is untouched -- both remain valid,
playable evaluation opponents forever via PolicyNetwork's format-versioned
JSON despite their own checkpoints/generation being gone.
- training/checkpoints/*, training/logs/* removed (~3.5GB of working tree,
all generation 1-3 experiment runs).
- Game/bots/*.json flat dump removed (superseded exports; main_menu.gd's
Spectate dropdown will just be empty until the first generation-4 export).
- curriculum_state.json -> curriculum_state_gen3.json, archived alongside
the existing _gen1/_gen2 logs (all three are referenced as postmortem
evidence in TRAINING.md/curriculum.py). A fresh curriculum_state.json
will be created on the next curriculum.py run (load_state() already
handles a missing file).
training/eval_history.json is deliberately NOT reset -- it's the one
continuous cross-generation progress record.
NOT YET PUSHED: this needs the remote Linux training box quiesced first
(kill any active tmux session, confirm it's synced to origin) so its own
run_training.sh doesn't race a still-running job's final commit against
this deletion.
Three curriculum generations (2026-07-21 through 2026-08-04) all tried
gating *when* the policy could use vertical thrust/pitch-roll on top of a
continuous Gaussian action space, and all three failed the same way: PPO's
action-distribution std collapsed within ~10% of steps and never recovered,
landing at a 15-32% win rate vs the grounded reference regardless of
mechanism (hard mask, then a gradual ramp). Generation 3's final attempt
just landed at 24% — the worst of the three.
Root cause, verified against this project's own physics: hovering this ship
requires *holding* thrust.y ~= 0.408 continuously (mass 5.0, vertical_thrust
120, gravity 9.8). A collapsed near-zero-mean Gaussian can brush that value
but never sustain it long enough to earn the reward gradient that would
move the mean — no amount of gating *when* the axis acts fixes a problem in
*how* the policy represents a decision on it. This also independently found
and fixes a real bug: godot_rl never marks an episode timeout as a
truncation, so PPO was bootstrapping V(s)=0 on every 30s draw in every
generation to date.
- Game/scripts/ship_action_codec.gd (new): single source of truth for a
per-axis MultiDiscrete action space (7 heads, nvec [5,5,5,5,5,5,2]) shared
by training and in-game inference, replacing the continuous Gaussian.
thrust_y's bins are deliberately asymmetric so a random policy drifts
through the volume instead of floor-pinning. Legacy continuous decode
(ai_ship_controller.gd's old logic) preserved verbatim so every
pre-generation-4 export (e.g. Game/bots/promoted/easy.json) keeps working
unchanged via an optional "action_space" JSON field.
- ship_observations.gd: append own contact state (SIZE 31 -> 35, append-only)
so the value function can see what wall_contact_penalty fires on.
- ship_ai_controller.gd: action space/decode via the codec; drop the
vertical_ramp/pitch_roll_ramp mechanism entirely; tilt_penalty default
lowered 4x (aerial approaches require pitching); flight telemetry
(airborne_fraction, mean_altitude, air_touch_fraction, vertical_thrust_mean)
and truncation-snapshot fields on get_info().
- training_mode.gd: new air_drill_chance state-setter branch (ball spawned
high, ships low, kept clear of walls) so aerial practice is forced by the
environment instead of relying on reward-driven exploration alone; snapshot
terminal observations before a timeout reset for the truncation fix.
- cosmic_env.py: remap ShipAIController's truncated/terminal_obs info into
SB3's TimeLimit.truncated/terminal_observation keys.
- train.py: --reset-logits (+ --reset-logits-heads) replaces the
now-meaningless --reset-std; new EntropyFloorCallback (a persistent
per-rollout ent_coef controller replacing the one-shot std-reset shock)
and per-head entropy logging; FlightTelemetryCallback; --air-drill-chance/
--tilt-penalty flags; optional AbortIfCallback kill-criterion.
- export_policy.py: writes the action_space block for MultiDiscrete models;
index-level parity check (argmax per head) instead of comparing floats.
- curriculum.py: full rewrite — 3 stages (bootstrap/selfplay/gauntlet), no
grounded stage, full action space live from step 1; deletes generation
1-3's checkpoint-lineage machinery (nothing to resume from); final report
evaluates against both promoted/easy.json and the new
promoted/reference-grounded.json (a copy of curric-s5-aggression, the
strongest grounded-era artifact, kept as a fixed yardstick).
- run_training.sh/.gitignore: commit only final.zip, not the ~2400
intermediate checkpoint files a single stage was writing (~500MB ->
~0.2MB per run); requirements.txt pinned (behaviour here now depends on
specific library internals, not just public APIs).
- test_action_space.py (new): offline rung-0 check catching a head-order
mismatch before it silently corrupts 24h of training.
Validated: GDScript compiles clean (Godot --headless --import + script
validation), free_play.tscn and training.tscn both boot headless without
errors, offline action-space assertions pass. Not yet run: the actual
smoke-training/A-B validation ladder steps in TRAINING.md's "Generation 4"
section, before committing to the full ~32h curriculum.
See TRAINING.md's "Generation 4" section for the full design writeup.
Generation 2's first two real stage-1 attempts both independently restarted
from curric-s5-aggression (reset_retry_checkpoint) with identical flags and
landed at 32% and 27% win rate vs the reference -- a real regression either
way, but too much spread between "identical" runs for repeat fresh restarts
to be a controlled test of anything. The first attempt's own trajectory
(ep_rew_mean climbing from -10.86 toward ~0 by the 240M-step cutoff,
briefly touching positive) looked closer to convergence than the second's,
so retries now continue that attempt's own checkpoint for another full
timesteps budget instead of resetting to foundation again.
Drops retry1 and retry2 (checkpoints, logs, exported bots, eval_history
entries) -- retry2 never trained meaningfully before crashing on the
GoalRateCallback bug just fixed, and retry1 was the inferior of the two
real samples. curriculum_state.json rewinds to attempt 1, in_progress, so
the next run resumes 20260726-1904-curric-s1-unmask/final.zip directly.
CosmicClash Training Bot