Extends ShipObservations beyond the old self+1-opponent layout to padded
teammate/opponent arrays (MAX_TEAMMATES=4, MAX_OPPONENTS=5, SIZE=83),
zero-filling slots past the real roster size the same way the old single-
opponent slot was zero-filled when absent.
Slot stability across ticks requires a persistent identity: Ship gains
spawn_index (set once by GameMode.spawn_ship, never reassigned — there's no
despawn path anywhere in this codebase, so a roster is fixed for the whole
episode/match). ai_ship_controller.gd's opponent discovery is rewritten from
"first non-self ship" to classify every other ship by team and sort by
spawn_index; training_mode.gd/ship_ai_controller.gd carry the equivalent
sorted lists through the training path so both agree on slot assignment for
the same roster.
training_mode.gd and match_mode.gd both gain a team_size export (default 1,
so every existing curriculum script and match keeps today's 1v1 behaviour
unchanged). This is plumbing only: no 2v2+ curriculum or reward design, and
no match-mode UI to pick team size, has been done yet. The two checkpoints
in Game/bots/promoted/ are fitted to the old 35-float layout and are not
migrated — expected to go stale until the next training run.
VecMonitor's info_keywords does a bare info[key] lookup on a completed
episode's terminal info dict and raises KeyError -- crashing the whole
training run -- if a key is ever absent. get_info() was only including
airborne_fraction/mean_altitude/vertical_thrust_mean when _telemetry_ticks
was nonzero and air_touch_fraction when _touches was nonzero; an episode
with zero ball touches (common, especially early in training) crashed on
the very first rollout in a smoke-test run. All four now always default to
0.0 rather than being conditionally present.
Found via TRAINING.md's Generation 4 validation ladder (rung 2, a 60k-step
smoke run) -- confirmed fixed by rerunning the same smoke run clean, then
export/evaluate parity (rungs 2-3) against Game/bots/promoted/easy.json.
Three curriculum generations (2026-07-21 through 2026-08-04) all tried
gating *when* the policy could use vertical thrust/pitch-roll on top of a
continuous Gaussian action space, and all three failed the same way: PPO's
action-distribution std collapsed within ~10% of steps and never recovered,
landing at a 15-32% win rate vs the grounded reference regardless of
mechanism (hard mask, then a gradual ramp). Generation 3's final attempt
just landed at 24% — the worst of the three.
Root cause, verified against this project's own physics: hovering this ship
requires *holding* thrust.y ~= 0.408 continuously (mass 5.0, vertical_thrust
120, gravity 9.8). A collapsed near-zero-mean Gaussian can brush that value
but never sustain it long enough to earn the reward gradient that would
move the mean — no amount of gating *when* the axis acts fixes a problem in
*how* the policy represents a decision on it. This also independently found
and fixes a real bug: godot_rl never marks an episode timeout as a
truncation, so PPO was bootstrapping V(s)=0 on every 30s draw in every
generation to date.
- Game/scripts/ship_action_codec.gd (new): single source of truth for a
per-axis MultiDiscrete action space (7 heads, nvec [5,5,5,5,5,5,2]) shared
by training and in-game inference, replacing the continuous Gaussian.
thrust_y's bins are deliberately asymmetric so a random policy drifts
through the volume instead of floor-pinning. Legacy continuous decode
(ai_ship_controller.gd's old logic) preserved verbatim so every
pre-generation-4 export (e.g. Game/bots/promoted/easy.json) keeps working
unchanged via an optional "action_space" JSON field.
- ship_observations.gd: append own contact state (SIZE 31 -> 35, append-only)
so the value function can see what wall_contact_penalty fires on.
- ship_ai_controller.gd: action space/decode via the codec; drop the
vertical_ramp/pitch_roll_ramp mechanism entirely; tilt_penalty default
lowered 4x (aerial approaches require pitching); flight telemetry
(airborne_fraction, mean_altitude, air_touch_fraction, vertical_thrust_mean)
and truncation-snapshot fields on get_info().
- training_mode.gd: new air_drill_chance state-setter branch (ball spawned
high, ships low, kept clear of walls) so aerial practice is forced by the
environment instead of relying on reward-driven exploration alone; snapshot
terminal observations before a timeout reset for the truncation fix.
- cosmic_env.py: remap ShipAIController's truncated/terminal_obs info into
SB3's TimeLimit.truncated/terminal_observation keys.
- train.py: --reset-logits (+ --reset-logits-heads) replaces the
now-meaningless --reset-std; new EntropyFloorCallback (a persistent
per-rollout ent_coef controller replacing the one-shot std-reset shock)
and per-head entropy logging; FlightTelemetryCallback; --air-drill-chance/
--tilt-penalty flags; optional AbortIfCallback kill-criterion.
- export_policy.py: writes the action_space block for MultiDiscrete models;
index-level parity check (argmax per head) instead of comparing floats.
- curriculum.py: full rewrite — 3 stages (bootstrap/selfplay/gauntlet), no
grounded stage, full action space live from step 1; deletes generation
1-3's checkpoint-lineage machinery (nothing to resume from); final report
evaluates against both promoted/easy.json and the new
promoted/reference-grounded.json (a copy of curric-s5-aggression, the
strongest grounded-era artifact, kept as a fixed yardstick).
- run_training.sh/.gitignore: commit only final.zip, not the ~2400
intermediate checkpoint files a single stage was writing (~500MB ->
~0.2MB per run); requirements.txt pinned (behaviour here now depends on
specific library internals, not just public APIs).
- test_action_space.py (new): offline rung-0 check catching a head-order
mismatch before it silently corrupts 24h of training.
Validated: GDScript compiles clean (Godot --headless --import + script
validation), free_play.tscn and training.tscn both boot headless without
errors, offline action-space assertions pass. Not yet run: the actual
smoke-training/A-B validation ladder steps in TRAINING.md's "Generation 4"
section, before committing to the full ~32h curriculum.
See TRAINING.md's "Generation 4" section for the full design writeup.
Generation 2's single "unmask" stage (flip vertical/pitch-roll locomotion
from grounded-only to full 3D in one step) failed 3 independent 240M-step
attempts, landing at a stable 32% / 28% / 31% win rate vs curric-s5-aggression
each time -- not noise, and not fixable by more training time (attempts 2-3
each continued the same checkpoint lineage for another full 240M steps with
zero improvement). Every attempt shows train/std collapsing from ~0.30 to
~0.13-0.15 within the first ~10% of steps and never recovering: the policy
locks the newly-opened axes back down before ever meaningfully exploring
them.
Replaces the boolean allow_vertical/allow_pitch_roll mask on ShipAIController
with float vertical_ramp/pitch_roll_ramp multipliers (0.0-1.0), scaling axis
effect in set_action() instead of gating it outright -- the action space
never changes shape, so checkpoints stay resumable across ramp values. The
single unmask stage in curriculum.py becomes 4: three ungated warmup stages
(25%/50%/75% authority, airborne_penalty ramping in step) that train,
checkpoint, and always advance with no eval gate, then the measured stage at
full authority -- same reference, opponent mode, and 240M budget as the 3
failed attempts, for a direct comparison. Adds a "gated" flag/branch to
main()'s loop for the ungated stages.
This is generation 3 of the curriculum; generation 2's state is archived to
curriculum_state_gen2.json (mirroring the earlier gen1 -> gen2 archival) and
curriculum_state.json resets fresh, since its stage 0 no longer means what it
used to. See TRAINING.md's "Generation 3" section for the full postmortem,
stage table, and the open question about whether scaling action effect in
Godot (which PPO's own entropy/exploration math never sees) actually
addresses the collapse.
Stage 5 (aggression) passed (41-47 vs grounded curric-s2-defend, within
the lenient gate but not yet a clear win). Rather than keep the locomotion
mask on indefinitely, stage 6 reopens full 3D controls on top of the
aggression retune and pairs it with a new dense airborne_penalty (scaled
by height above the floor) so the policy learns to prefer staying grounded
through incentives instead of a hard mask — same regime shift that
regressed stage 3, but this time with a mitigation and ~12x the training
time (~240M timesteps / ~24h vs ~20M / ~2h) to actually re-converge
instead of stalling mid-shift.
airborne_penalty follows the existing SHIP_AI_OVERRIDES pattern: default
0 (off) on ship_ai_controller.gd, exposed via train.py's new
--airborne-penalty flag, added to training_mode.gd's allow-list. Also adds
a per-stage timesteps override in curriculum.py (STAGES[n]["timesteps"])
since this is the first stage to need a different budget than the rest.
Josh Creek