feat(training): curriculum generation 4 — MultiDiscrete action space redesign

Three curriculum generations (2026-07-21 through 2026-08-04) all tried
gating *when* the policy could use vertical thrust/pitch-roll on top of a
continuous Gaussian action space, and all three failed the same way: PPO's
action-distribution std collapsed within ~10% of steps and never recovered,
landing at a 15-32% win rate vs the grounded reference regardless of
mechanism (hard mask, then a gradual ramp). Generation 3's final attempt
just landed at 24% — the worst of the three.

Root cause, verified against this project's own physics: hovering this ship
requires *holding* thrust.y ~= 0.408 continuously (mass 5.0, vertical_thrust
120, gravity 9.8). A collapsed near-zero-mean Gaussian can brush that value
but never sustain it long enough to earn the reward gradient that would
move the mean — no amount of gating *when* the axis acts fixes a problem in
*how* the policy represents a decision on it. This also independently found
and fixes a real bug: godot_rl never marks an episode timeout as a
truncation, so PPO was bootstrapping V(s)=0 on every 30s draw in every
generation to date.

- Game/scripts/ship_action_codec.gd (new): single source of truth for a
  per-axis MultiDiscrete action space (7 heads, nvec [5,5,5,5,5,5,2]) shared
  by training and in-game inference, replacing the continuous Gaussian.
  thrust_y's bins are deliberately asymmetric so a random policy drifts
  through the volume instead of floor-pinning. Legacy continuous decode
  (ai_ship_controller.gd's old logic) preserved verbatim so every
  pre-generation-4 export (e.g. Game/bots/promoted/easy.json) keeps working
  unchanged via an optional "action_space" JSON field.
- ship_observations.gd: append own contact state (SIZE 31 -> 35, append-only)
  so the value function can see what wall_contact_penalty fires on.
- ship_ai_controller.gd: action space/decode via the codec; drop the
  vertical_ramp/pitch_roll_ramp mechanism entirely; tilt_penalty default
  lowered 4x (aerial approaches require pitching); flight telemetry
  (airborne_fraction, mean_altitude, air_touch_fraction, vertical_thrust_mean)
  and truncation-snapshot fields on get_info().
- training_mode.gd: new air_drill_chance state-setter branch (ball spawned
  high, ships low, kept clear of walls) so aerial practice is forced by the
  environment instead of relying on reward-driven exploration alone; snapshot
  terminal observations before a timeout reset for the truncation fix.
- cosmic_env.py: remap ShipAIController's truncated/terminal_obs info into
  SB3's TimeLimit.truncated/terminal_observation keys.
- train.py: --reset-logits (+ --reset-logits-heads) replaces the
  now-meaningless --reset-std; new EntropyFloorCallback (a persistent
  per-rollout ent_coef controller replacing the one-shot std-reset shock)
  and per-head entropy logging; FlightTelemetryCallback; --air-drill-chance/
  --tilt-penalty flags; optional AbortIfCallback kill-criterion.
- export_policy.py: writes the action_space block for MultiDiscrete models;
  index-level parity check (argmax per head) instead of comparing floats.
- curriculum.py: full rewrite — 3 stages (bootstrap/selfplay/gauntlet), no
  grounded stage, full action space live from step 1; deletes generation
  1-3's checkpoint-lineage machinery (nothing to resume from); final report
  evaluates against both promoted/easy.json and the new
  promoted/reference-grounded.json (a copy of curric-s5-aggression, the
  strongest grounded-era artifact, kept as a fixed yardstick).
- run_training.sh/.gitignore: commit only final.zip, not the ~2400
  intermediate checkpoint files a single stage was writing (~500MB ->
  ~0.2MB per run); requirements.txt pinned (behaviour here now depends on
  specific library internals, not just public APIs).
- test_action_space.py (new): offline rung-0 check catching a head-order
  mismatch before it silently corrupts 24h of training.

Validated: GDScript compiles clean (Godot --headless --import + script
validation), free_play.tscn and training.tscn both boot headless without
errors, offline action-space assertions pass. Not yet run: the actual
smoke-training/A-B validation ladder steps in TRAINING.md's "Generation 4"
section, before committing to the full ~32h curriculum.

See TRAINING.md's "Generation 4" section for the full design writeup.

training/test_action_space.py

@@ -0,0 +1,105 @@
+"""Rung 0 of TRAINING.md's validation ladder: offline, no Godot, seconds to
+run. Catches the single most likely silent killer in the generation-4
+action-space redesign — a head-order mismatch between the Python trainer and
+Game/scripts/ship_action_codec.gd's HEADS. If they disagree, training still
+runs happily for 24h+ (pitch commands driving strafe thrusters, say) and
+only surfaces as inexplicably-bad behaviour, not an error. This can't
+directly parse the GDScript file, but it locks the two real, checkable
+invariants an order mismatch would actually depend on: that gymnasium's Dict
+key-sorting produces the exact order ship_action_codec.gd's HEADS is written
+in, and that godot_rl's ActionSpaceProcessor converts that into the
+MultiDiscrete nvec the trainer expects. export_policy.py's own index-level
+parity check plus a real in-game round trip (rung 3) are what catch anything
+this can't.
+
+Usage:
+    .venv/bin/python test_action_space.py
+"""
+
+import sys
+
+import gymnasium as gym
+import numpy as np
+from godot_rl.core.utils import ActionSpaceProcessor
+
+from export_policy import ACTION_HEADS
+
+# The order Game/scripts/ship_action_codec.gd's HEADS is written in — kept
+# here as a literal, independent restatement (not derived from ACTION_HEADS)
+# so this test can actually catch export_policy.py's own list being edited
+# out of order too, not just catch nothing because both sides changed
+# together.
+EXPECTED_ORDER = ["rot_x", "rot_y", "rot_z", "thrust_x", "thrust_y", "thrust_z", "turbo"]
+
+
+def check_action_heads_match_expected_order() -> None:
+    names = [head["name"] for head in ACTION_HEADS]
+    assert names == EXPECTED_ORDER, (
+        f"export_policy.ACTION_HEADS order {names} != expected {EXPECTED_ORDER} — "
+        "this must match Game/scripts/ship_action_codec.gd's HEADS exactly"
+    )
+
+
+def check_gymnasium_sorts_to_expected_order() -> None:
+    # Build the Dict deliberately out of order (reversed) to prove it's
+    # gymnasium's sort doing the work here, not insertion order — this is
+    # exactly what godot_env.py does with the dict Godot sends over the wire
+    # (see godot_env.py's from_dict, which builds a spaces.Dict from
+    # ShipAIController.get_action_space()'s Dictionary).
+    sizes = {head["name"]: len(head["bins"]) for head in ACTION_HEADS}
+    reversed_dict = gym.spaces.Dict({name: gym.spaces.Discrete(sizes[name]) for name in reversed(EXPECTED_ORDER)})
+    sorted_names = list(reversed_dict.keys())
+    assert sorted_names == EXPECTED_ORDER, (
+        f"gymnasium.spaces.Dict sorted {sorted_names}, expected {EXPECTED_ORDER} — "
+        "if this changed, every export from this generation onward would be silently "
+        "mis-ordered relative to ship_action_codec.gd"
+    )
+
+
+def check_action_space_processor_produces_expected_multi_discrete() -> None:
+    sizes = [len(head["bins"]) for head in ACTION_HEADS]
+    tuple_space = gym.spaces.Tuple([gym.spaces.Discrete(n) for n in sizes])
+    processor = ActionSpaceProcessor(tuple_space, convert=True)
+    assert isinstance(processor.action_space, gym.spaces.MultiDiscrete), (
+        f"expected MultiDiscrete, got {type(processor.action_space)} — the all-discrete branch "
+        "in godot_rl's ActionSpaceProcessor may have changed (see requirements.txt's pin note)"
+    )
+    assert list(processor.action_space.nvec) == sizes, (
+        f"MultiDiscrete nvec {list(processor.action_space.nvec)} != expected {sizes}"
+    )
+
+
+def check_round_trip_preserves_per_head_values() -> None:
+    # An integer action per env, one column per head in EXPECTED_ORDER —
+    # confirms to_original_dist splits a MultiDiscrete action back into the
+    # same per-head order it was built from (this is what set_action() on
+    # the Godot side receives, keyed by head name).
+    sizes = [len(head["bins"]) for head in ACTION_HEADS]
+    tuple_space = gym.spaces.Tuple([gym.spaces.Discrete(n) for n in sizes])
+    processor = ActionSpaceProcessor(tuple_space, convert=True)
+
+    n_envs = 3
+    rng = np.random.default_rng(0)
+    action = np.stack([rng.integers(0, n, size=n_envs) for n in sizes], axis=1).astype(np.int64)
+    original = processor.to_original_dist(action)
+    assert len(original) == len(sizes)
+    for head_index, expected_column in enumerate(action.T):
+        np.testing.assert_array_equal(np.asarray(original[head_index]), expected_column)
+
+
+def main() -> int:
+    checks = [
+        check_action_heads_match_expected_order,
+        check_gymnasium_sorts_to_expected_order,
+        check_action_space_processor_produces_expected_multi_discrete,
+        check_round_trip_preserves_per_head_values,
+    ]
+    for check in checks:
+        check()
+        print(f"PASS: {check.__name__}")
+    print(f"\nAll {len(checks)} action-space checks passed.")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())