feat(training): curriculum generation 4 — MultiDiscrete action space redesign

Three curriculum generations (2026-07-21 through 2026-08-04) all tried
gating *when* the policy could use vertical thrust/pitch-roll on top of a
continuous Gaussian action space, and all three failed the same way: PPO's
action-distribution std collapsed within ~10% of steps and never recovered,
landing at a 15-32% win rate vs the grounded reference regardless of
mechanism (hard mask, then a gradual ramp). Generation 3's final attempt
just landed at 24% — the worst of the three.

Root cause, verified against this project's own physics: hovering this ship
requires *holding* thrust.y ~= 0.408 continuously (mass 5.0, vertical_thrust
120, gravity 9.8). A collapsed near-zero-mean Gaussian can brush that value
but never sustain it long enough to earn the reward gradient that would
move the mean — no amount of gating *when* the axis acts fixes a problem in
*how* the policy represents a decision on it. This also independently found
and fixes a real bug: godot_rl never marks an episode timeout as a
truncation, so PPO was bootstrapping V(s)=0 on every 30s draw in every
generation to date.

- Game/scripts/ship_action_codec.gd (new): single source of truth for a
  per-axis MultiDiscrete action space (7 heads, nvec [5,5,5,5,5,5,2]) shared
  by training and in-game inference, replacing the continuous Gaussian.
  thrust_y's bins are deliberately asymmetric so a random policy drifts
  through the volume instead of floor-pinning. Legacy continuous decode
  (ai_ship_controller.gd's old logic) preserved verbatim so every
  pre-generation-4 export (e.g. Game/bots/promoted/easy.json) keeps working
  unchanged via an optional "action_space" JSON field.
- ship_observations.gd: append own contact state (SIZE 31 -> 35, append-only)
  so the value function can see what wall_contact_penalty fires on.
- ship_ai_controller.gd: action space/decode via the codec; drop the
  vertical_ramp/pitch_roll_ramp mechanism entirely; tilt_penalty default
  lowered 4x (aerial approaches require pitching); flight telemetry
  (airborne_fraction, mean_altitude, air_touch_fraction, vertical_thrust_mean)
  and truncation-snapshot fields on get_info().
- training_mode.gd: new air_drill_chance state-setter branch (ball spawned
  high, ships low, kept clear of walls) so aerial practice is forced by the
  environment instead of relying on reward-driven exploration alone; snapshot
  terminal observations before a timeout reset for the truncation fix.
- cosmic_env.py: remap ShipAIController's truncated/terminal_obs info into
  SB3's TimeLimit.truncated/terminal_observation keys.
- train.py: --reset-logits (+ --reset-logits-heads) replaces the
  now-meaningless --reset-std; new EntropyFloorCallback (a persistent
  per-rollout ent_coef controller replacing the one-shot std-reset shock)
  and per-head entropy logging; FlightTelemetryCallback; --air-drill-chance/
  --tilt-penalty flags; optional AbortIfCallback kill-criterion.
- export_policy.py: writes the action_space block for MultiDiscrete models;
  index-level parity check (argmax per head) instead of comparing floats.
- curriculum.py: full rewrite — 3 stages (bootstrap/selfplay/gauntlet), no
  grounded stage, full action space live from step 1; deletes generation
  1-3's checkpoint-lineage machinery (nothing to resume from); final report
  evaluates against both promoted/easy.json and the new
  promoted/reference-grounded.json (a copy of curric-s5-aggression, the
  strongest grounded-era artifact, kept as a fixed yardstick).
- run_training.sh/.gitignore: commit only final.zip, not the ~2400
  intermediate checkpoint files a single stage was writing (~500MB ->
  ~0.2MB per run); requirements.txt pinned (behaviour here now depends on
  specific library internals, not just public APIs).
- test_action_space.py (new): offline rung-0 check catching a head-order
  mismatch before it silently corrupts 24h of training.

Validated: GDScript compiles clean (Godot --headless --import + script
validation), free_play.tscn and training.tscn both boot headless without
errors, offline action-space assertions pass. Not yet run: the actual
smoke-training/A-B validation ladder steps in TRAINING.md's "Generation 4"
section, before committing to the full ~32h curriculum.

See TRAINING.md's "Generation 4" section for the full design writeup.

Game/scripts/ai_ship_controller.gd

@@ -17,13 +17,16 @@ extends ShipController
 # Uniform noise magnitude added to each action axis (0 = play at full skill).
 @export_range(0.0, 1.0) var action_noise: float = 0.0
 
-# Must mirror whatever the model was actually trained with (see
-# ShipAIController's identical exports on the training side, curriculum
-# stages 1-2 in TRAINING.md). A model trained grounded (mask on) never got a
-# reward gradient on these axes, so its raw output there is untrained noise —
-# leaving this true for such a model doesn't make it fly well, it just lets
-# that noise reach the ship instead of being discarded like it was in
-# training. Set false to match a grounded-trained model's actual behaviour.
+# Only meaningful for a "continuous"-action_space model (see
+# ShipActionCodec) — i.e. one exported before curriculum generation 4, such
+# as Game/bots/promoted/reference-grounded.json. Must mirror whatever the
+# model was actually trained with: a model trained grounded (mask on) never
+# got a reward gradient on these axes, so its raw output there is untrained
+# noise — leaving this true for such a model doesn't make it fly well, it
+# just lets that noise reach the ship instead of being discarded like it was
+# in training. Set false to match a grounded-trained model's actual
+# behaviour. Generation-4-onward (multi_discrete) models train the full
+# action space from the start, so these flags are ignored for them.
 @export var allow_vertical := true
 @export var allow_pitch_roll := true
 
@@ -59,26 +62,18 @@ func get_action() -> ShipAction:
 func _decide() -> void:
 	var obs := ShipObservations.build(_ship, _opponent, _ball, _attack_goal_position)
 	var out := _policy.forward(obs)
-	# Output layout is the trainer's flattened action space (Box(7)), which
-	# gymnasium orders by SORTED key name — rotation xyz, thrust xyz, turbo
-	# (> 0 means on) — NOT ShipAction's thrust-first declaration order.
-	_action.rotation = Vector3(
-		_axis(out[0]) if allow_pitch_roll else 0.0,
-		_axis(out[1]),
-		_axis(out[2]) if allow_pitch_roll else 0.0
-	)
-	_action.thrust = Vector3(
-		_axis(out[3]),
-		_axis(out[4]) if allow_vertical else 0.0,
-		_axis(out[5])
-	)
-	_action.turbo = out[6] > 0.0
-
-
-func _axis(value: float) -> float:
-	if action_noise > 0.0:
-		value += randf_range(-action_noise, action_noise)
-	return clampf(value, -1.0, 1.0)
+	# See ShipActionCodec for the decode — the single source of truth shared
+	# with the training side, so this must never reimplement layout/ordering
+	# locally (see that file's header for why).
+	if _policy.action_space.get("type", "continuous") == "continuous":
+		_action = ShipActionCodec.from_continuous(out, action_noise)
+		if not allow_pitch_roll:
+			_action.rotation.x = 0.0
+			_action.rotation.z = 0.0
+		if not allow_vertical:
+			_action.thrust.y = 0.0
+	else:
+		_action = ShipActionCodec.from_logits(out, action_noise)
 
 
 # Find ship/ball/opponent/goal once everything is spawned. ShipAction axes

Game/scripts/policy_network.gd

@@ -14,9 +14,18 @@ extends RefCounted
 #     {"weights": [[out x in floats]], "biases": [out floats], "activation": "tanh" | "linear"},
 #     ...
 #   ]
+#   "action_space": {"type": "multi_discrete", "heads": [{"name","bins"}, ...]}  // optional
 # }
+#
+# "action_space" is absent from every model exported before curriculum
+# generation 4 (e.g. Game/bots/promoted/easy.json) — absence means
+# {"type": "continuous"}, decoded via ShipActionCodec.from_continuous, the
+# same flattened-Box(7)-mean-output path this class has always produced.
+# This class itself never changes behaviour based on it; only the caller
+# (AIShipController._decide) branches on action_space["type"].
 
 var input_size: int = 0
+var action_space: Dictionary = {"type": "continuous"}
 var _layers: Array = []
 
 
@@ -32,6 +41,7 @@ static func load_from_file(path: String) -> PolicyNetwork:
 
 	var net := PolicyNetwork.new()
 	net.input_size = int(data.get("input_size", 0))
+	net.action_space = data.get("action_space", {"type": "continuous"})
 	for layer in data["layers"]:
 		# Flatten each layer's weights into a PackedFloat64Array for speed
 		var out_size: int = layer["biases"].size()
@@ -55,7 +65,14 @@ static func load_from_file(path: String) -> PolicyNetwork:
 
 
 func forward(observation: Array) -> Array:
-	var x := PackedFloat64Array(observation)
+	if observation.size() < input_size:
+		push_error("PolicyNetwork: observation has %d values, model expects %d" % [observation.size(), input_size])
+	# Slice rather than trust the caller: ShipObservations.SIZE only ever
+	# grows (append-only), so an older/smaller model must still decode
+	# correctly against a newer, longer observation vector — the extra
+	# trailing values it never trained on are simply dropped here rather
+	# than corrupting the first layer's dot product by accident.
+	var x := PackedFloat64Array(observation.slice(0, input_size))
 	for layer in _layers:
 		var in_size: int = layer["in_size"]
 		var out_size: int = layer["out_size"]

Game/scripts/ship.gd

@@ -119,6 +119,16 @@ func _ready():
 			controller = child
 			break
 
+	# Always on (moved here from ShipAIController.setup, which only enabled
+	# it for training-side ships): ShipObservations now reads own-contact
+	# state (see its "contact" section) for every ship, training or shipped,
+	# so the RigidBody3D contact list must exist unconditionally rather than
+	# only for whichever ship happened to be a training agent. Cheap — a
+	# short per-tick contact list from the physics engine, not a rendering
+	# cost like the headless skips just below.
+	contact_monitor = true
+	max_contacts_reported = 8
+
 	_apply_team_color()
 
 	_boundary = get_tree().get_first_node_in_group("arena_boundary")

Game/scripts/ship_action_codec.gd

@@ -0,0 +1,128 @@
+class_name ShipActionCodec
+extends RefCounted
+
+# Single source of truth for the RL action layout — shared by training
+# (ShipAIController.get_action_space/set_action) and in-game inference
+# (AIShipController._decide via PolicyNetwork) so a trained policy's action
+# output is decoded identically in both contexts. Mirrors ShipObservations'
+# "do not fork this logic" role for observations; the train/inference seam
+# broke once before over exactly this kind of divergence (commit 8c15c46).
+#
+# Curriculum generation 4 replaces the old continuous Gaussian action space
+# (Box(7), see the "continuous" path below) with a per-axis MultiDiscrete
+# space: PPO's Gaussian std reliably collapsed to ~0.13-0.15 within the first
+# ~10% of every training run across 3 generations and never recovered, which
+# made a *sustained* set-point (e.g. hovering, thrust.y ~= 0.408 given this
+# ship's mass/thrust — see TRAINING.md) essentially unreachable: the
+# collapsed distribution can brush the hover value but never hold it long
+# enough to accumulate the reward signal that would move the mean. A
+# discrete bin is a single, atomic, repeatable choice with non-zero
+# probability under any softmax, which does not have that failure mode.
+#
+# HEADS order is deliberately gymnasium's *sorted* key order (verified:
+# "rot_x" < "rot_y" < "rot_z" < "thrust_x" < "thrust_y" < "thrust_z" <
+# "turbo") — godot_rl's ActionSpaceProcessor builds the Tuple action space
+# from a gymnasium Dict, which sorts keys regardless of insertion order, so
+# this order is what SB3/PPO actually samples/trains against and what
+# set_action() receives keyed by. Do not reorder without re-verifying that
+# sort order.
+const HEADS := [
+	{"name": "rot_x", "bins": [-1.0, -0.5, 0.0, 0.5, 1.0]},
+	{"name": "rot_y", "bins": [-1.0, -0.5, 0.0, 0.5, 1.0]},
+	{"name": "rot_z", "bins": [-1.0, -0.5, 0.0, 0.5, 1.0]},
+	{"name": "thrust_x", "bins": [-1.0, -0.5, 0.0, 0.5, 1.0]},
+	# Deliberately asymmetric: hovering this ship (mass 5.0, vertical_thrust
+	# 120, default gravity 9.8 m/s^2 — see ship.gd/ship.tscn) requires a
+	# sustained thrust.y ~= 0.408. Uniform-random selection over these 5 bins
+	# averages 0.34 — just below neutral buoyancy, so a fresh policy drifts
+	# gently through the volume instead of pinning to the floor (symmetric
+	# bins) or sticking to the ceiling (ceiling_pull_strength 11.5 > gravity
+	# 9.8, so the ceiling is easy to over-shoot into). This is the direct
+	# analogue of the RLGym/RLBot community fix for the same failure mode
+	# ("add more jump actions to the discrete action parser").
+	{"name": "thrust_y", "bins": [-0.5, 0.0, 0.45, 0.75, 1.0]},
+	{"name": "thrust_z", "bins": [-1.0, -0.5, 0.0, 0.5, 1.0]},
+	{"name": "turbo", "bins": [0.0, 1.0]},
+]
+
+
+static func action_space_dict() -> Dictionary:
+	var space := {}
+	for head in HEADS:
+		space[head["name"]] = {"size": head["bins"].size(), "action_type": "discrete"}
+	return space
+
+
+# Training side: `action` is the Dictionary godot_rl's Sync node hands
+# set_action() — one entry per HEADS key, each an int (or int-valued float)
+# bin index in [0, bins.size()).
+static func from_indices(action: Dictionary) -> ShipAction:
+	var result := ShipAction.new()
+	var values := {}
+	for head in HEADS:
+		var index: int = clampi(int(round(float(action[head["name"]]))), 0, head["bins"].size() - 1)
+		values[head["name"]] = head["bins"][index]
+	result.rotation = Vector3(values["rot_x"], values["rot_y"], values["rot_z"])
+	result.thrust = Vector3(values["thrust_x"], values["thrust_y"], values["thrust_z"])
+	result.turbo = values["turbo"] > 0.0
+	return result
+
+
+# In-game inference for a MultiDiscrete-trained export: `logits` is the raw
+# policy_network.gd output — 32 floats (5+5+5+5+5+5+2), one contiguous slice
+# per head in HEADS order (matches export_policy.py's action_net layer,
+# which concatenates SB3's per-head categorical logits in that same order).
+# argmax within each slice picks that head's bin, same as SB3's
+# MultiCategoricalDistribution.mode() under deterministic inference.
+static func from_logits(logits: Array, noise: float) -> ShipAction:
+	var result := ShipAction.new()
+	var values := {}
+	var offset := 0
+	for head in HEADS:
+		var bins: Array = head["bins"]
+		var index := 0
+		if noise > 0.0 and randf() < noise:
+			# eps-random-bin: the discrete analogue of continuous action_noise
+			# (see ai_ship_controller.gd) — degrades gracefully and keeps the
+			# same 0..1 monotonic difficulty semantics as the continuous path.
+			index = randi() % bins.size()
+		else:
+			var best_value: float = logits[offset]
+			for i in range(1, bins.size()):
+				if logits[offset + i] > best_value:
+					best_value = logits[offset + i]
+					index = i
+		values[head["name"]] = bins[index]
+		offset += bins.size()
+	result.rotation = Vector3(values["rot_x"], values["rot_y"], values["rot_z"])
+	result.thrust = Vector3(values["thrust_x"], values["thrust_y"], values["thrust_z"])
+	result.turbo = values["turbo"] > 0.0
+	return result
+
+
+# Legacy continuous decode — moved verbatim from ai_ship_controller.gd so
+# every model exported before generation 4 (no "action_space" block in its
+# JSON, e.g. Game/bots/promoted/easy.json) keeps behaving byte-identically.
+# `out` is the trainer's flattened Box(7) output, gymnasium-sorted: rotation
+# xyz, thrust xyz, turbo (> 0 means on) — NOT ShipAction's thrust-first
+# declaration order.
+static func from_continuous(out: Array, noise: float) -> ShipAction:
+	var result := ShipAction.new()
+	result.rotation = Vector3(
+		_continuous_axis(out[0], noise),
+		_continuous_axis(out[1], noise),
+		_continuous_axis(out[2], noise)
+	)
+	result.thrust = Vector3(
+		_continuous_axis(out[3], noise),
+		_continuous_axis(out[4], noise),
+		_continuous_axis(out[5], noise)
+	)
+	result.turbo = out[6] > 0.0
+	return result
+
+
+static func _continuous_axis(value: float, noise: float) -> float:
+	if noise > 0.0:
+		value += randf_range(-noise, noise)
+	return clampf(value, -1.0, 1.0)

Game/scripts/ship_action_codec.gd.uid

@@ -0,0 +1 @@
+uid://cvpbp3mj58ejd

Game/scripts/ship_ai_controller.gd

@@ -7,10 +7,11 @@ extends AIController3D
 # the trainer are written into an RLShipController, which the ship pulls like
 # any other controller.
 #
-# Action space is ShipAction verbatim: 6 continuous axes (thrust xyz,
-# rotation xyz, each -1..1) + binary turbo. ShipAction axes are ship-local
-# (body frame), so they need no team mirroring — only observations do
-# (see ShipObservations.canon).
+# Action space/layout is owned by ShipActionCodec (get_action_space/
+# set_action just delegate to it) — see that file for the per-axis
+# MultiDiscrete design and why. ShipAction axes are ship-local (body frame),
+# so they need no team mirroring — only observations do (see
+# ShipObservations.canon).
 
 # Reward shaping weights. Dense terms accrue per physics tick (60 sim-ticks
 # per sim-second); event terms fire once. Exported so tuning needs no code
@@ -54,9 +55,13 @@ extends AIController3D
 # relative to the (then far weaker) ball-seeking shaping.
 @export var wall_contact_penalty := 0.0025
 # Per-tick penalty for not being upright, scaled by tilt: 0 when flat, full
-# value (-0.12/s) when inverted. A penalty rather than an upright bonus so a
-# flat, idle ship farms nothing.
-@export var tilt_penalty := 0.002
+# value when inverted. A penalty rather than an upright bonus so a flat, idle
+# ship farms nothing. Lowered 4x for curriculum generation 4 (was 0.002,
+# -0.12/s): a genuine aerial approach to a high ball requires pitching, and
+# the old value quietly opposed the exact behaviour generation 4 is trying
+# to teach. Not removed outright — an always-inverted bot still looks bad in
+# a shipped game.
+@export var tilt_penalty := 0.0005
 # Per-tick bonus for own speed: 0 stationary, full value (+0.24/s) at
 # max_speed. Run07 lesson: after the kickoff flurry both ships parked next to
 # a cornered ball — with every other dense term near zero there, standing
@@ -79,27 +84,15 @@ extends AIController3D
 # unaffected; the floor-lock curriculum stage turns it on.
 @export var airborne_penalty := 0.0
 
-# Locomotion curriculum: scales how much of the corresponding action axes
-# actually reaches the ship, from 0.0 (fully discarded, grounded-only) to
-# 1.0 (full effect) — this scales the *effect* of thrust.y/rotation.x/
-# rotation.z in set_action, not the action space's shape: the policy always
-# outputs values for these axes (always contributing to PPO's entropy/log-
-# prob), they're just attenuated here, so checkpoints stay resumable across
-# ramp values.
-#
-# A hard 0/1 flip (the original bool mask) let PPO's action-distribution
-# std collapse to ~0.13-0.15 within the first ~10% of steps, before the
-# policy ever meaningfully explored the newly-unmasked axes — 3 independent
-# 240M-step attempts at the all-or-nothing flip all landed at a stable
-# ~28-32% win rate vs curric-s5-aggression (see TRAINING.md's generation 3
-# section). A gradual ramp across several short curriculum stages, each
-# resuming from the previous ramp value's checkpoint, lets the policy adopt
-# each axis incrementally instead of all at once.
-@export_range(0.0, 1.0) var vertical_ramp := 1.0
-@export_range(0.0, 1.0) var pitch_roll_ramp := 1.0
+# Height above which a touch counts toward air_touch_fraction telemetry
+# (see get_info) — not a reward term itself, see set_action/get_info's
+# comments on why generation 4 deliberately does not add a standalone
+# air-touch reward.
+const AIR_TOUCH_HEIGHT := 5.0
 
 # Contact normals with y above this are floor contact (exempt from the wall
 # penalty); below it they read as wall (sideways) or ceiling (downward).
+# Mirrors ShipObservations.FLOOR_NORMAL_MIN_Y (see that file's comment).
 const FLOOR_NORMAL_MIN_Y := 0.7
 
 # Longest possible ship-to-ball separation: the enclosure's interior diagonal.
@@ -125,8 +118,32 @@ var attack_goal_position: Vector3
 # (true on goal, false on timeout) rather than toggle, so no reset is needed.
 var goal_scored_this_episode := false
 
+# Set only in the timeout branch (TrainingMode._physics_process), never on a
+# goal — a goal is a genuine terminal (V(s)=0 is correct there); a timeout
+# is an artificial episode boundary the value function should be bootstrapped
+# through instead (see get_info). Same "always overwritten by both call
+# sites, never cleared in reset()" pattern as goal_scored_this_episode above,
+# for the same reason.
+var truncated_this_episode := false
+var terminal_obs: Array = []
+
 var _ticks_since_ball_touch := 1 << 30  # large so the first touch always pays
 
+# Flight telemetry (see get_info) — leading indicators for whether the
+# policy is actually using its vertical/pitch-roll authority, visible from
+# the first rollout instead of only in a win-rate number measured a full
+# training run later. Accumulated per-tick, reset() zeroes them each episode;
+# get_info() reports the running fraction/mean so the *final* tick of an
+# episode (the one VecMonitor's info_keywords captures) holds the whole
+# episode's aggregate.
+const AIRBORNE_ALTITUDE_THRESHOLD := 3.0
+var _telemetry_ticks := 0
+var _airborne_ticks := 0
+var _altitude_sum := 0.0
+var _thrust_y_sum := 0.0
+var _touches := 0
+var _air_touches := 0
+
 
 # Wire up references after the ship is spawned. `attack_goal` is the goal
 # this ship scores into (goal.team == opponent's team).
@@ -138,10 +155,8 @@ func setup(p_ship: Ship, p_rl_controller: RLShipController, p_ball: RigidBody3D,
 	attack_goal_position = p_attack_goal_position
 	init(ship)
 
-	# Contact monitoring for the ball-touch reward (training-only cost;
-	# the shipped game leaves contact_monitor off).
-	ship.contact_monitor = true
-	ship.max_contacts_reported = 8
+	# ship.contact_monitor is on unconditionally (see ship.gd) since
+	# ShipObservations now reads it for every ship, not just training agents.
 	ship.body_entered.connect(_on_ship_body_entered)
 
 
@@ -153,36 +168,48 @@ func get_reward() -> float:
 	return reward
 
 
-# Symmetric across both self-play agents: reports whether this episode ended
-# in a goal at all, not which team scored — a clean "goal rate" signal
-# distinct from rollout/ep_rew_mean, which mixes this with dense shaping
-# (ball chasing/touching). See train.py's GoalRateCallback.
+# Symmetric across both self-play agents. "goal_scored": whether this
+# episode ended in a goal at all (not which team) — a clean "goal rate"
+# signal distinct from rollout/ep_rew_mean, which mixes this with dense
+# shaping (ball chasing/touching); see train.py's GoalRateCallback.
+# "truncated"/"terminal_obs": only present on a timeout tick — remapped by
+# cosmic_env.py into SB3's expected "TimeLimit.truncated"/
+# "terminal_observation" keys so PPO bootstraps V(s) through episode
+# timeouts instead of treating every 30s draw as a true terminal state (a
+# real, previously-unnoticed bug independent of the action-space work — see
+# TRAINING.md). Flight telemetry fields are leading indicators for
+# generation 4's core hypothesis (see train.py's FlightTelemetryCallback).
 func get_info() -> Dictionary:
-	return {"goal_scored": goal_scored_this_episode}
+	var info := {"goal_scored": goal_scored_this_episode}
+	if truncated_this_episode:
+		info["truncated"] = true
+		info["terminal_obs"] = terminal_obs
+	if _telemetry_ticks > 0:
+		info["airborne_fraction"] = float(_airborne_ticks) / _telemetry_ticks
+		info["mean_altitude"] = _altitude_sum / _telemetry_ticks
+		info["vertical_thrust_mean"] = _thrust_y_sum / _telemetry_ticks
+	if _touches > 0:
+		info["air_touch_fraction"] = float(_air_touches) / _touches
+	return info
 
 
 func get_action_space() -> Dictionary:
-	return {
-		"thrust": {"size": 3, "action_type": "continuous"},
-		"rotation": {"size": 3, "action_type": "continuous"},
-		"turbo": {"size": 2, "action_type": "discrete"},
-	}
+	return ShipActionCodec.action_space_dict()
 
 
 func set_action(action) -> void:
-	var thrust: Array = action["thrust"]
-	var rot: Array = action["rotation"]
-	var thrust_y: float = thrust[1] * vertical_ramp
-	var pitch: float = rot[0] * pitch_roll_ramp
-	var roll: float = rot[2] * pitch_roll_ramp
-	rl_controller.action.thrust = Vector3(thrust[0], thrust_y, thrust[2])
-	rl_controller.action.rotation = Vector3(pitch, rot[1], roll)
-	rl_controller.action.turbo = int(action["turbo"]) == 1
+	rl_controller.action = ShipActionCodec.from_indices(action)
 
 
 func reset():
 	super()
 	_ticks_since_ball_touch = 1 << 30
+	_telemetry_ticks = 0
+	_airborne_ticks = 0
+	_altitude_sum = 0.0
+	_thrust_y_sum = 0.0
+	_touches = 0
+	_air_touches = 0
 
 
 func _physics_process(delta):
@@ -238,19 +265,17 @@ func _physics_process(delta):
 		var height := maxf(ship.global_position.y, 0.0)
 		reward -= airborne_penalty * height / ArenaBoundary.INNER_HEIGHT
 
+	# Flight telemetry accumulation (see get_info) — not reward, just
+	# observation of what the policy is actually doing this tick.
+	_telemetry_ticks += 1
+	_altitude_sum += ship.global_position.y
+	if ship.global_position.y > AIRBORNE_ALTITUDE_THRESHOLD:
+		_airborne_ticks += 1
+	_thrust_y_sum += rl_controller.action.thrust.y
+
 
 func _wall_or_ceiling_contact() -> bool:
-	var state := PhysicsServer3D.body_get_direct_state(ship.get_rid())
-	if state == null:
-		return false
-	for i in state.get_contact_count():
-		if not state.get_contact_collider_object(i) is ArenaBoundary:
-			continue
-		# Normal points from the surface into the ship: floor ≈ +Y (exempt),
-		# anything flatter or downward is a wall or the ceiling.
-		if state.get_contact_local_normal(i).y < FLOOR_NORMAL_MIN_Y:
-			return true
-	return false
+	return ShipObservations.contact_normal(ship) != Vector3.ZERO
 
 
 func _on_ship_body_entered(body: Node) -> void:
@@ -264,3 +289,12 @@ func _on_ship_body_entered(body: Node) -> void:
 		alignment = clampf(ball.linear_velocity.normalized().dot(to_goal.normalized()), 0.0, 1.0)
 	reward += ball_touch_reward * lerpf(ball_touch_direction_floor, 1.0, alignment)
 	_ticks_since_ball_touch = 0
+
+	# Telemetry only (see get_info's air_touch_fraction) — not a reward term.
+	# Generation 4 deliberately doesn't reward high touches directly (see
+	# TRAINING.md's "why no air-touch reward" note); this just measures
+	# whether the air-drill state setter is producing genuine aerial
+	# contests, so a future decision to add one is data-driven.
+	_touches += 1
+	if ball.global_position.y > AIR_TOUCH_HEIGHT:
+		_air_touches += 1

Game/scripts/ship_observations.gd

@@ -19,8 +19,20 @@ const POSITION_SCALE := Vector3(20.0, 10.0, 20.0)
 const BALL_SPEED_SCALE := 30.0
 const GOAL_DISTANCE_SCALE := 40.0
 
-# Number of floats build() returns; the policy input size.
-const SIZE := 31
+# Contact normals with y above this are floor contact; below it they read as
+# wall (sideways) or ceiling (downward) — mirrors
+# ShipAIController.FLOOR_NORMAL_MIN_Y (kept here too since ShipAIController's
+# wall_contact_penalty and this observation feature must agree on what
+# counts as "in contact" for the reward/observation to stay consistent).
+const FLOOR_NORMAL_MIN_Y := 0.7
+
+# Number of floats build() returns; the policy input size. APPEND-ONLY: new
+# features go on the end and existing indices never move, so an old exported
+# model (whose network was trained against a shorter SIZE) still decodes its
+# first N inputs identically when SIZE grows — see PolicyNetwork.forward's
+# input_size slice/guard. Do not retune an *existing* index without
+# retraining every model in Game/bots/.
+const SIZE := 35
 
 
 # 180° rotation about Y for team 1; identity for team 0. A proper rotation
@@ -62,6 +74,17 @@ static func build(ship: Ship, opponent: Ship, ball: RigidBody3D, attack_goal_pos
 	_append(obs, canon(goal_rel, team) / POSITION_SCALE)
 	obs.append(goal_rel.length() / GOAL_DISTANCE_SCALE)
 
+	# Own contact state (appended — see SIZE's append-only invariant).
+	# Added for generation 4: ShipAIController's wall_contact_penalty used to
+	# fire on a condition the observation vector couldn't see coming,
+	# leaving the value function to predict a reward with no supporting
+	# signal. Also gives the policy a direct "am I resting on a surface"
+	# signal it can use to push off (a real aerial mechanic), distinct from
+	# inferring it indirectly from position/up-vector.
+	var normal := contact_normal(ship)
+	_append(obs, canon(normal, team))
+	obs.append(1.0 if normal != Vector3.ZERO else 0.0)
+
 	return obs
 
 
@@ -69,3 +92,22 @@ static func _append(obs: Array, v: Vector3) -> void:
 	obs.append(v.x)
 	obs.append(v.y)
 	obs.append(v.z)
+
+
+# Aggregate wall/ceiling contact normal (zero if none, or if only floor
+# contact — floor contact is excluded so "in_contact" means "touching
+# something other than the ground it's expected to rest on", matching
+# ShipAIController.wall_contact_penalty's own floor exemption). Requires
+# ship.contact_monitor (see ship.gd's _ready — on unconditionally for every
+# ship so training and in-game inference see identical observations).
+static func contact_normal(ship: Ship) -> Vector3:
+	var state := PhysicsServer3D.body_get_direct_state(ship.get_rid())
+	if state == null:
+		return Vector3.ZERO
+	for i in state.get_contact_count():
+		if not state.get_contact_collider_object(i) is ArenaBoundary:
+			continue
+		var normal := state.get_contact_local_normal(i)
+		if normal.y < FLOOR_NORMAL_MIN_Y:
+			return normal
+	return Vector3.ZERO

Game/scripts/training_mode.gd

@@ -54,6 +54,16 @@ extends GameMode
 # trainee's near-goal resets are always finishing chances, not a coin flip
 # between attacking and defending an empty net.
 @export_range(0.0, 1.0) var attack_goal_bias := 0.5
+# Fourth episode-start branch (after kickoff/near-goal, before the fully-
+# random fallback): ball spawned high, both ships spawned low and lateral —
+# unsolvable without climbing. Default 0 (off) so ordinary runs are
+# unaffected. Added for curriculum generation 4: the existing random branch
+# already samples ship/ball Y across the full arena height, but that only
+# randomizes the *initial* state — under gravity+drag a floor-pinned policy
+# sinks back to the floor in ~1.5s, so the *stationary* state distribution
+# stayed floor-pinned even though the initial one wasn't. See
+# _place_air_drill.
+@export_range(0.0, 1.0) var air_drill_chance := 0.0
 
 # Placement bounds for randomized episode starts, derived from the standard
 # enclosure (ArenaBoundary). The inset keeps a randomly oriented ship (1x1x4
@@ -188,7 +198,7 @@ func _parse_eval_args() -> void:
 # of silently matching an unrelated inherited export.
 const TRAINING_MODE_OVERRIDES := [
 	"goal_reward", "draw_penalty", "kickoff_state_chance",
-	"ball_near_goal_chance", "attack_goal_bias",
+	"ball_near_goal_chance", "attack_goal_bias", "air_drill_chance",
 ]
 # ShipAIController @export names a curriculum run may override, read as
 # --ai_<name>=<value> to avoid colliding with the names above.
@@ -196,7 +206,7 @@ const SHIP_AI_OVERRIDES := [
 	"ball_touch_reward", "ball_touch_cooldown_ticks", "ball_touch_direction_floor",
 	"velocity_to_ball_weight", "ball_velocity_to_goal_weight", "ball_distance_penalty",
 	"wall_contact_penalty", "tilt_penalty", "speed_reward_weight", "time_penalty",
-	"airborne_penalty", "vertical_ramp", "pitch_roll_ramp",
+	"airborne_penalty",
 ]
 
 
@@ -243,11 +253,10 @@ func _ai_default(name: String) -> Variant:
 		"ball_velocity_to_goal_weight": return 0.004
 		"ball_distance_penalty": return 0.002
 		"wall_contact_penalty": return 0.0025
-		"tilt_penalty": return 0.002
+		"tilt_penalty": return 0.0005
 		"speed_reward_weight": return 0.004
 		"time_penalty": return 0.001
 		"airborne_penalty": return 0.0
-		"vertical_ramp", "pitch_roll_ramp": return 1.0
 		_: return null
 
 
@@ -299,6 +308,14 @@ func _physics_process(_delta):
 				agent.reward -= draw_penalty
 				agent.done = true
 				agent.goal_scored_this_episode = false
+				# Snapshot BEFORE _reset_episode() below, which moves the ship/
+				# ball and would otherwise make this the post-reset state, not
+				# the terminal one PPO needs to bootstrap V(s) from (see
+				# ShipAIController.get_info / cosmic_env.py's truncation remap).
+				# A goal (_on_goal_scored) does NOT do this — a goal is a
+				# genuine terminal, V(s)=0 is correct there.
+				agent.truncated_this_episode = true
+				agent.terminal_obs = ShipObservations.build(agent.ship, agent.opponent, agent.ball, agent.attack_goal_position)
 		_reset_episode()
 		return
 
@@ -330,6 +347,7 @@ func _on_goal_scored(conceding_team: int) -> void:
 		agent.reward += goal_reward if agent.ship.team != conceding_team else -goal_reward
 		agent.done = true
 		agent.goal_scored_this_episode = true
+		agent.truncated_this_episode = false  # genuine terminal, not a timeout
 	_reset_episode()
 
 
@@ -363,6 +381,8 @@ func _reset_episode() -> void:
 	elif roll < kickoff_state_chance + ball_near_goal_chance:
 		_place_ships_random()
 		_place_ball_near_goal()
+	elif roll < kickoff_state_chance + ball_near_goal_chance + air_drill_chance:
+		_place_air_drill()
 	else:
 		_place_ships_random()
 		_place_ball_random()
@@ -373,6 +393,46 @@ func _place_ball_random() -> void:
 	_place_body(ball, Transform3D(Basis.IDENTITY, _random_position()), velocity, Vector3.ZERO)
 
 
+# Extra clearance for the air drill's ball placement specifically — well
+# beyond SPAWN_INSET, and well beyond the ball's own radius. The ball (unlike
+# _random_position) has no collision-avoidance resample, so this is the
+# anti-exploit measure: the RLGym wall-bounce exploit ("hits the ball off a
+# wall high up instead of doing a real aerial") needs a wall to bounce off,
+# so simply not generating ball states anywhere near one removes the exploit
+# from the training distribution entirely, rather than trying to price it
+# out via reward shaping.
+const AIR_DRILL_BALL_WALL_CLEARANCE := 5.0
+
+# Air drill state (see air_drill_chance): ball spawned high, both ships
+# spawned low and lateral, so the state is unsolvable without climbing.
+func _place_air_drill() -> void:
+	var ball_half_x := ArenaBoundary.INNER_HALF_X - AIR_DRILL_BALL_WALL_CLEARANCE
+	var ball_half_z := ArenaBoundary.GOAL_LINE_Z - AIR_DRILL_BALL_WALL_CLEARANCE
+	var ball_position := Vector3(
+		randf_range(-ball_half_x, ball_half_x),
+		randf_range(ArenaBoundary.INNER_HEIGHT * 0.45, FIELD_MAX_Y),
+		randf_range(-ball_half_z, ball_half_z)
+	)
+	var ball_velocity := _random_direction() * randf_range(0.0, MAX_RANDOM_BALL_SPEED * 0.5)
+	_place_body(ball, Transform3D(Basis.IDENTITY, ball_position), ball_velocity, Vector3.ZERO)
+
+	for ship in ships:
+		if ship in _inert_ships:
+			continue
+		var lateral_offset := Vector3(randf_range(-1, 1), 0.0, randf_range(-1, 1))
+		lateral_offset = lateral_offset.normalized() if lateral_offset.length_squared() > 0.001 else Vector3.FORWARD
+		lateral_offset *= randf_range(6.0, 14.0)
+		var ship_position := Vector3(
+			clampf(ball_position.x + lateral_offset.x, -FIELD_HALF_X, FIELD_HALF_X),
+			randf_range(FIELD_MIN_Y, 4.0),
+			clampf(ball_position.z + lateral_offset.z, -FIELD_HALF_Z, FIELD_HALF_Z)
+		)
+		var orientation := Basis.from_euler(Vector3(
+			randf_range(-0.4, 0.4), randf_range(-PI, PI), randf_range(-0.4, 0.4)
+		))
+		_place_body(ship, Transform3D(orientation, ship_position), Vector3.ZERO, Vector3.ZERO)
+
+
 # Attacking/defending drill states: ball close to a goal, moving toward it.
 # Which goal is picked is biased by attack_goal_bias (0.5 = uniform between
 # both, matching historical behaviour; 1.0 = always the goal team 0 attacks).