fix(training): make the stage-5 air-intercept drill physically solvable

productive_air_touch_fraction sat at exactly 0.0 across nine Stage-5
attempts and 540M timesteps. Two rounds of reward shaping were aimed at
it (air_approach_weight, then air_touch_bonus_weight); both worked --
airborne_fraction 0.223->0.258, mean_altitude 2.59->3.25,
vertical_thrust_mean 0.004->0.063 -- and the ship now visibly plays the
ball in the air. The metric could not see it because it counts only
touches with the ball above AIR_TOUCH_HEIGHT (5m), and
_place_air_intercept never produced a reachable one.

Simulating the spawn distribution against the ship's flight envelope
(vertical_thrust 120 / mass 5 = 24 m/s^2 less gravity, drag capping
climb near 12 m/s): a ball spawned 6-12m up at 6-11 m/s is above 5m for
a median of 0.80s, while the ship spawned 7-13m behind, 3-10m below, and
at a dead stop. An ideal interceptor -- point mass, instant attitude, no
righting torque, zero reaction delay -- makes that touch in 0.00% of
episodes and reaches the ball at all in 0.5%.

Retune the drill instead of the reward: ball higher (8-14m) and slower
(4-8 m/s), ship closer (4-9m behind), narrower lateral spread, and a
6-14 m/s planar run-up rather than a standing start -- the dead stop was
the largest single factor. Ideal interceptor now reaches the ball in
~98% of episodes and above 5m in ~37%, so the 0.005 floor has headroom.
AIR_TOUCH_HEIGHT stays 5.0 so the metric remains comparable with earlier
generations.

Resume from retry2 rather than restarting from Stage 4: that rule guards
against a changed reward function invalidating the value function, and
the reward function is untouched here -- only the state distribution
moved, so the policy that already learned to fly is what should be
pointed at a reachable target. Adds a one-shot resume_override to
generation5_state.json, consumed on first use.
This commit is contained in:
Josh Creek
2026-08-21 15:14:36 +01:00
parent 23e3dd18f9
commit 818f8e89cd
4 changed files with 146 additions and 13 deletions
+39 -8
View File
@@ -558,20 +558,45 @@ func _place_ground_start() -> void:
)
# Air-intercept drill geometry. These six ranges are not free tuning knobs —
# together they decide whether the drill is solvable at all, and the original
# values made it arithmetically impossible (see the Round 9 note in
# training/generation5.py). The constraint: the ball is only above
# ShipAIController.AIR_TOUCH_HEIGHT (5m) for a fixed window after the spawn,
# and the ship has to cross the gap within it. The ship's own numbers cap what
# it can do — vertical_thrust 120 / mass 5 = 24 m/s^2 up, less 9.8 gravity, and
# drag_coefficient 0.98/tick caps climb at roughly 12 m/s — so the window has
# to be sized against those, not chosen for how the drill looks. The values
# below were picked by simulating the spawn distribution against that flight
# envelope: an ideal interceptor now reaches the ball in ~98% of episodes and
# can do so above 5m in ~37%, versus 0% before.
const AIR_INTERCEPT_BALL_Y := Vector2(8.0, 14.0) # higher: more fall time above 5m
const AIR_INTERCEPT_BALL_SPEED := Vector2(4.0, 8.0) # slower: the ball outran the ship
const AIR_INTERCEPT_BEHIND := Vector2(4.0, 9.0) # closer: less gap to close
const AIR_INTERCEPT_LATERAL := 5.0
const AIR_INTERCEPT_SHIP_Y := 4.0 # upper bound; FIELD_MIN_Y is the lower
# A ship in real play is already moving; spawning at a dead stop spent most of
# the drill window just building speed, which was the single largest cause of
# the old geometry being unreachable. Planar only, aimed at the ball, so the
# climb itself is still the ship's own problem to solve.
const AIR_INTERCEPT_SHIP_SPEED := Vector2(6.0, 14.0)
# Goal-relevant aerial intercept: a high ball is already travelling toward a
# randomly selected goal, while ships begin low and behind/lateral to its
# path. The generous wall clearance prevents rebound farming and an upright
# yaw-only spawn avoids wasting the short drill window on random recovery.
# path, already carrying planar speed toward it. The generous wall clearance
# prevents rebound farming and an upright yaw-only spawn avoids wasting the
# short drill window on random recovery.
func _place_air_intercept() -> void:
var goal := _goal_for_team(randi() % 2)
var ball_position := Vector3(
randf_range(-8.0, 8.0),
randf_range(6.0, minf(12.0, FIELD_MAX_Y)),
randf_range(AIR_INTERCEPT_BALL_Y.x, minf(AIR_INTERCEPT_BALL_Y.y, FIELD_MAX_Y)),
randf_range(-10.0, 10.0)
)
var to_goal := (goal.global_position - ball_position).normalized()
var ball_velocity := (to_goal + Vector3(randf_range(-0.15, 0.15), randf_range(0.0, 0.15), 0.0)).normalized() \
* randf_range(6.0, 11.0)
* randf_range(AIR_INTERCEPT_BALL_SPEED.x, AIR_INTERCEPT_BALL_SPEED.y)
_place_body(ball, Transform3D(Basis.IDENTITY, ball_position), ball_velocity, Vector3.ZERO)
var placed: Array[Vector3] = []
@@ -581,17 +606,23 @@ func _place_air_intercept() -> void:
continue
var ship_position := Vector3.ZERO
for _attempt in 20:
var lateral := Vector3(-behind.z, 0.0, behind.x) * randf_range(-7.0, 7.0)
ship_position = ball_position + behind * randf_range(7.0, 13.0) + lateral
var lateral := Vector3(-behind.z, 0.0, behind.x) \
* randf_range(-AIR_INTERCEPT_LATERAL, AIR_INTERCEPT_LATERAL)
ship_position = ball_position \
+ behind * randf_range(AIR_INTERCEPT_BEHIND.x, AIR_INTERCEPT_BEHIND.y) + lateral
ship_position.x = clampf(ship_position.x, -FIELD_HALF_X, FIELD_HALF_X)
ship_position.y = randf_range(FIELD_MIN_Y, 3.0)
ship_position.y = randf_range(FIELD_MIN_Y, AIR_INTERCEPT_SHIP_Y)
ship_position.z = clampf(ship_position.z, -FIELD_HALF_Z, FIELD_HALF_Z)
if _spawn_position_clear(ship_position) and _far_enough_from(ship_position, placed):
break
placed.append(ship_position)
var face_ball := ball_position - ship_position
var yaw := atan2(-face_ball.x, -face_ball.z)
_place_body(ship, Transform3D(Basis.from_euler(Vector3(0.0, yaw, 0.0)), ship_position), Vector3.ZERO, Vector3.ZERO)
var run_up := Vector3(face_ball.x, 0.0, face_ball.z)
run_up = run_up.normalized() * randf_range(
AIR_INTERCEPT_SHIP_SPEED.x, AIR_INTERCEPT_SHIP_SPEED.y
) if run_up.length_squared() > 0.0001 else Vector3.ZERO
_place_body(ship, Transform3D(Basis.from_euler(Vector3(0.0, yaw, 0.0)), ship_position), run_up, Vector3.ZERO)
# Attacking/defending drill states: ball close to a goal, moving toward it.