chore(training): add air_approach_weight and restart stage-5 intercepts

Stage 5 blocked all three attempts on productive_air_touch_fraction
stuck exactly at 0.0 across a continuous 180M-step lineage, while
goal_rate/upright_fraction/forward_motion_fraction kept improving on
the same budget. forward_velocity_to_ball_weight (the term that solved
Stage 4's ground pursuit) is hard-gated below GROUND_HANDLING_HEIGHT
and does nothing in the air, so Stage 5's air_intercept_chance had no
matching aerial incentive to learn from. air_approach_weight adds the
airborne mirror (nose-first 3D closing speed, no uprightness
multiplier) and folds into HANDLING_REWARD_FLAGS so Stage 6 inherits
it too. Deleted the three blocked attempts and reset state to resume
Stage 5 from the Stage-4 checkpoint with the new term.
This commit is contained in:
Josh Creek
2026-08-18 16:03:56 +01:00
parent 0e685edcf8
commit 88591e031f
15 changed files with 93 additions and 254 deletions
+31
View File
@@ -151,9 +151,40 @@ STANDING_ARGS = ["--ent-coef", "0.01", "--entropy-floor"]
# floor, and air-drill-chance goes to 0. The reward terms already built
# are left exactly as they were — they should finally pull in a direction
# the ship can go.
#
# Round 7 (2026-08-18): Stage 4 closed by human override (see TRAINING.md).
# Stage 5 (intercepts) then blocked all three attempts on the same single
# floor every time — rollout/productive_air_touch_fraction stayed exactly
# 0.0 across a continuous 180M-step lineage (each retry resumes the
# previous attempt's checkpoint, not a fresh run), while air_touch_fraction
# sat at noise level (0.00008 -> 0.00006 -> 0.00006) and goal_rate/
# upright_fraction/forward_motion_fraction all kept improving on the same
# budget. A dead-flat metric across that much continued training, next to
# metrics that keep moving, is the missing-mechanism signature from Round 6
# again, not a slow-learning one: forward_velocity_to_ball_weight -- the
# term that actually solved ground handling -- is hard-gated to
# ship.global_position.y < GROUND_HANDLING_HEIGHT and does nothing in the
# air, so air_intercept_chance (added for Stage 5) was asking for aerial
# pursuit with only the generic, orientation-agnostic velocity_to_ball_
# weight (0.04) to learn it from -- the same class of gap as Stage 4's
# missing ground-tilt/non-forward pressure before those were added.
#
# air_approach_weight (ship_ai_controller.gd) is the airborne mirror:
# nose-first 3D closing speed on the ball, active above
# GROUND_HANDLING_HEIGHT instead of below it (mutually exclusive with
# forward_velocity_to_ball_weight by altitude), with no uprightness
# multiplier since a real aerial requires pitching away from level. Set to
# 0.15 to match forward_velocity_to_ball_weight's proven-effective
# magnitude; added to HANDLING_REWARD_FLAGS (not just Stage 5's flags) so
# it also carries into Stage 6, which reuses these flags and its own
# air_intercept_chance. Stage 5 restarts from Stage 4's checkpoint rather
# than continuing retry2's, same reasoning as every previous mechanism
# change in this file: don't resume a policy shaped by an absent term into
# one where it now exists.
HANDLING_REWARD_FLAGS = [
"--velocity-to-ball-weight", "0.04",
"--forward-velocity-to-ball-weight", "0.15",
"--air-approach-weight", "0.15",
"--ball-distance-penalty", "0.01",
"--ball-touch-reward", "0.7",
"--ball-velocity-to-goal-weight", "0.06",