fix(training): make uprightness a multiplier on the forward-approach reward

Rounds 2 and 3 showed that tuning grounded_upright_reward's magnitude only
slides along a tradeoff instead of resolving it: at 0.015 upright_fraction
climbed to 0.331 while goal_rate sagged to 0.542 (then farmed outright at
0.696/0.366), and at 0.004 goal_rate climbed 0.569->0.604 while
upright_fraction went flat at ~0.26. An additive uprightness bonus is an
alternative to playing well, so the policy just picks whichever is cheaper
and no magnitude buys both behaviours.

Change the mechanism rather than the number: grounded_upright_reward drops
to 0, and uprightness becomes a multiplier inside the nose-led approach
term, which already requires moving forward at the ball. Parked-and-upright
and fast-but-sideways now both pay zero; only upright, forward, nose-on to
the ball pays full. forward-velocity-to-ball rises 0.06 -> 0.15 to offset
the ~2-3x expected-value cut from the new factor, and ground-tilt-penalty
drops 0.05 -> 0.02 now that uprightness is paid positively during play.
Delete the three blocked attempts and reset state to restart from the
Stage-3 foundation.

training/generation5.py

@@ -75,18 +75,43 @@ STANDING_ARGS = ["--ent-coef", "0.01", "--entropy-floor"]
 # continuing from the farming checkpoint, same reasoning as the ground_tilt/
 # non_forward_penalty retune: don't resume a policy shaped by one reward
 # balance into a meaningfully different one.
+#
+# Round 3 (2026-08-12): 0.004 stopped the farming (vertical_thrust_mean
+# stayed positive, airborne_fraction flat) and goal_rate rose across the
+# chain 0.569->0.598->0.604 — but upright_fraction went flat at ~0.26, and
+# retry1 posted the best head-to-head in Stage-4 history (eval goal_rate
+# 0.820, 53-29-18). Lining rounds 2 and 3 up by attempt shows the actual
+# problem: where upright climbed goal_rate sagged, and where goal_rate
+# climbed upright went flat. An *additive* uprightness bonus is an
+# alternative to playing well, so the policy just picks whichever is
+# cheaper and the magnitude only slides along that tradeoff — no value can
+# buy both. Round 4 therefore changes the mechanism instead of the number:
+# grounded_upright_reward drops to 0, and uprightness becomes a multiplier
+# inside the nose-led approach term (ship_ai_controller.gd), which already
+# requires moving forward at the ball. Upright now pays only *while*
+# playing, so parked-and-upright and fast-but-sideways both pay zero and
+# only all three behaviours together pay full.
+#
+# forward-velocity-to-ball rises 0.06 -> 0.15 because multiplying by
+# uprightness cuts that term's expected per-tick value roughly 2-3x at
+# current behaviour; without the raise the approach incentive would quietly
+# weaken. ground-tilt-penalty drops 0.05 -> 0.02: it was the term most
+# likely to teach floor-avoidance, and with uprightness now positively paid
+# during play it only needs to remain a guardrail against travelling on the
+# roof rather than carry the whole posture signal. non-forward-penalty is
+# unchanged at 0.04 — it targets a specific behaviour and has not misfired.
 HANDLING_REWARD_FLAGS = [
     "--velocity-to-ball-weight", "0.04",
-    "--forward-velocity-to-ball-weight", "0.06",
+    "--forward-velocity-to-ball-weight", "0.15",
     "--ball-distance-penalty", "0.01",
     "--ball-touch-reward", "0.7",
     "--ball-velocity-to-goal-weight", "0.06",
     "--goal-reward", "80",
     "--speed-reward-weight", "0.0",
     "--tilt-penalty", "0.0002",
-    "--ground-tilt-penalty", "0.05",
+    "--ground-tilt-penalty", "0.02",
     "--non-forward-penalty", "0.04",
-    "--grounded-upright-reward", "0.004",
+    "--grounded-upright-reward", "0.0",
 ]
 
 STAGES = [