fix(physics): make upright a real state, and actually start ships on the floor

Six rounds of reward shaping (~700M steps) failed to produce upright ground
driving. A critical review of the simulation rather than the reward found
why:

1. The hull was a 1x1x4 box with inertia (1,1,1) and no restoring torque
   anywhere, so belly-down and rolled-90 were geometrically identical
   resting states. "Upright" was not a physically distinguished state at
   all - the reward was paying for a property the simulation did not have.
2. ~65% of episodes spawned ships via _random_position, which samples Y
   uniformly over the full 18m volume (mean ~8.7m). The measured
   airborne_fraction ~0.44 was largely that spawn distribution, and every
   ground-handling term fades out above 3m, so the shaping being tuned
   barely ever applied.
3. air_drill_chance 0.20 spawned deliberately unreachable-without-climbing
   states in the stage meant to teach ground driving, and its own
   air_touch_fraction (0.0002) shows the drills were never solved.

Fixes land in the physics and the task distribution, not the reward:
- ship.tscn: hull 1x1x4 -> 1.6x0.6x4 so it has one stable resting face;
  inertia (1,1,1) -> (7,1,7), physically correct for the hull, making
  tumbling reluctant while keeping yaw snappy.
- ship.gd: new altitude-faded righting torque (spring-damper toward
  belly-down, faded out by 3m so aerials keep full attitude freedom).
  This is the grav-plating analogue of Rocket League's auto-righting and
  helps human pilots land cleanly too.
- training_mode.gd: new ground_start_chance branch spawning ships level and
  resting on the floor with a floor-level ball - the state the handling
  stage's rewards are actually written for.
- generation5.py: ground-start-chance 0.50, air-drill-chance 0.20 -> 0.0.

Reward terms are left exactly as they were; they should finally pull in a
direction the ship can go.
This commit is contained in:
Josh Creek
2026-08-16 08:17:16 +01:00
parent 033a774c92
commit ea756bd5ba
15 changed files with 146 additions and 270 deletions
+31 -1
View File
@@ -122,6 +122,35 @@ STANDING_ARGS = ["--ent-coef", "0.01", "--entropy-floor"]
# so it likely cannot measure the behaviour the 0.45 floor was meant to
# capture. Re-baseline that floor from what the new signal reports rather
# than from another round of reshaping.
#
# Round 6 (2026-08-16): round 5 read grounded_upright_fraction 0.050 /
# 0.069 / 0.052 — when the ship touches the floor it is upright about 1
# time in 17 — and the user's own observation was "it spends the vast
# majority of the time on its side, driving upwards towards the ball". A
# critical review of the *simulation* rather than the reward found why six
# rounds of shaping could never work:
#
# 1. The hull was a 1x1x4 box with inertia (1,1,1) and no restoring
# torque anywhere, so belly-down and rolled-90 were geometrically
# identical resting states. "Upright" was not a physically
# distinguished state at all — the reward was paying for a property
# the simulation did not have.
# 2. ~65% of episodes spawned ships from _random_position, which samples
# Y uniformly over the full 18m volume (mean ~8.7m). The measured
# airborne_fraction ~0.44 was largely that spawn distribution, and
# every ground-handling term fades out above 3m, so the shaping
# being tuned barely ever applied.
# 3. air_drill_chance 0.20 spawned deliberately unreachable-without-
# climbing states in the stage meant to teach ground driving, and its
# own air_touch_fraction (0.0002) shows the drills were never solved.
#
# Fixes land in the physics and the task distribution instead of the
# reward: an altitude-faded righting torque plus a flat-bottomed hull and
# realistic inertia (ship.gd / ship.tscn) make belly-down a genuine
# attractor, ground_start_chance 0.50 actually starts the ship on the
# floor, and air-drill-chance goes to 0. The reward terms already built
# are left exactly as they were — they should finally pull in a direction
# the ship can go.
HANDLING_REWARD_FLAGS = [
"--velocity-to-ball-weight", "0.04",
"--forward-velocity-to-ball-weight", "0.15",
@@ -145,8 +174,9 @@ STAGES = [
"--opponent-mode", "self_play",
"--kickoff-chance", "0.15",
"--near-goal-chance", "0.25",
"--air-drill-chance", "0.20",
"--air-drill-chance", "0.0",
"--air-intercept-chance", "0.0",
"--ground-start-chance", "0.50",
*HANDLING_REWARD_FLAGS,
],
# Conservative catastrophe floors, not claims of mastery. Tail values