fix(physics): make upright a real state, and actually start ships on the floor

Six rounds of reward shaping (~700M steps) failed to produce upright ground
driving. A critical review of the simulation rather than the reward found
why:

1. The hull was a 1x1x4 box with inertia (1,1,1) and no restoring torque
   anywhere, so belly-down and rolled-90 were geometrically identical
   resting states. "Upright" was not a physically distinguished state at
   all - the reward was paying for a property the simulation did not have.
2. ~65% of episodes spawned ships via _random_position, which samples Y
   uniformly over the full 18m volume (mean ~8.7m). The measured
   airborne_fraction ~0.44 was largely that spawn distribution, and every
   ground-handling term fades out above 3m, so the shaping being tuned
   barely ever applied.
3. air_drill_chance 0.20 spawned deliberately unreachable-without-climbing
   states in the stage meant to teach ground driving, and its own
   air_touch_fraction (0.0002) shows the drills were never solved.

Fixes land in the physics and the task distribution, not the reward:
- ship.tscn: hull 1x1x4 -> 1.6x0.6x4 so it has one stable resting face;
  inertia (1,1,1) -> (7,1,7), physically correct for the hull, making
  tumbling reluctant while keeping yaw snappy.
- ship.gd: new altitude-faded righting torque (spring-damper toward
  belly-down, faded out by 3m so aerials keep full attitude freedom).
  This is the grav-plating analogue of Rocket League's auto-righting and
  helps human pilots land cleanly too.
- training_mode.gd: new ground_start_chance branch spawning ships level and
  resting on the floor with a floor-level ball - the state the handling
  stage's rewards are actually written for.
- generation5.py: ground-start-chance 0.50, air-drill-chance 0.20 -> 0.0.

Reward terms are left exactly as they were; they should finally pull in a
direction the ship can go.
This commit is contained in:
Josh Creek
2026-08-16 08:17:16 +01:00
parent 033a774c92
commit ea756bd5ba
15 changed files with 146 additions and 270 deletions
+63 -4
View File
@@ -69,6 +69,15 @@ extends GameMode
# a real goal and ships start low behind/lateral to it, so a useful touch is
# naturally reinforced by the existing goal-directed ball rewards.
@export_range(0.0, 1.0) var air_intercept_chance := 0.0
# Ground-start branch for the generation-5 handling stage: ships spawn level
# and resting on the floor with a low, floor-level ball. Every other branch
# samples ship Y uniformly across the full 18m volume (see _random_position),
# so ~65% of episodes previously began at a mean altitude near 8.7m — the
# measured airborne_fraction ~0.44 was largely that spawn distribution rather
# than a policy preference, and the ground-handling reward terms (which all
# fade out above 3m) barely ever applied. A stage that means to teach driving
# has to actually start the ship on the ground.
@export_range(0.0, 1.0) var ground_start_chance := 0.0
# Ships per team. Default 1 preserves every existing curriculum script's 1v1
# behaviour unchanged; up to 5 matches ShipObservations.MAX_TEAMMATES/
@@ -77,13 +86,19 @@ extends GameMode
@export_range(1, 5) var team_size: int = 1
# Placement bounds for randomized episode starts, derived from the standard
# enclosure (ArenaBoundary). The inset keeps a randomly oriented ship (1x1x4
# box, worst-case half-extent ~2.05) from spawning intersecting the walls,
# ceiling, or goal sensors.
# enclosure (ArenaBoundary). The inset keeps a randomly oriented ship
# (1.6x0.6x4 box, worst-case half-extent ~2.18) from spawning intersecting
# the walls, ceiling, or goal sensors.
const SPAWN_INSET := 2.5
const FIELD_HALF_X := ArenaBoundary.INNER_HALF_X - SPAWN_INSET
const FIELD_HALF_Z := ArenaBoundary.GOAL_LINE_Z - SPAWN_INSET
const FIELD_MIN_Y := 1.5
# Resting heights for the ground-start branch: half the ship hull's 0.6 height
# and the ball's 0.5 radius (Godot's SphereShape3D default, see ball.tscn),
# each plus a little clearance so bodies settle onto the floor instead of
# spawning interpenetrated with it.
const GROUND_START_Y := 0.35
const GROUND_START_BALL_Y := 0.55
const FIELD_MAX_Y := ArenaBoundary.INNER_HEIGHT - SPAWN_INSET
# The corner curves reach at most their chord plane |x| + |z| = INNER_HALF_X
# + INNER_HALF_Z - CORNER_RADIUS; spawns keep the same SPAWN_INSET clearance
@@ -243,7 +258,7 @@ func _parse_eval_args() -> void:
const TRAINING_MODE_OVERRIDES := [
"goal_reward", "draw_penalty", "kickoff_state_chance",
"ball_near_goal_chance", "attack_goal_bias", "air_drill_chance",
"air_intercept_chance", "team_size",
"air_intercept_chance", "ground_start_chance", "team_size",
]
# ShipAIController @export names a curriculum run may override, read as
# --ai_<name>=<value> to avoid colliding with the names above.
@@ -273,6 +288,7 @@ func _parse_curriculum_args() -> void:
set(name, _typed_like(args[name], get(name)))
var start_probability := kickoff_state_chance + ball_near_goal_chance \
+ air_drill_chance + air_intercept_chance
start_probability += ground_start_chance
if start_probability > 1.0:
push_error("TrainingMode: episode-start probabilities sum to %.3f (> 1.0)" % start_probability)
@@ -428,6 +444,9 @@ func _reset_episode() -> void:
_place_air_drill()
elif roll < kickoff_state_chance + ball_near_goal_chance + air_drill_chance + air_intercept_chance:
_place_air_intercept()
elif roll < kickoff_state_chance + ball_near_goal_chance + air_drill_chance \
+ air_intercept_chance + ground_start_chance:
_place_ground_start()
else:
_place_ships_random()
_place_ball_random()
@@ -497,6 +516,46 @@ func _place_air_drill() -> void:
_place_body(ship, Transform3D(orientation, ship_position), Vector3.ZERO, Vector3.ZERO)
# Ground start (see ground_start_chance): ships resting level on the floor,
# yaw-only so they begin belly-down rather than needing to recover attitude
# first, and a floor-level ball rolling slowly. This is the state the
# handling stage's rewards are actually written for — every ground term
# (ground_tilt_penalty, non_forward_penalty, the nose-led approach bonus)
# fades out by GROUND_HANDLING_HEIGHT, so they only bite in states like this
# one. The ball gets a modest planar-only velocity so it stays reachable
# without a climb.
func _place_ground_start() -> void:
var ball_velocity := _random_direction()
ball_velocity.y = 0.0
ball_velocity = ball_velocity.normalized() * randf_range(0.0, MAX_RANDOM_BALL_SPEED * 0.5)
var ball_position := Vector3(
randf_range(-FIELD_HALF_X, FIELD_HALF_X),
GROUND_START_BALL_Y,
randf_range(-FIELD_HALF_Z, FIELD_HALF_Z)
)
_place_body(ball, Transform3D(Basis.IDENTITY, ball_position), ball_velocity, Vector3.ZERO)
var placed: Array[Vector3] = []
for ship in ships:
if ship in _inert_ships:
continue
var ship_position := Vector3.ZERO
for _attempt in 20:
ship_position = Vector3(
randf_range(-FIELD_HALF_X, FIELD_HALF_X),
GROUND_START_Y,
randf_range(-FIELD_HALF_Z, FIELD_HALF_Z)
)
if _spawn_position_clear(ship_position) and _far_enough_from(ship_position, placed):
break
placed.append(ship_position)
var yaw := randf_range(-PI, PI)
_place_body(
ship, Transform3D(Basis.from_euler(Vector3(0.0, yaw, 0.0)), ship_position),
Vector3.ZERO, Vector3.ZERO
)
# Goal-relevant aerial intercept: a high ball is already travelling toward a
# randomly selected goal, while ships begin low and behind/lateral to its
# path. The generous wall clearance prevents rebound farming and an upright