fix(training): restore ground_tilt_penalty, add grounded_upright_fraction diagnostic

Round 4 changed three things at once and two of them cut upright pressure:
grounded_upright_reward went to 0 and ground_tilt_penalty was cut 2.5x,
while the new uprightness multiplier only pays below GROUND_HANDLING_HEIGHT
*and* while moving forward *and* facing the ball - a far narrower slice of
ticks than the penalty it was meant to replace. Net pressure fell and
upright_fraction fell with it (0.268 -> 0.239 -> 0.238, the lowest of any
round). Restore ground_tilt_penalty to 0.05 and change nothing else, so
this is a genuine single-variable test of multiplier plus full tilt
pressure.

The conjunctive mechanism itself held up: forward_motion_fraction reached
its best sustained value (0.242) without goal_rate sagging, ep_rew_mean
turned positive for the first time (+0.28), and eval win rate hit 49% with
no reward hacking.

Also adds grounded_upright_fraction: a diagnostic, deliberately ungated
metric measuring uprightness over real floor-contact ticks instead of
sub-3m ticks. upright_fraction has never exceeded 0.331 across four rounds
and ~560M steps without cheating, and its denominator is dominated by
ballistic transit (airborne_fraction ~0.45, mean_altitude ~4.4m) where
attitude is not meaningfully controllable - so it likely cannot measure
what the 0.45 floor was meant to capture. Re-baseline that floor from what
this reports rather than from another round of reshaping.
This commit is contained in:
Josh Creek
2026-08-14 19:36:45 +01:00
parent 7da5b8316f
commit 8f7f672a15
13 changed files with 68 additions and 269 deletions
+27
View File
@@ -204,6 +204,18 @@ var _ground_ticks := 0
var _upright_ground_ticks := 0
var _moving_ground_ticks := 0
var _forward_moving_ground_ticks := 0
# Diagnostic (non-gating) counterpart to _ground_ticks/_upright_ground_ticks.
# Those use altitude (< GROUND_HANDLING_HEIGHT) as a proxy for "on the
# ground", but with airborne_fraction ~0.45 and mean_altitude ~4.4m a large
# share of sub-3m ticks are really ballistic transit — climbing, descending,
# or tumbling after contact — where attitude is neither controllable nor
# meaningful, so upright_fraction systematically understates how upright the
# ship is when it is actually driving. These count only ticks with genuine
# floor contact, which is the thing "keep the belly on the floor" actually
# means. Kept separate from (not a replacement for) upright_fraction so the
# gated metric's definition stays comparable across every past stage.
var _floor_contact_ticks := 0
var _upright_floor_contact_ticks := 0
# Wire up references after the ship is spawned. `attack_goal` is the goal
@@ -264,6 +276,12 @@ func get_info() -> Dictionary:
info["productive_air_touch_fraction"] = float(_productive_air_touches) / _touches if _touches > 0 else 0.0
info["upright_fraction"] = float(_upright_ground_ticks) / _ground_ticks if _ground_ticks > 0 else 0.0
info["forward_motion_fraction"] = float(_forward_moving_ground_ticks) / _moving_ground_ticks if _moving_ground_ticks > 0 else 0.0
# Diagnostic only — deliberately NOT in any stage's telemetry_floors (see
# generation5.py). Unlike the counters above, _floor_contact_ticks can
# legitimately be 0 for a whole episode (a policy that never touches down),
# so the 0.0 default here is load-bearing, not just defensive.
info["grounded_upright_fraction"] = \
float(_upright_floor_contact_ticks) / _floor_contact_ticks if _floor_contact_ticks > 0 else 0.0
return info
@@ -291,6 +309,8 @@ func reset():
_upright_ground_ticks = 0
_moving_ground_ticks = 0
_forward_moving_ground_ticks = 0
_floor_contact_ticks = 0
_upright_floor_contact_ticks = 0
func _physics_process(delta):
@@ -421,6 +441,13 @@ func _physics_process(delta):
_altitude_sum += ship.global_position.y
if ship.global_position.y > AIRBORNE_ALTITUDE_THRESHOLD:
_airborne_ticks += 1
# Diagnostic: uprightness measured only while genuinely touching the floor
# (see _floor_contact_ticks). Same UPRIGHT_DOT_THRESHOLD as the altitude-
# based metric so the two are directly comparable.
if ShipObservations.is_floor_contact(ship):
_floor_contact_ticks += 1
if ship.global_transform.basis.y.dot(Vector3.UP) >= UPRIGHT_DOT_THRESHOLD:
_upright_floor_contact_ticks += 1
_thrust_y_sum += rl_controller.action.thrust.y
if ship.global_position.y < GROUND_HANDLING_HEIGHT:
_ground_ticks += 1