fix(training): restore ground_tilt_penalty, add grounded_upright_fraction diagnostic

Round 4 changed three things at once and two of them cut upright pressure:
grounded_upright_reward went to 0 and ground_tilt_penalty was cut 2.5x,
while the new uprightness multiplier only pays below GROUND_HANDLING_HEIGHT
*and* while moving forward *and* facing the ball - a far narrower slice of
ticks than the penalty it was meant to replace. Net pressure fell and
upright_fraction fell with it (0.268 -> 0.239 -> 0.238, the lowest of any
round). Restore ground_tilt_penalty to 0.05 and change nothing else, so
this is a genuine single-variable test of multiplier plus full tilt
pressure.

The conjunctive mechanism itself held up: forward_motion_fraction reached
its best sustained value (0.242) without goal_rate sagging, ep_rew_mean
turned positive for the first time (+0.28), and eval win rate hit 49% with
no reward hacking.

Also adds grounded_upright_fraction: a diagnostic, deliberately ungated
metric measuring uprightness over real floor-contact ticks instead of
sub-3m ticks. upright_fraction has never exceeded 0.331 across four rounds
and ~560M steps without cheating, and its denominator is dominated by
ballistic transit (airborne_fraction ~0.45, mean_altitude ~4.4m) where
attitude is not meaningfully controllable - so it likely cannot measure
what the 0.45 floor was meant to capture. Re-baseline that floor from what
this reports rather than from another round of reshaping.
This commit is contained in:
Josh Creek
2026-08-14 19:36:45 +01:00
parent 7da5b8316f
commit 8f7f672a15
13 changed files with 68 additions and 269 deletions
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+27
View File
@@ -204,6 +204,18 @@ var _ground_ticks := 0
var _upright_ground_ticks := 0
var _moving_ground_ticks := 0
var _forward_moving_ground_ticks := 0
# Diagnostic (non-gating) counterpart to _ground_ticks/_upright_ground_ticks.
# Those use altitude (< GROUND_HANDLING_HEIGHT) as a proxy for "on the
# ground", but with airborne_fraction ~0.45 and mean_altitude ~4.4m a large
# share of sub-3m ticks are really ballistic transit — climbing, descending,
# or tumbling after contact — where attitude is neither controllable nor
# meaningful, so upright_fraction systematically understates how upright the
# ship is when it is actually driving. These count only ticks with genuine
# floor contact, which is the thing "keep the belly on the floor" actually
# means. Kept separate from (not a replacement for) upright_fraction so the
# gated metric's definition stays comparable across every past stage.
var _floor_contact_ticks := 0
var _upright_floor_contact_ticks := 0
# Wire up references after the ship is spawned. `attack_goal` is the goal
@@ -264,6 +276,12 @@ func get_info() -> Dictionary:
info["productive_air_touch_fraction"] = float(_productive_air_touches) / _touches if _touches > 0 else 0.0
info["upright_fraction"] = float(_upright_ground_ticks) / _ground_ticks if _ground_ticks > 0 else 0.0
info["forward_motion_fraction"] = float(_forward_moving_ground_ticks) / _moving_ground_ticks if _moving_ground_ticks > 0 else 0.0
# Diagnostic only — deliberately NOT in any stage's telemetry_floors (see
# generation5.py). Unlike the counters above, _floor_contact_ticks can
# legitimately be 0 for a whole episode (a policy that never touches down),
# so the 0.0 default here is load-bearing, not just defensive.
info["grounded_upright_fraction"] = \
float(_upright_floor_contact_ticks) / _floor_contact_ticks if _floor_contact_ticks > 0 else 0.0
return info
@@ -291,6 +309,8 @@ func reset():
_upright_ground_ticks = 0
_moving_ground_ticks = 0
_forward_moving_ground_ticks = 0
_floor_contact_ticks = 0
_upright_floor_contact_ticks = 0
func _physics_process(delta):
@@ -421,6 +441,13 @@ func _physics_process(delta):
_altitude_sum += ship.global_position.y
if ship.global_position.y > AIRBORNE_ALTITUDE_THRESHOLD:
_airborne_ticks += 1
# Diagnostic: uprightness measured only while genuinely touching the floor
# (see _floor_contact_ticks). Same UPRIGHT_DOT_THRESHOLD as the altitude-
# based metric so the two are directly comparable.
if ShipObservations.is_floor_contact(ship):
_floor_contact_ticks += 1
if ship.global_transform.basis.y.dot(Vector3.UP) >= UPRIGHT_DOT_THRESHOLD:
_upright_floor_contact_ticks += 1
_thrust_y_sum += rl_controller.action.thrust.y
if ship.global_position.y < GROUND_HANDLING_HEIGHT:
_ground_ticks += 1