mirror of
https://github.com/jcreek/CosmicClash.git
synced 2026-09-11 08:23:45 +00:00
fix(training): restore ground_tilt_penalty, add grounded_upright_fraction diagnostic
Round 4 changed three things at once and two of them cut upright pressure: grounded_upright_reward went to 0 and ground_tilt_penalty was cut 2.5x, while the new uprightness multiplier only pays below GROUND_HANDLING_HEIGHT *and* while moving forward *and* facing the ball - a far narrower slice of ticks than the penalty it was meant to replace. Net pressure fell and upright_fraction fell with it (0.268 -> 0.239 -> 0.238, the lowest of any round). Restore ground_tilt_penalty to 0.05 and change nothing else, so this is a genuine single-variable test of multiplier plus full tilt pressure. The conjunctive mechanism itself held up: forward_motion_fraction reached its best sustained value (0.242) without goal_rate sagging, ep_rew_mean turned positive for the first time (+0.28), and eval win rate hit 49% with no reward hacking. Also adds grounded_upright_fraction: a diagnostic, deliberately ungated metric measuring uprightness over real floor-contact ticks instead of sub-3m ticks. upright_fraction has never exceeded 0.331 across four rounds and ~560M steps without cheating, and its denominator is dominated by ballistic transit (airborne_fraction ~0.45, mean_altitude ~4.4m) where attitude is not meaningfully controllable - so it likely cannot measure what the 0.45 floor was meant to capture. Re-baseline that floor from what this reports rather than from another round of reshaping.
This commit is contained in:
+28
-6
@@ -95,11 +95,33 @@ STANDING_ARGS = ["--ent-coef", "0.01", "--entropy-floor"]
|
||||
# forward-velocity-to-ball rises 0.06 -> 0.15 because multiplying by
|
||||
# uprightness cuts that term's expected per-tick value roughly 2-3x at
|
||||
# current behaviour; without the raise the approach incentive would quietly
|
||||
# weaken. ground-tilt-penalty drops 0.05 -> 0.02: it was the term most
|
||||
# likely to teach floor-avoidance, and with uprightness now positively paid
|
||||
# during play it only needs to remain a guardrail against travelling on the
|
||||
# roof rather than carry the whole posture signal. non-forward-penalty is
|
||||
# unchanged at 0.04 — it targets a specific behaviour and has not misfired.
|
||||
# weaken. non-forward-penalty is unchanged at 0.04 — it targets a specific
|
||||
# behaviour and has not misfired.
|
||||
#
|
||||
# Round 5 (2026-08-14): round 4 also dropped ground-tilt-penalty 0.05 ->
|
||||
# 0.02 on the theory that the multiplier could carry posture on its own.
|
||||
# That confounded the experiment — two of the three changes *reduced*
|
||||
# upright pressure at once (grounded_upright_reward to 0, tilt penalty cut
|
||||
# 2.5x) while the multiplier only pays below GROUND_HANDLING_HEIGHT *and*
|
||||
# while moving forward *and* facing the ball, i.e. a far narrower slice of
|
||||
# ticks than the penalty it replaced. Net pressure fell and so did
|
||||
# upright_fraction (0.268 -> 0.239 -> 0.238, the lowest of any round). The
|
||||
# conjunctive part worked though: forward_motion_fraction reached its best
|
||||
# sustained value (0.242) *without* goal_rate sagging, ep_rew_mean turned
|
||||
# positive for the first time (+0.28), and the eval win rate hit 49% with
|
||||
# no reward hacking. So round 5 restores ground-tilt-penalty to 0.05 and
|
||||
# changes nothing else — a genuine single-variable test of multiplier plus
|
||||
# full tilt pressure.
|
||||
#
|
||||
# Also added this round: grounded_upright_fraction, a *diagnostic, ungated*
|
||||
# telemetry signal measuring uprightness over real floor-contact ticks
|
||||
# rather than sub-3m ticks. upright_fraction has never exceeded 0.331
|
||||
# across four rounds and ~560M steps without the policy cheating, and its
|
||||
# denominator is dominated by ballistic transit (airborne_fraction ~0.45,
|
||||
# mean_altitude ~4.4m) where attitude is not meaningfully controllable —
|
||||
# so it likely cannot measure the behaviour the 0.45 floor was meant to
|
||||
# capture. Re-baseline that floor from what the new signal reports rather
|
||||
# than from another round of reshaping.
|
||||
HANDLING_REWARD_FLAGS = [
|
||||
"--velocity-to-ball-weight", "0.04",
|
||||
"--forward-velocity-to-ball-weight", "0.15",
|
||||
@@ -109,7 +131,7 @@ HANDLING_REWARD_FLAGS = [
|
||||
"--goal-reward", "80",
|
||||
"--speed-reward-weight", "0.0",
|
||||
"--tilt-penalty", "0.0002",
|
||||
"--ground-tilt-penalty", "0.02",
|
||||
"--ground-tilt-penalty", "0.05",
|
||||
"--non-forward-penalty", "0.04",
|
||||
"--grounded-upright-reward", "0.0",
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user