fix(training): make uprightness a multiplier on the forward-approach reward

Browse Source

Rounds 2 and 3 showed that tuning grounded_upright_reward's magnitude only
slides along a tradeoff instead of resolving it: at 0.015 upright_fraction
climbed to 0.331 while goal_rate sagged to 0.542 (then farmed outright at
0.696/0.366), and at 0.004 goal_rate climbed 0.569->0.604 while
upright_fraction went flat at ~0.26. An additive uprightness bonus is an
alternative to playing well, so the policy just picks whichever is cheaper
and no magnitude buys both behaviours.

Change the mechanism rather than the number: grounded_upright_reward drops
to 0, and uprightness becomes a multiplier inside the nose-led approach
term, which already requires moving forward at the ball. Parked-and-upright
and fast-but-sideways now both pay zero; only upright, forward, nose-on to
the ball pays full. forward-velocity-to-ball rises 0.06 -> 0.15 to offset
the ~2-3x expected-value cut from the new factor, and ground-tilt-penalty
drops 0.05 -> 0.02 now that uprightness is paid positively during play.
Delete the three blocked attempts and reset state to restart from the
Stage-3 foundation.

...

This commit is contained in:

Josh Creek

2026-08-12 13:04:20 +01:00

parent 5260124274

commit 4f3cf56e28

12 changed files with 62 additions and 267 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

Game/bots/20260811-1606-gen5-s4-handling.json

-1

File diff suppressed because one or more lines are too long

Game/bots/20260811-2216-gen5-s4-handling-retry1.json

-1

File diff suppressed because one or more lines are too long

Game/bots/20260812-0424-gen5-s4-handling-retry2.json

-1

File diff suppressed because one or more lines are too long