fix(training): cut grounded_upright_reward, restart stage-4 from Stage-3 foundation

Browse Source

grounded_upright_reward at 0.015 overshot: four force-retries pushed
upright_fraction from 0.265 to a plateauing 0.331, then the fifth jumped it
to 0.696 (55% over the 0.45 floor) while goal_rate collapsed 0.542->0.366
and forward_motion_fraction fell 0.244->0.184 (vertical_thrust_mean went
negative) - the policy learned to sit pinned upright and farm the bonus
instead of chasing the ball. It was sized "comparable to
time_penalty/ball_distance_penalty" but at 0.015/tick it was actually above
ball_distance_penalty's 0.01/tick worst case, so idling near the ball beat
playing. Cut to 0.004/tick (episode ceiling ~7.2, below
ball_distance_penalty's ~18 worst case). Delete the five blocked attempts
and reset generation5_state.json so the next run starts fresh from the
Stage-3 foundation rather than continuing from the farming checkpoint.

...

This commit is contained in:

Josh Creek

2026-08-11 15:57:27 +01:00

parent f5b0a79cea

commit b8e2a7b57a

18 changed files with 29 additions and 439 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

Game/bots/20260809-1340-gen5-s4-handling.json

-1

File diff suppressed because one or more lines are too long

Game/bots/20260809-1955-gen5-s4-handling-retry1.json

-1

File diff suppressed because one or more lines are too long

Game/bots/20260810-0211-gen5-s4-handling-retry2.json

-1

File diff suppressed because one or more lines are too long

Game/bots/20260810-1338-gen5-s4-handling-retry3.json

-1

File diff suppressed because one or more lines are too long

Game/bots/20260811-0858-gen5-s4-handling-retry4.json

-1

File diff suppressed because one or more lines are too long