jcreek/CosmicClash
1
0
Fork 0
mirror of https://github.com/jcreek/CosmicClash.git synced 2026-09-11 00:14:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(training): cut grounded_upright_reward, restart stage-4 from Stage-3 foundation

Browse Source 
grounded_upright_reward at 0.015 overshot: four force-retries pushed
upright_fraction from 0.265 to a plateauing 0.331, then the fifth jumped it
to 0.696 (55% over the 0.45 floor) while goal_rate collapsed 0.542->0.366
and forward_motion_fraction fell 0.244->0.184 (vertical_thrust_mean went
negative) - the policy learned to sit pinned upright and farm the bonus
instead of chasing the ball. It was sized "comparable to
time_penalty/ball_distance_penalty" but at 0.015/tick it was actually above
ball_distance_penalty's 0.01/tick worst case, so idling near the ball beat
playing. Cut to 0.004/tick (episode ceiling ~7.2, below
ball_distance_penalty's ~18 worst case). Delete the five blocked attempts
and reset generation5_state.json so the next run starts fresh from the
Stage-3 foundation rather than continuing from the farming checkpoint.
This commit is contained in:
Josh Creek
2026-08-11 15:57:27 +01:00
parent f5b0a79cea
commit b8e2a7b57a
18 changed files with 29 additions and 439 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Game/scripts/ship_ai_controller.gd
+8 -5
View File
@@ -87,11 +87,14 @@ extends AIController3D
# policy that's still bad at ground handling could "solve" those penalties
# by just avoiding the floor rather than by getting better at handling on
# it — worsening Stage 3's already-airborne-heavy baseline instead of
# fixing it. Kept an order of magnitude below ball_touch_reward/goal_reward
# and comparable to time_penalty/ball_distance_penalty so grounding well is
# attractive without making idling upright on the spot, away from the ball,
# competitive with actually playing (see ball_distance_penalty's run04
# lesson on why a flat positional bonus needs a countervailing cost).
# fixing it. An initial 0.015 overshot this: it's a *guaranteed* per-tick
# reward, so it needs to stay below ball_distance_penalty's worst case
# (idling at the arena's far corner), not just "comparable" to it — at
# 0.015 (above ball_distance_penalty's 0.01 ceiling) a Stage-4 run
# converged on sitting pinned upright and farming this instead of chasing
# the ball, cratering goal_rate. Keep this term's episode-long ceiling
# (value * ~1800 ticks) below ball_distance_penalty's worst-case episode
# cost, not just below ball_touch_reward/goal_reward.
@export var grounded_upright_reward := 0.0
# Per-tick bonus for own speed: 0 stationary, full value (+0.24/s) at
# max_speed. Run07 lesson: after the kickoff flurry both ships parked next to