chore(training): retune stage-4 handling penalties and restart from Stage-3 foundation

Stage 4's upright/forward-motion telemetry plateaued flat across all three
blocked attempts because ground_tilt_penalty (0.003) was too weak to matter
and nothing penalized sideways/reverse motion at all. Raise
ground_tilt_penalty to 0.05 and add a new non_forward_penalty term
(ship_ai_controller.gd) that directly costs non-forward planar velocity near
the floor, independent of the ball. Delete the three blocked attempts'
checkpoints/logs/exports and reset generation5_state.json so the next run
starts fresh from the Stage-3 foundation checkpoint instead of continuing
from the drifted retry2 weights.
This commit is contained in:
Josh Creek
2026-08-09 13:09:12 +01:00
parent 005cd0c66e
commit c56f5ed1a3
14 changed files with 41 additions and 263 deletions
+11 -2
View File
@@ -43,7 +43,15 @@ STANDING_ARGS = ["--ent-coef", "0.01", "--entropy-floor"]
# Scoring/ball-direction shaping inherited from generation 4. Handling
# replaces half the orientation-agnostic closing reward and all generic speed
# reward with nose-led ground approach, while keeping global tilt pressure
# small enough for flight and adding a stronger floor-local term.
# small enough for flight. The first three Stage-4 attempts (2026-08-08/09)
# plateaued with upright_fraction/forward_motion_fraction flat at ~0.22-0.26
# against 0.45/0.25 floors for 120M cumulative timesteps: ground_tilt_penalty
# at 0.003 only cost a fully-sideways episode ~2.7 reward, trivial next to a
# goal (80) or a touch (0.7). ground_tilt_penalty is raised ~17x to 0.05 (a
# full sideways episode now costs ~45, comparable to a goal) and
# non_forward_penalty is a new term (ship_ai_controller.gd) directly costing
# sideways/reverse planar velocity near the floor, independent of the ball,
# since nothing previously penalized that at all.
HANDLING_REWARD_FLAGS = [
"--velocity-to-ball-weight", "0.04",
"--forward-velocity-to-ball-weight", "0.06",
@@ -53,7 +61,8 @@ HANDLING_REWARD_FLAGS = [
"--goal-reward", "80",
"--speed-reward-weight", "0.0",
"--tilt-penalty", "0.0002",
"--ground-tilt-penalty", "0.003",
"--ground-tilt-penalty", "0.05",
"--non-forward-penalty", "0.04",
]
STAGES = [