mirror of
https://github.com/jcreek/CosmicClash.git
synced 2026-09-13 12:22:04 +00:00
chore(training): add air_touch_bonus_weight and restart stage-5 intercepts
air_approach_weight alone didn't move productive_air_touch_fraction after a further 180M steps (360M cumulative across all six Stage-5 attempts): an unredirected air-intercept ball falls short of the goal from gravity and just lands on the floor, so the already-solved ground game collects the same episode reward whether or not anything touched the ball in the air. air_touch_bonus_weight adds a conjunctive event bonus on top of ball_touch_reward for a touch that's both genuinely aerial and goal-directed, targeting the actual measured behaviour instead of only the approach to it.
This commit is contained in:
@@ -181,10 +181,40 @@ STANDING_ARGS = ["--ent-coef", "0.01", "--entropy-floor"]
|
||||
# than continuing retry2's, same reasoning as every previous mechanism
|
||||
# change in this file: don't resume a policy shaped by an absent term into
|
||||
# one where it now exists.
|
||||
#
|
||||
# Round 8 (2026-08-19): air_approach_weight alone did not move the needle
|
||||
# either -- another full 180M-step chain (3 more attempts, 360M cumulative
|
||||
# across all six Stage-5 attempts) closed with productive_air_touch_fraction
|
||||
# still exactly 0.0 and air_touch_fraction at noise level, while goal_rate
|
||||
# kept passing its (lower) floor. Working out the physics instead of just
|
||||
# re-tuning a number found why: an unredirected air-intercept ball (spawned
|
||||
# 6-12m up, aimed at a goal whose collision box sits at ~0-1.5m) sags well
|
||||
# short of the goal from gravity alone over the required flight distance --
|
||||
# it does not auto-score -- so it simply falls to the floor, and the
|
||||
# already-solved ground game (forward_velocity_to_ball_weight, ball_touch_
|
||||
# reward, goal_reward) collects the exact same total episode reward either
|
||||
# way. Nothing ever made touching the ball while it was still genuinely
|
||||
# airborne worth more than waiting the second or two for it to land, so
|
||||
# air_approach_weight's dense closing-speed shaping had nothing to reinforce
|
||||
# -- nowhere near a training-duration problem, a second missing-incentive
|
||||
# gap in the same stage.
|
||||
#
|
||||
# air_touch_bonus_weight (ship_ai_controller.gd) closes it directly: an
|
||||
# event bonus on top of ball_touch_reward, paid only for a touch that is
|
||||
# both above AIR_TOUCH_HEIGHT and goal-directed, scaled by the exact same
|
||||
# alignment factor already gating the base touch reward -- conjunctive, not
|
||||
# standalone, so it can't be farmed by batting the ball in a useless
|
||||
# direction, and it targets exactly the behaviour productive_air_touch_
|
||||
# fraction measures instead of only the approach to it. Set to 0.5 (roughly
|
||||
# ball_touch_reward's own magnitude, so a fully-aligned aerial touch pays
|
||||
# ~1.7x a fully-aligned ground one). Also folded into HANDLING_REWARD_FLAGS
|
||||
# so Stage 6 inherits it. Restarts Stage 5 from Stage 4's checkpoint again,
|
||||
# same reasoning as every prior mechanism change here.
|
||||
HANDLING_REWARD_FLAGS = [
|
||||
"--velocity-to-ball-weight", "0.04",
|
||||
"--forward-velocity-to-ball-weight", "0.15",
|
||||
"--air-approach-weight", "0.15",
|
||||
"--air-touch-bonus-weight", "0.5",
|
||||
"--ball-distance-penalty", "0.01",
|
||||
"--ball-touch-reward", "0.7",
|
||||
"--ball-velocity-to-goal-weight", "0.06",
|
||||
|
||||
Reference in New Issue
Block a user