mirror of
https://github.com/jcreek/CosmicClash.git
synced 2026-09-11 08:23:45 +00:00
fix(training): make uprightness a multiplier on the forward-approach reward
Rounds 2 and 3 showed that tuning grounded_upright_reward's magnitude only slides along a tradeoff instead of resolving it: at 0.015 upright_fraction climbed to 0.331 while goal_rate sagged to 0.542 (then farmed outright at 0.696/0.366), and at 0.004 goal_rate climbed 0.569->0.604 while upright_fraction went flat at ~0.26. An additive uprightness bonus is an alternative to playing well, so the policy just picks whichever is cheaper and no magnitude buys both behaviours. Change the mechanism rather than the number: grounded_upright_reward drops to 0, and uprightness becomes a multiplier inside the nose-led approach term, which already requires moving forward at the ball. Parked-and-upright and fast-but-sideways now both pay zero; only upright, forward, nose-on to the ball pays full. forward-velocity-to-ball rises 0.06 -> 0.15 to offset the ~2-3x expected-value cut from the new factor, and ground-tilt-penalty drops 0.05 -> 0.02 now that uprightness is paid positively during play. Delete the three blocked attempts and reset state to restart from the Stage-3 foundation.
This commit is contained in:
@@ -95,6 +95,18 @@ extends AIController3D
|
||||
# the ball, cratering goal_rate. Keep this term's episode-long ceiling
|
||||
# (value * ~1800 ticks) below ball_distance_penalty's worst-case episode
|
||||
# cost, not just below ball_touch_reward/goal_reward.
|
||||
#
|
||||
# SUPERSEDED (2026-08-12), kept at 0 for older curricula that set it: the
|
||||
# magnitude was never the real problem. Retuning it 0.015 -> 0.004 only
|
||||
# moved along a tradeoff — at 0.015 upright_fraction climbed while
|
||||
# goal_rate sagged, at 0.004 goal_rate climbed while upright_fraction went
|
||||
# flat — because an *additive* uprightness reward is an alternative to
|
||||
# playing well, so the policy just picks whichever is cheaper. Uprightness
|
||||
# is now a multiplier inside the forward-approach term below instead, which
|
||||
# makes it conjunctive with (not competing against) moving forward at the
|
||||
# ball. Prefer that pattern for any future posture shaping; only reach for
|
||||
# a standalone additive posture bonus if there is genuinely nothing to
|
||||
# condition it on.
|
||||
@export var grounded_upright_reward := 0.0
|
||||
# Per-tick bonus for own speed: 0 stationary, full value (+0.24/s) at
|
||||
# max_speed. Run07 lesson: after the kickoff flurry both ships parked next to
|
||||
@@ -296,9 +308,22 @@ func _physics_process(delta):
|
||||
var closing_speed := ship.linear_velocity.dot(to_ball.normalized())
|
||||
reward += velocity_to_ball_weight * closing_speed / ship.max_speed
|
||||
|
||||
# Ground-handling shaping: forward planar motion while the nose faces the
|
||||
# ball. It fades out with altitude so an aerial remains free to approach a
|
||||
# ball using whatever body attitude is effective.
|
||||
# Ground-handling shaping: upright, forward planar motion while the nose
|
||||
# faces the ball. It fades out with altitude so an aerial remains free to
|
||||
# approach a ball using whatever body attitude is effective.
|
||||
#
|
||||
# Uprightness is a *multiplier* here rather than a separate additive term,
|
||||
# and that is the whole point. Stage 4's earlier rounds paid uprightness
|
||||
# additively (grounded_upright_reward): because additive terms let a
|
||||
# policy collect whichever one is cheapest, it could either play well
|
||||
# (tilted, scoring) or sit parked upright (still, not scoring) — and it
|
||||
# picked one or the other depending purely on that term's magnitude, so
|
||||
# upright_fraction and goal_rate moved in opposite directions at every
|
||||
# value tried. As a multiplier, uprightness pays only while the ship is
|
||||
# also moving forward and nose-on to the ball, so no subset of the three
|
||||
# behaviours can be farmed in isolation: parked pays zero (forward_speed
|
||||
# is zero), on-its-side pays zero (uprightness is zero), and only doing
|
||||
# all three at once pays full.
|
||||
if forward_velocity_to_ball_weight > 0.0 and ship.global_position.y < GROUND_HANDLING_HEIGHT:
|
||||
var planar_forward := Vector3(-ship.global_transform.basis.z.x, 0.0, -ship.global_transform.basis.z.z)
|
||||
var planar_velocity := Vector3(ship.linear_velocity.x, 0.0, ship.linear_velocity.z)
|
||||
@@ -307,8 +332,10 @@ func _physics_process(delta):
|
||||
planar_forward = planar_forward.normalized()
|
||||
var facing_ball: float = maxf(planar_forward.dot(planar_to_ball.normalized()), 0.0)
|
||||
var forward_speed: float = maxf(planar_velocity.dot(planar_forward), 0.0) / ship.max_speed
|
||||
var approach_uprightness: float = maxf(ship.global_transform.basis.y.dot(Vector3.UP), 0.0)
|
||||
var handling_ground_factor: float = 1.0 - clampf(ship.global_position.y / GROUND_HANDLING_HEIGHT, 0.0, 1.0)
|
||||
reward += forward_velocity_to_ball_weight * forward_speed * facing_ball * handling_ground_factor
|
||||
reward += forward_velocity_to_ball_weight * forward_speed * facing_ball \
|
||||
* approach_uprightness * handling_ground_factor
|
||||
|
||||
# Dense penalty: distance to the ball, so idling far away bleeds reward
|
||||
# instead of scoring a safe zero (see ball_distance_penalty).
|
||||
|
||||
Reference in New Issue
Block a user