fix(training): correct non-forward penalty math and add a grounding incentive

Adversarial review of the previous stage-4 retune found two problems:
non_forward_speed used planar_speed - forward_component, which under-charges
diagonal motion relative to true lateral speed (e.g. ~29% penalty at 45
degrees off the nose instead of the correct ~71%); fixed to the Pythagorean
magnitude for forward-facing angles, full speed for backward-facing ones.

Also, ground_tilt_penalty and non_forward_penalty only ever cost reward near
the floor with nothing offsetting them above it, which could teach a policy
that's still bad at ground handling to just avoid the floor rather than get
better at it. Added grounded_upright_reward (ship_ai_controller.gd) plus a
new ShipObservations.is_floor_contact helper for genuine belly-on-floor
contact detection, so grounding well while upright is the locally profitable
choice, not just the least-punished one.
This commit is contained in:
Josh Creek
2026-08-09 13:23:00 +01:00
parent c56f5ed1a3
commit 6f7536f03c
5 changed files with 71 additions and 5 deletions
+3 -2
View File
@@ -251,8 +251,8 @@ const SHIP_AI_OVERRIDES := [
"ball_touch_reward", "ball_touch_cooldown_ticks", "ball_touch_direction_floor",
"velocity_to_ball_weight", "ball_velocity_to_goal_weight", "ball_distance_penalty",
"forward_velocity_to_ball_weight", "wall_contact_penalty", "tilt_penalty",
"ground_tilt_penalty", "non_forward_penalty", "speed_reward_weight", "time_penalty",
"airborne_penalty",
"ground_tilt_penalty", "non_forward_penalty", "grounded_upright_reward",
"speed_reward_weight", "time_penalty", "airborne_penalty",
]
@@ -313,6 +313,7 @@ func _ai_default(name: String) -> Variant:
"tilt_penalty": return 0.0005
"ground_tilt_penalty": return 0.0
"non_forward_penalty": return 0.0
"grounded_upright_reward": return 0.0
"speed_reward_weight": return 0.004
"time_penalty": return 0.001
"airborne_penalty": return 0.0