Stage 5 (aggression) passed (41-47 vs grounded curric-s2-defend, within
the lenient gate but not yet a clear win). Rather than keep the locomotion
mask on indefinitely, stage 6 reopens full 3D controls on top of the
aggression retune and pairs it with a new dense airborne_penalty (scaled
by height above the floor) so the policy learns to prefer staying grounded
through incentives instead of a hard mask — same regime shift that
regressed stage 3, but this time with a mitigation and ~12x the training
time (~240M timesteps / ~24h vs ~20M / ~2h) to actually re-converge
instead of stalling mid-shift.
airborne_penalty follows the existing SHIP_AI_OVERRIDES pattern: default
0 (off) on ship_ai_controller.gd, exposed via train.py's new
--airborne-penalty flag, added to training_mode.gd's allow-list. Also adds
a per-stage timesteps override in curriculum.py (STAGES[n]["timesteps"])
since this is the first stage to need a different budget than the rest.
Re-ran stage-3 (curric-s3-no_draws vs curric-s2-defend) and the missing
stage-4 gate now that the locomotion-mask inference bugfix is in. Both
reverse or contradict the pre-fix bookkeeping: curric-s2-defend (grounded)
beats curric-s3-no_draws 60-26 and curric-s4-mechanics 57-24 when fairly
evaluated, so lifting the locomotion mask in stage 3 was a real regression
in floor play, not the improvement the buggy eval reported.
Adds a stage-5 "aggression" curriculum entry that resumes from stage 2
directly (via new resume_from_experiment/reference_experiment stage-dict
overrides in curriculum.py) instead of compounding the regression through
stages 3-4, keeps the locomotion mask on, and retunes ball-pursuit reward
weights for much more aggressive floor play. Extends train.py with the
three new --velocity-to-ball-weight/--ball-distance-penalty/--ball-touch-reward
flags needed to forward that retune to Godot's existing SHIP_AI_OVERRIDES.
curriculum_state.json and TRAINING.md are corrected/annotated in place
rather than silently rewritten, so the regression stays visible in history.
AIShipController (eval + real gameplay) ran the raw policy output unmasked
regardless of allow_vertical/allow_pitch_roll, while ShipAIController
(training) correctly discarded those axes for grounded curriculum stages.
A grounded-trained model's untrained vertical/pitch-roll output reached the
ship as noise during eval, understating it against models that were never
handicapped this way.
Josh Creek