Generation 2's single "unmask" stage (flip vertical/pitch-roll locomotion
from grounded-only to full 3D in one step) failed 3 independent 240M-step
attempts, landing at a stable 32% / 28% / 31% win rate vs curric-s5-aggression
each time -- not noise, and not fixable by more training time (attempts 2-3
each continued the same checkpoint lineage for another full 240M steps with
zero improvement). Every attempt shows train/std collapsing from ~0.30 to
~0.13-0.15 within the first ~10% of steps and never recovering: the policy
locks the newly-opened axes back down before ever meaningfully exploring
them.
Replaces the boolean allow_vertical/allow_pitch_roll mask on ShipAIController
with float vertical_ramp/pitch_roll_ramp multipliers (0.0-1.0), scaling axis
effect in set_action() instead of gating it outright -- the action space
never changes shape, so checkpoints stay resumable across ramp values. The
single unmask stage in curriculum.py becomes 4: three ungated warmup stages
(25%/50%/75% authority, airborne_penalty ramping in step) that train,
checkpoint, and always advance with no eval gate, then the measured stage at
full authority -- same reference, opponent mode, and 240M budget as the 3
failed attempts, for a direct comparison. Adds a "gated" flag/branch to
main()'s loop for the ungated stages.
This is generation 3 of the curriculum; generation 2's state is archived to
curriculum_state_gen2.json (mirroring the earlier gen1 -> gen2 archival) and
curriculum_state.json resets fresh, since its stage 0 no longer means what it
used to. See TRAINING.md's "Generation 3" section for the full postmortem,
stage table, and the open question about whether scaling action effect in
Godot (which PPO's own entropy/exploration math never sees) actually
addresses the collapse.
Generation 2's first two real stage-1 attempts both independently restarted
from curric-s5-aggression (reset_retry_checkpoint) with identical flags and
landed at 32% and 27% win rate vs the reference -- a real regression either
way, but too much spread between "identical" runs for repeat fresh restarts
to be a controlled test of anything. The first attempt's own trajectory
(ep_rew_mean climbing from -10.86 toward ~0 by the 240M-step cutoff,
briefly touching positive) looked closer to convergence than the second's,
so retries now continue that attempt's own checkpoint for another full
timesteps budget instead of resetting to foundation again.
Drops retry1 and retry2 (checkpoints, logs, exported bots, eval_history
entries) -- retry2 never trained meaningfully before crashing on the
GoalRateCallback bug just fixed, and retry1 was the inferior of the two
real samples. curriculum_state.json rewinds to attempt 1, in_progress, so
the next run resumes 20260726-1904-curric-s1-unmask/final.zip directly.
Re-ran stage-3 (curric-s3-no_draws vs curric-s2-defend) and the missing
stage-4 gate now that the locomotion-mask inference bugfix is in. Both
reverse or contradict the pre-fix bookkeeping: curric-s2-defend (grounded)
beats curric-s3-no_draws 60-26 and curric-s4-mechanics 57-24 when fairly
evaluated, so lifting the locomotion mask in stage 3 was a real regression
in floor play, not the improvement the buggy eval reported.
Adds a stage-5 "aggression" curriculum entry that resumes from stage 2
directly (via new resume_from_experiment/reference_experiment stage-dict
overrides in curriculum.py) instead of compounding the regression through
stages 3-4, keeps the locomotion mask on, and retunes ball-pursuit reward
weights for much more aggressive floor play. Extends train.py with the
three new --velocity-to-ball-weight/--ball-distance-penalty/--ball-touch-reward
flags needed to forward that retune to Godot's existing SHIP_AI_OVERRIDES.
curriculum_state.json and TRAINING.md are corrected/annotated in place
rather than silently rewritten, so the regression stays visible in history.
CosmicClash Training Bot