Compare commits

...

6 Commits

Author SHA1 Message Date
Josh Creek efda6c1a05 chore(training): Resume stage 1 unmask from its own checkpoint, not foundation
Generation 2's first two real stage-1 attempts both independently restarted
from curric-s5-aggression (reset_retry_checkpoint) with identical flags and
landed at 32% and 27% win rate vs the reference -- a real regression either
way, but too much spread between "identical" runs for repeat fresh restarts
to be a controlled test of anything. The first attempt's own trajectory
(ep_rew_mean climbing from -10.86 toward ~0 by the 240M-step cutoff,
briefly touching positive) looked closer to convergence than the second's,
so retries now continue that attempt's own checkpoint for another full
timesteps budget instead of resetting to foundation again.

Drops retry1 and retry2 (checkpoints, logs, exported bots, eval_history
entries) -- retry2 never trained meaningfully before crashing on the
GoalRateCallback bug just fixed, and retry1 was the inferior of the two
real samples. curriculum_state.json rewinds to attempt 1, in_progress, so
the next run resumes 20260726-1904-curric-s1-unmask/final.zip directly.
2026-07-29 08:35:34 +01:00
Josh Creek 259b2adc07 fix(training): Guard GoalRateCallback against a missing goal_scored key
The vendored godot_rl sync bridge (Game/addons/godot_rl_agents/sync.gd,
_training_process) snapshots each agent's info dict once per tick and its
own inline comment already flags that reset-timing path as incomplete
("NEEDS REFACTOR"); at least one agent's terminal-step info can arrive
without "goal_scored" at all. Indexing it directly crashed a training run
(20260729-0607-curric-s1-unmask-retry2) within minutes of starting. Skip
episodes missing the key instead of crashing training over a
monitoring-only metric.
2026-07-29 08:35:20 +01:00
CosmicClash Training Bot 8bc16ee048 chore(training): curriculum progress after 20260729-0607-curric-s1-unmask-retry2 2026-07-29 06:09:25 +01:00
CosmicClash Training Bot a6c2ed6177 chore(training): Add 20260729-0607-curric-s1-unmask-retry2 checkpoints, logs, and exported policy 2026-07-29 06:08:10 +01:00
CosmicClash Training Bot 0d29fc44e0 chore(training): curriculum progress after 20260728-0031-curric-s1-unmask-retry1 2026-07-29 06:07:26 +01:00
CosmicClash Training Bot cb3732fcb7 chore(training): Add 20260728-0031-curric-s1-unmask-retry1 checkpoints, logs, and exported policy 2026-07-29 06:05:33 +01:00
2 changed files with 24 additions and 6 deletions
+15 -4
View File
@@ -110,9 +110,7 @@ STAGES = [
# and the new weights were too small to compete with the unchanged
# ball-pursuit terms. Generation 2's stage 1 instead:
# - resumes from FOUNDATION_EXPERIMENT (curric-s5-aggression)
# directly (resume_from_experiment below, plus
# reset_retry_checkpoint so this stage's own retries reset here
# too instead of drifting a failed attempt forward).
# directly (resume_from_experiment below) for the first attempt.
# - raises velocity_to_ball_weight and ball_distance_penalty
# further (the actual ball-chasing terms, unchanged since stage
# 5 despite three failed attempts) and ball_touch_reward
@@ -134,11 +132,24 @@ STAGES = [
"--goal-reward", "80", # up from 60 (40 default)
"--draw-penalty", "5",
],
# 2026-07-29: generation 2's own first two attempts (both independently
# resumed from FOUNDATION_EXPERIMENT under reset_retry_checkpoint,
# identical flags/budget) landed at 32% and 27% win rate — a real
# regression either way, but with enough run-to-run spread that
# "identical fresh restart" isn't a controlled test of anything. The
# first attempt's own trajectory (ep_rew_mean climbing from -10.86
# toward ~0 by the 240M-step cutoff, briefly touching positive) looked
# closer to convergence than the second's, so rather than another
# independent restart from foundation, retries now continue *that*
# attempt's own checkpoint for another full timesteps budget — an
# actual test of "did it just need more time," not another coin flip.
# A third, unrelated attempt crashed immediately (see train.py's
# GoalRateCallback KeyError fix) before contributing any real signal
# and was discarded rather than counted.
"grounded": False,
"timesteps": 240_000_000, # ~24h at the standing n-parallel/speedup (20M took ~2h)
"resume_from_experiment": FOUNDATION_EXPERIMENT,
"reference_experiment": FOUNDATION_EXPERIMENT,
"reset_retry_checkpoint": True,
},
]
+9 -2
View File
@@ -43,8 +43,15 @@ class GoalRateCallback(BaseCallback):
def _on_rollout_end(self) -> None:
if len(self.model.ep_info_buffer) == 0:
return
goal_rate = safe_mean([ep_info["goal_scored"] for ep_info in self.model.ep_info_buffer])
self.logger.record("rollout/goal_rate", goal_rate)
# The vendored godot_rl sync bridge (Game/addons/godot_rl_agents/sync.gd,
# _training_process) snapshots each agent's info dict once per tick and
# has its own "NEEDS REFACTOR" comment on the reset-timing path, so an
# episode's terminal info entry can arrive without "goal_scored" at all
# (observed crashing a run after 2026-07-28). Skip those rather than
# crash training over a monitoring-only metric.
rates = [ep_info["goal_scored"] for ep_info in self.model.ep_info_buffer if "goal_scored" in ep_info]
if rates:
self.logger.record("rollout/goal_rate", safe_mean(rates))
def parse_args():