fix(multiplayer): server no longer rate-limits a backlog it caused itself

Closes task 5.10's three recording gaps, and the gap-closing found a real
input-loss bug.

Replay log: a failed write now ends the log permanently instead of
desyncing every later record's framing; close() is called from _exit_tree
with a summary, since the RefCounted destructor closes it implicitly but
never says whether the log is complete; rejected packets are recorded
with their reason in the kind byte (framing unchanged, FORMAT_VERSION 2
so "no rejects" differs from "this build never recorded them"). Recording
is capped at 8 per peer per window - uncapped, the diagnostic is a remote
disk-fill amplifier, since the attacker picks the packet rate. Uncapped
totals live on MatchSim and survive the peer's disconnect.

The bug: a 2s host stall has the client sending at 60Hz throughout, and
ENet delivers that whole backlog in the first window after resume - 70 of
an honest client's packets rejected as "rate limit exceeded". Redundancy
does not cover it, because the dropped packets are contiguous: 0 of 70
rescued, and 82 of 923 sequences (8.88%, ~1.4s of input) never reached
the server, against 0.00% with no stall. Every prediction gate passed.

Fixed by granting each already-tracked peer a capped, two-window packet
grace when the server detects its own wall-clock stall. Rate-limit
rejects 70 -> 0, sequences missing 8.88% -> 0.00%, seq-guard rejects
9 -> 0. Controls on the unfixed build lost 4.34/7.52/7.86%. All three
abuse roles still disconnect and no flood induced a stall, so the grace
cannot be farmed.

Also corrects an earlier wrong conclusion: the reviewer's free-flight
p95 0.688 is real and reproduces on two processes with 0.0% snapshot
loss. The plain --role=client drive fails the 0.5 free-flight bound in
3 of 8 runs because that drive is mostly a contact test - the harness
comment already said so - leaving a cohort as small as 12 samples.
Near-surface error is genuinely several times open-air error, so the
calibrated bound now belongs to --exercise-free-flight alone and the
plain role asserts the always-well-sampled all-cohort percentiles at
1.2/2.0, printing the free-flight numbers as reported-not-asserted.
6/6 plain runs pass where 3/7 failed; tightening to 0.3 still fails.

tools/replay_dump.gd reads a log back: counts by kind, plus how much of
the input sequence stream reached the server once redundancy is counted.

multiplayer-todo.md

@@ -1014,7 +1014,25 @@ Verified against a control: hardcoding the snapshot byte back to `0` fails both
 
 **`--check-only --script` is the only thing that catches a parse error in `networked_match.gd`.** The unit runner never loads it, so `bot_model_path` being undefined (and later `ReplayLog` being unregistered) both passed 81/87 unit tests while breaking every two-process run. Validate touched scripts directly. A newly added `class_name` also needs `godot --headless --path Game --import` before anything can resolve it — and the same `--import` is the fix when a *previously working* `class_name` stops resolving, which happens on its own: `.godot/global_script_class_cache.cfg` silently lost `MatchState` between sessions, and every two-process run then died with `Cannot infer the type of "live" variable` at the `MatchState.is_live()` call, with nothing in `git status` to explain it. Read that error as "the class cache is stale", not "the code is wrong".
 
-**The three-process prediction-quality caveat could not be reproduced, and the durable fix was diagnosis, not a code change.** The second adversarial review reported a 3-process run failing the free-flight gate at p95 0.688 (bar 0.5) with roughly a third of snapshots missing. Separating the two candidate causes — a third process merely competing for CPU, versus a spectator that the server must actually serve — showed a **spectator costs a small but real amount and an idle third process costs nothing**: two-process p95/p99 0.084/0.098, idle third process 0.084/0.094, spectator 0.084–0.098 / 0.094–0.146 across four runs. All of that is an order of magnitude inside the 0.5/2.0 gates. Snapshot loss stayed at **0.0% even under deliberate 2x CPU oversubscription** (20 spinners on 10 cores), where the only thing that moved was `snapshot_age` (14ms → 32.3ms) and the run still passed. 0.688 never recurred. What *was* worth keeping is that a percentile alone cannot distinguish "the predictor regressed" from "the client never received the data", so the client gate now prints `snapshot_loss` / `snapshot_age` / `rtt` on every run and, on a quality failure with >20% loss, says explicitly that the run was transport-starved — **without converting the failure into a pass**, because a client that cannot receive snapshots is still a failed run. Both directions of that branch were verified non-vacuously (forced true → it fires and formats; restored → it stays quiet on a healthy run while the INFO line still prints).
+**The reviewer's p95 0.688 was real, and the three-process framing was a red herring — mine as much as the reviewer's.** The report was "a 3-process run failed the free-flight gate at p95 0.688 (bar 0.5) with roughly a third of snapshots missing", so the first investigation compared process counts: two-process p95/p99 0.084/0.098, idle third process 0.084/0.094, spectator 0.084–0.098 / 0.094–0.146 over four runs, and 0.0% snapshot loss even under deliberate 2x CPU oversubscription (20 spinners on 10 cores, where only `snapshot_age` moved, 14ms → 32.3ms). Every one of those runs passed, so the conclusion recorded here was "not reproducible". **That conclusion was wrong, and it was wrong because every probe used `--exercise-free-flight` — the one mode the 0.5 bound was calibrated on.**
+
+It reproduces on *two* processes, on an idle machine, with 0.0% snapshot loss: **the plain `--role=client` drive fails the free-flight gate roughly a third of the time.** Eight plain-role runs measured a free-flight cohort of 12–257 samples with p95 0.275–0.726, failing the 0.5 bound in 3 of 8. The harness's own `_run_free_flight_trace` comment had already said why — "a straight forward trace reaches the goal/wall in seconds and turns the supposed free-flight QA run into a contact test" — but the plain role went on asserting the open-volume bound against whatever free-flight samples that contact-heavy drive happened to leave behind, sometimes as few as 12.
+
+The underlying difference is not noise. Prediction error near the arena's surface-pull field is genuinely several times higher than in open air: the same build measures 0.084–0.111 under `--exercise-free-flight` and 0.275–0.726 on the plain drive. Both are honest numbers about different flight profiles, and one bound cannot serve both. `--exercise-free-flight` keeps the calibrated 0.5/2.0 gate (~5x margin). The plain role now asserts the **all-cohort** percentiles instead — always well-sampled (545–696, versus a free-flight cohort that can collapse to 12) and much tighter in spread (raw_p95 0.354–0.609, raw_p99 0.362–0.742) — at 1.2/2.0, ~2x above the worst observed, and prints the free-flight numbers explicitly marked *reported, not asserted*. `free_flight_hard_snaps == 0` is still asserted in both modes, and anything past 2.0m is a hard snap by definition, so a genuine free-flight regression cannot hide behind the looser bound. Verified: 6/6 plain-role runs pass where 3/7 previously failed, all four other modes (free-flight, 80±20ms latency, input transitions, ball contact, match state) still pass, and tightening the new bound to 0.3 makes it fail — the gate is evaluated, not skipped.
+
+The other durable improvement from the first investigation still stands: a percentile alone cannot distinguish "the predictor regressed" from "the client never received the data", so the client gate prints `snapshot_loss` / `snapshot_age` / `rtt` on every run and, on a quality failure with >20% loss, says explicitly that the run was transport-starved — **without converting the failure into a pass**. Both directions verified non-vacuously. It is also what proved the 0.688 was not transport: every reproduction reported 0.0% loss.
+
+**Lesson worth more than the fix: probing only with the purpose-built mode is how a flaky gate stays invisible.** The first pass ran eight variations of process count and CPU load and never once ran the plain role that the reviewer had actually run.
+
+**Task 5.10's three recording gaps, and the real bug closing them found.** The review flagged that the replay log ignored `store_*` failures, never recorded the packets the server *rejected*, and had no caller for `close()`. All three are fixed: a failed write now ends the log permanently rather than desyncing every later record's framing (`write_failed`, checked via `FileAccess.get_error()` once per record); `close()` is called from `_exit_tree` with a summary line, because letting the RefCounted's destructor do it implicitly never tells anyone whether the log is complete; and rejected packets are recorded with their reason in the kind byte (`REJECTED_MALFORMED` / `REJECTED_RATE_LIMIT` / `REJECTED_SEQ_GUARD`, framing unchanged, `FORMAT_VERSION` 2 so "no rejects" can be told from "this build never recorded them"). Recording is capped at 8 per peer per rate-limit window — without that cap the diagnostic is a remote disk-fill amplifier, since the attacker chooses the packet rate. Verified end to end: an honest client logs 0 rejects; `client-abuse-malformed` sends 25 and logs exactly 8; `client-abuse-flood` sustains ~2400 packets/s and logs exactly 8. Uncapped totals are kept separately (`MatchSim.get_reject_totals()`) and survive the peer's disconnect — the first version stored them on `_PeerInputState`, which is erased on disconnect, so every summary printed an empty dictionary.
+
+**And the bug the recording immediately found: the server rate-limited a backlog it caused itself.** A 2s host stall (`SIGSTOP`, standing in for a GC/IO/scheduler hitch) has the client sending at 60Hz throughout, and ENet delivers the whole backlog in the first window after resume — **70 of an honest client's input packets rejected as "rate limit exceeded"**, against a limit that client never came close to violating. Redundancy does not cover it, and that was the assumption worth checking rather than asserting: the dropped packets are *contiguous*, so each one's redundancy window falls inside the same dropped run. Measured with the new log: **0 of 70 rescued, and 82 of 923 sequences (8.88%, ~1.4s of that player's input) never reached the server at all**, versus 0.00% on an otherwise identical run with no stall. Every prediction gate still passed — this is the same class as the Phase 3/4 input-death bugs, invisible to every gate that reads only the client's own state.
+
+Fixed by not policing a backlog the server caused: `MatchSim._physics_process` watches for a wall-clock gap over `STALL_DETECT_MS` (a stalled process doesn't run that callback either, so the first frame after the stall sees the whole gap, which is exactly the size of the backlog about to arrive) and grants each *already-tracked* peer a capped, two-window packet grace. The leaky bucket drains against the same graced budget, or a stall would still accumulate excess toward a disconnect for traffic the server just explicitly allowed. Results: 2s stall, rate-limit rejects 70 → **0**, sequences missing 8.88% → **0.00%**, and `REJECTED_SEQ_GUARD` 9 → 0 as a second-order confirmation (the guard was firing partly *because* the dropped backlog let the client's epoch run away). Across eight stall runs on the fixed build, 7 measured 0.00% missing; the eighth measured 23.54% with zero rate-limit rejects and the seq-guard resync visibly doing its job — a separate, occasional transport-level loss during the stall that this change does not address and does not make worse. The three control runs on the unfixed build lost 4.34%, 7.52% and 7.86%, every time.
+
+Abuse detection is unweakened and this was checked rather than argued: all three abuse roles still disconnect, and **no flood induced a server stall in any run**, so the grace cannot be farmed by flooding. An attacker who *can* induce server stalls to earn budget already has a strictly worse capability than sending extra input packets.
+
+`tools/replay_dump.gd` reads a log back — record counts by kind, plus how much of the input sequence stream actually reached the server once redundancy is counted. It is committed rather than left in a scratch directory because it is what turned "the server dropped some input" into the numbers above, and a log nobody can read is half a feature.
 
 **New/changed test surface:** `--exercise-match-state` (both roles; host forces a goal, client validates the whole observed sequence and the wire byte), `--role=host-disconnect` for the 5.6/5.7 three-process scenario, `--match-length=<s>` to reach `FULL_TIME` in a short run, `--replay-log=<path>`, `--fill-bots`/`--no-fill-bots`, `--max-spectators=<n>`. The ball-contact scenario now **steers at the ball with closed-loop real input** instead of a hand-tuned fixed-heading burst, which 5.3 broke by adding `KICKOFF_YAW_JITTER` (0 contacts in 3/3 runs); it thrusts while turning rather than hovering to aim, which took it from 2/3 to 5/5.