fix(multiplayer): server no longer rate-limits a backlog it caused itself

Closes task 5.10's three recording gaps, and the gap-closing found a real
input-loss bug.

Replay log: a failed write now ends the log permanently instead of
desyncing every later record's framing; close() is called from _exit_tree
with a summary, since the RefCounted destructor closes it implicitly but
never says whether the log is complete; rejected packets are recorded
with their reason in the kind byte (framing unchanged, FORMAT_VERSION 2
so "no rejects" differs from "this build never recorded them"). Recording
is capped at 8 per peer per window - uncapped, the diagnostic is a remote
disk-fill amplifier, since the attacker picks the packet rate. Uncapped
totals live on MatchSim and survive the peer's disconnect.

The bug: a 2s host stall has the client sending at 60Hz throughout, and
ENet delivers that whole backlog in the first window after resume - 70 of
an honest client's packets rejected as "rate limit exceeded". Redundancy
does not cover it, because the dropped packets are contiguous: 0 of 70
rescued, and 82 of 923 sequences (8.88%, ~1.4s of input) never reached
the server, against 0.00% with no stall. Every prediction gate passed.

Fixed by granting each already-tracked peer a capped, two-window packet
grace when the server detects its own wall-clock stall. Rate-limit
rejects 70 -> 0, sequences missing 8.88% -> 0.00%, seq-guard rejects
9 -> 0. Controls on the unfixed build lost 4.34/7.52/7.86%. All three
abuse roles still disconnect and no flood induced a stall, so the grace
cannot be farmed.

Also corrects an earlier wrong conclusion: the reviewer's free-flight
p95 0.688 is real and reproduces on two processes with 0.0% snapshot
loss. The plain --role=client drive fails the 0.5 free-flight bound in
3 of 8 runs because that drive is mostly a contact test - the harness
comment already said so - leaving a cohort as small as 12 samples.
Near-surface error is genuinely several times open-air error, so the
calibrated bound now belongs to --exercise-free-flight alone and the
plain role asserts the always-well-sampled all-cohort percentiles at
1.2/2.0, printing the free-flight numbers as reported-not-asserted.
6/6 plain runs pass where 3/7 failed; tightening to 0.3 still fails.

tools/replay_dump.gd reads a log back: counts by kind, plus how much of
the input sequence stream reached the server once redundancy is counted.
This commit is contained in:
Josh Creek
2026-08-21 16:10:43 +01:00
parent e51dc765a2
commit 866efa0d9b
7 changed files with 469 additions and 12 deletions
+36 -1
View File
@@ -392,7 +392,19 @@ func _owns_world_simulation() -> bool:
func _exit_tree() -> void:
pass
# Task 5.10. Freeing the RefCounted would close the file anyway, but only
# implicitly and only whenever the last reference happens to go — and it
# would never print the summary, which is the one line that tells whoever
# collected the log whether it is complete. Leaving the match scene is the
# real end of the recording, so end it here explicitly.
if _replay_log != null:
_replay_log.close()
print("NetworkedMatch: replay log closed — %d records, %d bytes, %d dropped%s; uncapped reject totals %s" % [
_replay_log.records_written, _replay_log.bytes_written, _replay_log.records_dropped,
" (WRITE FAILED — log is truncated)" if _replay_log.write_failed else "",
MatchSim.get_reject_totals(),
])
_replay_log = null
# ============================================================
@@ -433,6 +445,8 @@ func _start_server() -> void:
MatchSim.send_match_config(arena_path, peer_ids, teams, spawn_indices)
MatchSim.input_received.connect(_on_input_received)
if _replay_log != null:
MatchSim.input_rejected.connect(_on_input_rejected)
NetworkManager.client_disconnected.connect(_on_client_disconnected)
# Piggyback live state on the existing retry loop, so a peer that missed
# the join-time bootstrap gets one every time it re-asks for config.
@@ -522,6 +536,15 @@ func _on_input_received(peer_id: int, decoded: Dictionary) -> void:
if seq > seq_bound:
slot.consecutive_seq_rejects += 1
if slot.consecutive_seq_rejects < SEQ_REJECT_RESYNC_LIMIT:
# Recorded, not just counted: this is the drop that used to
# be permanent input death, and a log that shows only what
# the server accepted cannot distinguish "the client stopped
# sending" from "the server refused everything it sent".
if _replay_log != null:
_replay_log.record_rejected_input(
ReplayLog.RecordKind.REJECTED_SEQ_GUARD,
Engine.get_physics_frames(), peer_id, decoded.get("raw", PackedByteArray())
)
return
# Fall through and accept: this is the escape hatch, not a
# missing `return`.
@@ -538,6 +561,18 @@ func _on_input_received(peer_id: int, decoded: Dictionary) -> void:
_unknown_sender_input_count += 1
# Task 5.10, server only, connected only when a replay log is open. MatchSim
# rejects at the protocol layer and knows nothing about the replay format, so
# the reason-to-record-kind mapping lives here.
func _on_input_rejected(peer_id: int, reason: int, bytes: PackedByteArray) -> void:
if _replay_log == null:
return
var kind := ReplayLog.RecordKind.REJECTED_MALFORMED
if reason == MatchSim.InputRejectReason.RATE_LIMIT:
kind = ReplayLog.RecordKind.REJECTED_RATE_LIMIT
_replay_log.record_rejected_input(kind, Engine.get_physics_frames(), peer_id, bytes)
# --- §6.1 match state machine (task 5.1) -----------------------------------
#
# Deliberately does NOT gate physics, freezing or input this task. Tasks 5.3