fix(multiplayer): resolve composition regression from second adversarial review

A second adversarial review of the previous fix commit found two of its
nine fixes silently defeated each other: the seq-range guard (fix for a
MEDIUM epoch-mismatch finding) capped the exact variable the ring-overflow
resync (fix for the original CRITICAL finding) depends on, making the
resync unreachable in production and recreating permanent input death at
a lower failure threshold, reachable via ordinary server tick loss alone.

- CRITICAL: rebind the seq-range guard to InputJitterBuffer's own
  highest_ingested_seq (now public) instead of the consumer-side
  last_applied_seq, so it tracks the client's send epoch rather than a
  value that can lag arbitrarily far behind during a stall.
- HIGH: InputLeadController's release logic still ANDed the old
  `lead > LEAD_MIN` gate onto the new depth-driven condition, so a
  backlog the controller never caused still couldn't drain. Split into
  two independent decisions: the seq-duplicate action follows real
  depth alone; lead's own bookkeeping separately never drops below its
  floor.
- MEDIUM: widen the CI driver's movement/stalled sampling margin
  (run_seconds - 2.0, was - 0.5) and assert the peer is still in
  multiplayer.get_peers() at sample time, since the old margin let the
  check pass on residual starvation grace after a bot had already
  disconnected.
- LOW: measure horizontal-only displacement in the human smoke test's
  movement check — the old 3D-distance bar was beatable by pure
  gravity settling with fully dead input.
- LOW: fix a real "clean stderr" violation (match_net.gd broadcasting
  a departure notice to a peer whose ENet channels are already torn
  down, including a second peer disconnecting in the same poll batch)
  by deferring the notification to the next idle frame.
- Wire the server's per-slot stalled bit into the client debug overlay
  for real — a prior commit message claimed this already reached the
  overlay when only the CI gate actually read it.

Re-verified end-to-end against the real production RPC path (not just
unit tests in isolation, which is how the composition bug got past the
first round): a 2-bot CI match with a 1.5s host SIGSTOP freeze injected
mid-run, well past the 0.6s threshold the review reproduced the bug at,
now recovers cleanly on repeated runs with zero stderr noise.
This commit is contained in:
Josh Creek
2026-08-20 18:26:12 +01:00
parent 2325313ad2
commit cf73074e27
8 changed files with 180 additions and 53 deletions
+45 -11
View File
@@ -256,16 +256,35 @@ func _on_input_received(peer_id: int, decoded: Dictionary) -> void:
# ring" rationale wasn't actually achieved), and could silently
# drop an honest client's input forever the moment accumulated
# server tick loss closed whatever accidental head-start margin
# existed. Bound against this slot's own last_applied_seq
# instead — the client's own epoch, which the ring is already
# anchored to — using the ring's own capacity as the bound,
# exactly matching what InputJitterBuffer.consume()'s own
# overflow-resync logic treats as "unrecoverably far ahead"
# anyway. Falls back to seq itself (never rejects) before the
# buffer has ever been seeded — there's no baseline yet to
# bound against.
# existed.
#
# A first rebound bounded against this slot's own
# last_applied_seq — the CONSUMER's position — using the ring's
# capacity as the bound. A second adversarial review found this
# broke the ring-overflow resync it was landed alongside: capping
# every accepted seq at last_applied_seq + RING_SIZE also caps
# jb.highest_ingested_seq at that same ceiling, so
# consume()'s resync condition (which needs highest_ingested_seq
# to reach expected + RING_SIZE) could never fire in production —
# silently recreating the exact permanent-input-death bug this
# whole guard-rebound was part of fixing, at an even LOWER
# freeze threshold, reachable via ordinary server tick loss alone
# with no external trigger.
#
# Bound against jb.highest_ingested_seq instead — the highest
# seq this slot has ever actually been ALLOWED to ingest, i.e.
# the client's own send epoch — using the ring's own capacity as
# the bound, same as before. An honest client's consecutive
# packets differ by only a few seq (redundancy + a bounded
# input_lead skip), so this bound tracks a well-behaved client
# regardless of how far the CONSUMER has fallen behind, while
# still rejecting a single garbage-far-future jump: an attacker
# can only walk highest_ingested_seq forward at the rate the
# packet-rate limiter (§3.4) already allows. Falls back to seq
# itself (never rejects) before the buffer has ever been
# seeded — there's no baseline yet to bound against.
var jb := slot.jitter_buffer
var seq_bound: int = (jb.last_applied_seq if jb.last_applied_seq >= 0 else seq) + InputJitterBuffer.RING_SIZE
var seq_bound: int = (jb.highest_ingested_seq if jb.highest_ingested_seq >= 0 else seq) + InputJitterBuffer.RING_SIZE
if seq > seq_bound:
return
jb.ingest(seq, decoded["actions"])
@@ -343,8 +362,9 @@ func _ship_to_net_body_state(ship: Ship, stalled: bool) -> NetBodyState:
# §3.2: InputJitterBuffer.stalled was computed all along but never
# reached the wire — an adversarial review found this was the exact
# signal that would have made the ring-overflow bug (this session's
# critical fix) visible to the client, the debug overlay, and the CI
# gate, and its absence is part of why none of them ever noticed.
# critical fix) visible to the client and the CI gate, and its absence
# is part of why neither ever noticed. get_net_debug_stats() below is
# what actually surfaces it to the debug overlay now.
s.stalled = stalled
return s
@@ -588,11 +608,25 @@ func get_net_debug_stats() -> Dictionary:
# long has passed with nothing arriving at all.
var is_stale := _last_snapshot_wall_ms >= 0 and Time.get_ticks_msec() - _last_snapshot_wall_ms > SNAPSHOT_STALE_MS
var snapshot_loss_pct := 100.0 if is_stale else _snapshot_loss_ewma * 100.0
# The server's jitter_buffer.stalled bit for THIS client's own slot,
# round-tripped through NetBodyState onto the wire (§3.2) — added by the
# first adversarial-review fix round, but a second review found nothing
# actually read it client-side (net_interpolator.gd only passed it
# through lerp/extrapolate), so the commit's claim that it made the
# server-side starvation state "visible to the client, the debug
# overlay" was false; only the CI gate read it, and only via the
# server's own field directly, not the wire bit. Read it here for real.
var server_stalled := false
if is_instance_valid(_my_slot):
var latest := _my_slot.interpolator.latest()
if latest != null:
server_stalled = latest.stalled
return {
"input_buffer_depth": _last_known_input_buffer_depth,
"input_lead": _input_lead_controller.lead,
"snapshot_age_ms": snapshot_age_ms,
"snapshot_loss_pct": snapshot_loss_pct,
"server_stalled": server_stalled,
}