mirror of
https://github.com/jcreek/CosmicClash.git
synced 2026-09-13 02:52:01 +00:00
fix(multiplayer): adversarial review fixes for Phase 3
An Opus subagent's adversarial review of Phase 3 found a critical, silent, permanent bug plus eight smaller real issues, all empirically verified with real two- and three-process runs: CRITICAL: InputJitterBuffer's 32-entry ring permanently bricked a player's input once the un-consumed backlog exceeded the ring's capacity - a fresh arrival would land in the exact slot consume() was still waiting on, and since both counters only ever advance, the gap never closed. Reproduced with a real SIGSTOP/SIGCONT host freeze: client movement dropped from ~26m to 0.00m at ~0.7s, worse under real loss (a lossy link lowered the fatal threshold to ~400ms), and reachable via ordinary clock drift with no external trigger at all. Fixed by tracking the highest seq ever ingested and having consume() jump directly to what the ring can still provide once the gap exceeds capacity, instead of starving through an unrecoverable span. Re-verified with a 3s freeze (well past the original threshold): full recovery. HIGH: InputLeadController's release logic was gated on its own past attacks (lead > LEAD_MIN) rather than the real server-reported depth, so a backlog it didn't itself cause was never drained. Fixed to gate on actual depth vs target. MEDIUM-HIGH: the rate limiter's "N consecutive over-budget seconds" streak hard-reset to 0 on any clean window, letting a duty-cycled flood (burst, one clean window, repeat) sustain ~33x budget indefinitely with zero warnings. Replaced with a leaky-bucket accumulator immune to the same evasion by construction. MEDIUM: the seq > server_tick + 20 guard compared two unrelated clock epochs (server process uptime vs. client's own from-zero seq numbering), so it never actually protected anything on a long-running server and could silently drop an honest client's input forever. Bound against the buffer's own last_applied_seq instead. MEDIUM: InputJitterBuffer.stalled was computed but never reached the wire - the one signal that would have made the ring-overflow bug visible anywhere. Now wired through _ship_to_net_body_state. MEDIUM: task 3.6's CI driver's assertions didn't depend on client input reaching the server at all, so it kept passing with the ring-overflow bug actively triggered. Added real ship-movement and non-stalled checks, sampled while bots are still connected (an initial attempt sampled after their own legitimate disconnect, which starves identically to the bug). LOW-MEDIUM: a lead change silently mislabelled _input_history's older entries, since the wire format has no per-entry seq field. Fixed by handling each delta case (ordinary/release/attack) on its own terms. LOW: bandwidth and snapshot-loss overlay metrics froze at their last value during a total outage instead of decaying - exactly when they matter most. Both now report honest post-outage values. LOW: a guard comment on NetworkManager._ping misdescribed the actual disconnect_peer() arguments in use. Corrected. New permanent regression tests: test_ring_overflow_resyncs_to_fresh_data _instead_of_starving_forever, test_release_drains_a_backlog_it_never_ caused_itself, and client-abuse-flood-dutycycle (reproduces the exact duty-cycle evasion). Full regression suite, including the net-sim-latency milestone gate, all abuse roles, and the CI driver, re-run clean after every fix.
This commit is contained in:
@@ -35,6 +35,14 @@ var _ring_seq: PackedInt32Array = PackedInt32Array()
|
||||
# comment for why an un-seeded buffer would otherwise never converge with
|
||||
# what the client is actually sending.
|
||||
var _seeded := false
|
||||
# Highest seq ever seen by ingest(), regardless of whether it's still in the
|
||||
# ring — consume()'s only way to tell "the data is gone because the ring
|
||||
# overflowed" apart from "the data just hasn't arrived yet". See consume()'s
|
||||
# own comment for why this exists: an adversarial review found that without
|
||||
# it, a backlog bigger than RING_SIZE (a host stall, or persistent client/
|
||||
# server clock drift) permanently zeroed a connected player's input for the
|
||||
# rest of the match.
|
||||
var _highest_ingested_seq := -1
|
||||
|
||||
|
||||
func _init() -> void:
|
||||
@@ -64,6 +72,8 @@ func ingest(newest_seq: int, actions: Array) -> void:
|
||||
# "expected" with reality the moment real data first exists.
|
||||
last_applied_seq = newest_seq - actions.size()
|
||||
_seeded = true
|
||||
if newest_seq > _highest_ingested_seq:
|
||||
_highest_ingested_seq = newest_seq
|
||||
for i in actions.size():
|
||||
var seq: int = newest_seq - i
|
||||
if seq <= last_applied_seq:
|
||||
@@ -95,6 +105,25 @@ func consume() -> ShipAction:
|
||||
return last_action
|
||||
var expected := last_applied_seq + 1
|
||||
var idx := expected % RING_SIZE
|
||||
|
||||
# Ring-overflow resync. A fixed-size ring can only ever hold RING_SIZE
|
||||
# ticks of not-yet-consumed data at once — if the caller has fallen
|
||||
# further behind the newest data actually arriving than that (a host
|
||||
# stall, or persistent client/server clock drift), every tick between
|
||||
# "expected" and "_highest_ingested_seq - RING_SIZE" has already been
|
||||
# irrecoverably overwritten by more recent arrivals landing on the same
|
||||
# ring slots. Waiting for it tick-by-tick would starve — and, past
|
||||
# STARVE_ZERO_TICKS, zero this player's ship — for the ENTIRE gap even
|
||||
# though fresh, real input already exists in the ring right now. An
|
||||
# adversarial review found and reproduced this exact failure (a ~0.7s
|
||||
# host freeze permanently zeroed a connected player's input for the
|
||||
# rest of the match, with no self-recovery). Skip the unrecoverable
|
||||
# span and resync directly to what the ring can still actually provide.
|
||||
if _highest_ingested_seq - expected >= RING_SIZE:
|
||||
last_applied_seq = _highest_ingested_seq - RING_SIZE
|
||||
expected = last_applied_seq + 1
|
||||
idx = expected % RING_SIZE
|
||||
|
||||
if _ring_seq[idx] == expected:
|
||||
last_action = _ring_action[idx]
|
||||
starved_ticks = 0
|
||||
|
||||
@@ -38,6 +38,11 @@ const LEAD_MAX := 12
|
||||
const MIN_CHANGE_INTERVAL_TICKS := 30
|
||||
const RELEASE_INTERVAL_TICKS := 60
|
||||
const CLEAN_SURPLUS_TICKS := 120 # 2s at 60Hz
|
||||
# §3.3: "target_depth = 1 (16.7 ms), not 2." Release only fires when the
|
||||
# server-reported depth is genuinely ABOVE this — see update()'s own
|
||||
# comment for why gating on `lead` alone (an adversarial review's original
|
||||
# finding here) was wrong.
|
||||
const TARGET_DEPTH := 1
|
||||
|
||||
var lead := LEAD_MIN
|
||||
|
||||
@@ -72,7 +77,20 @@ func update(input_buffer_depth: int) -> int:
|
||||
return 1 + delta
|
||||
return 1
|
||||
|
||||
_clean_surplus_ticks += 1
|
||||
# Release must react to the ACTUAL server-reported depth, not to this
|
||||
# controller's own memory of past attacks. An adversarial review found
|
||||
# the original gate here was `lead > LEAD_MIN` — a self-tracked counter
|
||||
# of this controller's own past decisions — so any backlog it did NOT
|
||||
# itself create (a server hitch, persistent client/server clock drift,
|
||||
# a burst re-delivery) was never drained: `lead` stayed at its starting
|
||||
# value the whole time even while `input_buffer_depth` sat well above
|
||||
# target, permanently adding latency with the control loop reporting
|
||||
# itself perfectly healthy. Gate on the real signal instead.
|
||||
if input_buffer_depth > TARGET_DEPTH:
|
||||
_clean_surplus_ticks += 1
|
||||
else:
|
||||
_clean_surplus_ticks = 0
|
||||
|
||||
if _clean_surplus_ticks >= CLEAN_SURPLUS_TICKS and _ticks_since_change >= RELEASE_INTERVAL_TICKS and lead > LEAD_MIN:
|
||||
lead -= 1
|
||||
_ticks_since_change = 0
|
||||
|
||||
@@ -38,7 +38,16 @@ const RATE_LIMIT_PACKETS_PER_SEC := 110
|
||||
# sync by hand.
|
||||
const RATE_LIMIT_BYTES_PER_SEC := RATE_LIMIT_PACKETS_PER_SEC * (NetCodec.INPUT_HEADER_SIZE + NetCodec.MAX_REDUNDANCY * NetCodec.INPUT_ENTRY_SIZE)
|
||||
const RATE_LIMIT_WINDOW_MS := 1000
|
||||
const RATE_LIMIT_OVER_BUDGET_SECONDS_TO_DISCONNECT := 3
|
||||
# Leaky-bucket excess tolerance, expressed in the same "N seconds' worth of
|
||||
# budget" terms the original consecutive-streak design used. An adversarial
|
||||
# review found that design — a streak counter that HARD-RESET to 0 on any
|
||||
# single clean window — was trivially evaded by a duty-cycled flood (burst,
|
||||
# then one clean window, repeat): reproduced sustaining ~33x the packet
|
||||
# budget indefinitely with zero disconnect warnings. A leaky bucket doesn't
|
||||
# care how the excess is distributed in time — see the window-roll logic
|
||||
# below for how it accumulates and drains.
|
||||
const RATE_LIMIT_EXCESS_PACKETS_TO_DISCONNECT := RATE_LIMIT_PACKETS_PER_SEC * 3
|
||||
const RATE_LIMIT_EXCESS_BYTES_TO_DISCONNECT := RATE_LIMIT_BYTES_PER_SEC * 3
|
||||
const MALFORMED_LIMIT_TO_DISCONNECT := 20
|
||||
|
||||
|
||||
@@ -46,7 +55,14 @@ class _PeerInputState:
|
||||
var window_start_ms := 0
|
||||
var packets_this_window := 0
|
||||
var bytes_this_window := 0
|
||||
var over_budget_seconds := 0
|
||||
# Leaky bucket: grows by this window's actual total, drains by one
|
||||
# window's worth of budget, every window — regardless of whether that
|
||||
# window was itself over or under budget. A steady rate at or under
|
||||
# budget nets to zero forever (never accumulates); any sustained AVERAGE
|
||||
# above budget accumulates over time no matter how it's shaped into
|
||||
# bursts, unlike a streak counter a clean gap can reset to 0.
|
||||
var excess_packets := 0.0
|
||||
var excess_bytes := 0.0
|
||||
var malformed_count := 0
|
||||
|
||||
|
||||
@@ -57,7 +73,9 @@ var _peer_input_state: Dictionary = {} # peer_id -> _PeerInputState, server on
|
||||
# messages, not what §2's byte-budget analysis or a live overlay cares
|
||||
# about. Rolling per-second counters, recomputed opportunistically on each
|
||||
# send/receive rather than on a timer — nothing needs the rate outside of
|
||||
# an on-demand overlay read anyway.
|
||||
# an on-demand overlay read anyway. Use get_bytes_sent_per_sec() /
|
||||
# get_bytes_received_per_sec() to READ these, not the raw fields directly
|
||||
# — see those functions for why.
|
||||
const BANDWIDTH_WINDOW_MS := 1000
|
||||
var bytes_sent_per_sec := 0.0
|
||||
var bytes_received_per_sec := 0.0
|
||||
@@ -67,6 +85,25 @@ var _received_window_start_ms := 0
|
||||
var _received_window_bytes := 0
|
||||
|
||||
|
||||
# An adversarial review found bytes_*_per_sec only ever gets recomputed
|
||||
# INSIDE _track_sent()/_track_received() — i.e. only when traffic actually
|
||||
# arrives — so if traffic stops entirely (right before a disconnect, or
|
||||
# during exactly the kind of outage this overlay exists to diagnose), the
|
||||
# last computed rate displays forever instead of decaying toward zero.
|
||||
# Report zero once meaningfully more than one window has passed with
|
||||
# nothing tracked, rather than trusting a stale field.
|
||||
func get_bytes_sent_per_sec() -> float:
|
||||
if Time.get_ticks_msec() - _sent_window_start_ms > BANDWIDTH_WINDOW_MS * 2:
|
||||
return 0.0
|
||||
return bytes_sent_per_sec
|
||||
|
||||
|
||||
func get_bytes_received_per_sec() -> float:
|
||||
if Time.get_ticks_msec() - _received_window_start_ms > BANDWIDTH_WINDOW_MS * 2:
|
||||
return 0.0
|
||||
return bytes_received_per_sec
|
||||
|
||||
|
||||
func _ready() -> void:
|
||||
NetworkManager.client_disconnected.connect(func(peer_id: int) -> void: _peer_input_state.erase(peer_id))
|
||||
|
||||
@@ -161,13 +198,13 @@ func _recv_input(bytes: PackedByteArray) -> void:
|
||||
# when nothing is arriving anyway.
|
||||
var now_ms := Time.get_ticks_msec()
|
||||
if now_ms - state.window_start_ms >= RATE_LIMIT_WINDOW_MS:
|
||||
var was_over_budget := state.packets_this_window > RATE_LIMIT_PACKETS_PER_SEC or state.bytes_this_window > RATE_LIMIT_BYTES_PER_SEC
|
||||
state.over_budget_seconds = (state.over_budget_seconds + 1) if was_over_budget else 0
|
||||
state.excess_packets = maxf(0.0, state.excess_packets + float(state.packets_this_window) - float(RATE_LIMIT_PACKETS_PER_SEC))
|
||||
state.excess_bytes = maxf(0.0, state.excess_bytes + float(state.bytes_this_window) - float(RATE_LIMIT_BYTES_PER_SEC))
|
||||
state.window_start_ms = now_ms
|
||||
state.packets_this_window = 0
|
||||
state.bytes_this_window = 0
|
||||
if state.over_budget_seconds >= RATE_LIMIT_OVER_BUDGET_SECONDS_TO_DISCONNECT:
|
||||
_disconnect_abusive_peer(peer_id, "input rate limit exceeded for %d consecutive seconds" % state.over_budget_seconds)
|
||||
if state.excess_packets > RATE_LIMIT_EXCESS_PACKETS_TO_DISCONNECT or state.excess_bytes > RATE_LIMIT_EXCESS_BYTES_TO_DISCONNECT:
|
||||
_disconnect_abusive_peer(peer_id, "input rate limit exceeded (excess_packets=%.0f excess_bytes=%.0f)" % [state.excess_packets, state.excess_bytes])
|
||||
return
|
||||
|
||||
state.packets_this_window += 1
|
||||
|
||||
@@ -34,7 +34,7 @@ func _process(_delta: float) -> void:
|
||||
return
|
||||
if NetworkManager.is_server:
|
||||
_label.text = "NET: server, %d peer(s) out %s in %s" % [
|
||||
MatchNet.roster.size(), _format_kbps(MatchSim.bytes_sent_per_sec), _format_kbps(MatchSim.bytes_received_per_sec),
|
||||
MatchNet.roster.size(), _format_kbps(MatchSim.get_bytes_sent_per_sec()), _format_kbps(MatchSim.get_bytes_received_per_sec()),
|
||||
]
|
||||
elif NetworkManager.is_client:
|
||||
if NetworkManager.rtt_ms < 0.0:
|
||||
@@ -52,7 +52,7 @@ func _process(_delta: float) -> void:
|
||||
NetworkManager.rtt_ms, NetworkManager.jitter_ms, NetworkManager.clock_offset_ms,
|
||||
str(stats.get("input_buffer_depth", -1)), str(stats.get("input_lead", "-")),
|
||||
stats.get("snapshot_loss_pct", 0.0), stats.get("snapshot_age_ms", 0.0),
|
||||
_format_kbps(MatchSim.bytes_sent_per_sec), _format_kbps(MatchSim.bytes_received_per_sec),
|
||||
_format_kbps(MatchSim.get_bytes_sent_per_sec()), _format_kbps(MatchSim.get_bytes_received_per_sec()),
|
||||
]
|
||||
else:
|
||||
_label.text = "NET: offline"
|
||||
|
||||
@@ -186,11 +186,16 @@ func _ping(client_send_ms: int) -> void:
|
||||
var sender_id := multiplayer.get_remote_sender_id()
|
||||
# A single poll() call can process several queued RPCs from the same
|
||||
# peer in one batch — an earlier one in that same batch (e.g. task 3.4's
|
||||
# abuse-triggered disconnect_peer(..., now=true), which removes the
|
||||
# peer immediately rather than waiting for an acknowledged disconnect)
|
||||
# can leave this ping's sender no longer a valid peer by the time its
|
||||
# own turn in the batch comes up. NetSim's inactive/passthrough path
|
||||
# (the common case — no CLI flags) dispatches immediately with no
|
||||
# abuse-triggered match_sim.gd disconnect_peer() call, or the peer
|
||||
# disconnecting for any other reason mid-batch) can leave this ping's
|
||||
# sender no longer a valid peer by the time its own turn in the batch
|
||||
# comes up. Empirically confirmed reachable with disconnect_peer()'s
|
||||
# default arguments (a graceful, non-forced disconnect — match_sim.gd's
|
||||
# own disconnect call tried force=true as an alternative and reverted
|
||||
# it, since that left Godot's own peer-list bookkeeping inconsistent
|
||||
# and produced far MORE of this exact class of error, not fewer:
|
||||
# hundreds vs. one, verified). NetSim's inactive/passthrough path (the
|
||||
# common case — no CLI flags) dispatches immediately with no
|
||||
# validation of its own, so check here rather than relying on it.
|
||||
if sender_id not in multiplayer.get_peers():
|
||||
return
|
||||
|
||||
@@ -113,11 +113,13 @@ var _last_known_input_buffer_depth := -1 # client only: -1 = no snapshot with t
|
||||
const SNAPSHOT_LOSS_EWMA_ALPHA := 1.0 / 16.0
|
||||
var _snapshot_loss_ewma := 0.0
|
||||
var _expected_next_snapshot_tick := -1
|
||||
# §3.1 step 4. Not 120: InputLeadController.LEAD_MAX is 12, so anything
|
||||
# claiming to be further ahead of the current server tick than this is
|
||||
# broken or hostile, not just an honest client running a legitimately fast
|
||||
# lead.
|
||||
const MAX_SEQ_LEAD_TICKS := 20
|
||||
# An adversarial review found _snapshot_loss_ewma only updates on receipt —
|
||||
# during a TOTAL outage, exactly when this metric matters most, it freezes
|
||||
# at its last (probably low/healthy) value instead of climbing toward
|
||||
# 100%. Track wall-clock receipt time so get_net_debug_stats() can report
|
||||
# honestly once too long has passed with nothing arriving at all.
|
||||
var _last_snapshot_wall_ms := -1
|
||||
const SNAPSHOT_STALE_MS := 500.0 # ~30 ticks with nothing at all — treat as total loss, not "still fine"
|
||||
var _unknown_sender_input_count := 0 # server only, observability (§3.1 step 1)
|
||||
var _reset_gen := 0 # server only: bumped on every kickoff/goal reset so the client hard-snaps instead of interpolating across the teleport
|
||||
# Server only. _on_goal_scored's reset_ball()/reset_ships() only QUEUE
|
||||
@@ -241,18 +243,32 @@ func _on_input_received(peer_id: int, decoded: Dictionary) -> void:
|
||||
for slot in _slots:
|
||||
if slot.peer_id == peer_id:
|
||||
var seq: int = decoded["seq"]
|
||||
# §3.1 step 4. Not 120: input_lead is clamped to
|
||||
# InputLeadController.LEAD_MAX (12), so anything claiming to be
|
||||
# further ahead than this is broken or hostile, not just a fast
|
||||
# lead. This is also why InputJitterBuffer's ring can be fixed-
|
||||
# size — a client can never make the server allocate — but
|
||||
# rejecting the packet here still keeps garbage-far-future seq
|
||||
# values out of the ring entirely rather than letting them
|
||||
# silently overwrite a near-future slot some honest, in-range
|
||||
# packet is about to need.
|
||||
if seq > Engine.get_physics_frames() + MAX_SEQ_LEAD_TICKS:
|
||||
# §3.1 step 4, rebound after an adversarial review found the
|
||||
# original check (seq > Engine.get_physics_frames() + 20)
|
||||
# compared two unrelated epochs: get_physics_frames() counts
|
||||
# from the SERVER PROCESS's own start, while a client's
|
||||
# _input_seq starts at 0 when ITS match scene loads —
|
||||
# input_jitter_buffer.gd's own seeding logic exists specifically
|
||||
# because these share no baseline (see its header comment).
|
||||
# Bounding against server uptime meant this guard could never
|
||||
# fire on a long-running dedicated server (no real protection —
|
||||
# the stated "keeps garbage-far-future seq values out of the
|
||||
# ring" rationale wasn't actually achieved), and could silently
|
||||
# drop an honest client's input forever the moment accumulated
|
||||
# server tick loss closed whatever accidental head-start margin
|
||||
# existed. Bound against this slot's own last_applied_seq
|
||||
# instead — the client's own epoch, which the ring is already
|
||||
# anchored to — using the ring's own capacity as the bound,
|
||||
# exactly matching what InputJitterBuffer.consume()'s own
|
||||
# overflow-resync logic treats as "unrecoverably far ahead"
|
||||
# anyway. Falls back to seq itself (never rejects) before the
|
||||
# buffer has ever been seeded — there's no baseline yet to
|
||||
# bound against.
|
||||
var jb := slot.jitter_buffer
|
||||
var seq_bound: int = (jb.last_applied_seq if jb.last_applied_seq >= 0 else seq) + InputJitterBuffer.RING_SIZE
|
||||
if seq > seq_bound:
|
||||
return
|
||||
slot.jitter_buffer.ingest(seq, decoded["actions"])
|
||||
jb.ingest(seq, decoded["actions"])
|
||||
slot.last_client_send_ms = decoded["client_send_ms"]
|
||||
return
|
||||
# A connected-but-not-yet-slotted peer (or one whose slot somehow
|
||||
@@ -285,7 +301,7 @@ func _broadcast_snapshot() -> void:
|
||||
# total-garbage failure mode the moment that stops being true, and the
|
||||
# fix costs nothing.
|
||||
for slot in _slots:
|
||||
bodies.append(_ship_to_net_body_state(slot.ship) if is_instance_valid(slot.ship) else NetBodyState.new())
|
||||
bodies.append(_ship_to_net_body_state(slot.ship, slot.jitter_buffer.stalled) if is_instance_valid(slot.ship) else NetBodyState.new())
|
||||
if is_instance_valid(ball):
|
||||
bodies.append(_ball_to_net_body_state(ball))
|
||||
var segment := NetCodec.pack_snapshot_body_segment(server_tick, 0, _reset_gen, bodies)
|
||||
@@ -312,7 +328,7 @@ func _broadcast_snapshot() -> void:
|
||||
MatchSim.send_snapshot(slot.peer_id, bytes)
|
||||
|
||||
|
||||
func _ship_to_net_body_state(ship: Ship) -> NetBodyState:
|
||||
func _ship_to_net_body_state(ship: Ship, stalled: bool) -> NetBodyState:
|
||||
var s := NetBodyState.new()
|
||||
s.position = ship.global_position
|
||||
s.rotation = ship.global_transform.basis.get_rotation_quaternion()
|
||||
@@ -324,6 +340,12 @@ func _ship_to_net_body_state(ship: Ship) -> NetBodyState:
|
||||
# forward thrust drives the visible flame (see task 2.6).
|
||||
s.thrust_z = clampf(maxf(ship.controller.get_action().thrust.z if ship.controller else 0.0, 0.0), 0.0, 1.0)
|
||||
s.avel_range = NetCodec.SHIP_AVEL_RANGE
|
||||
# §3.2: InputJitterBuffer.stalled was computed all along but never
|
||||
# reached the wire — an adversarial review found this was the exact
|
||||
# signal that would have made the ring-overflow bug (this session's
|
||||
# critical fix) visible to the client, the debug overlay, and the CI
|
||||
# gate, and its absence is part of why none of them ever noticed.
|
||||
s.stalled = stalled
|
||||
return s
|
||||
|
||||
|
||||
@@ -433,23 +455,46 @@ func _send_local_input() -> void:
|
||||
# increments its send sequence by exactly one tick's worth), but a lead
|
||||
# change this tick skips extra sequence numbers (attack, more server-
|
||||
# side buffer margin) or duplicates the current one (release, delta 0 —
|
||||
# one tick of latency recovered). A duplicated tick can, in the narrow
|
||||
# case where an older redundant copy hasn't been superseded yet, smear
|
||||
# one of _input_history's older backup slots by one position — the
|
||||
# PRIMARY (freshest, most-recently-relevant) value for every seq is
|
||||
# unaffected, so this only ever degrades a backup copy, never the real
|
||||
# per-tick record; §3.3 itself only promises "skip or duplicate a
|
||||
# sequence number," not frame-perfect bookkeeping under a lead change.
|
||||
_input_seq += _input_lead_controller.update(_last_known_input_buffer_depth)
|
||||
# one tick of latency recovered).
|
||||
var delta := _input_lead_controller.update(_last_known_input_buffer_depth)
|
||||
_input_seq += delta
|
||||
# Redundancy (§3.1): carry the last MAX_REDUNDANCY ticks' actions,
|
||||
# newest-first, so a burst of up to (MAX_REDUNDANCY - 1) consecutive
|
||||
# packet losses still lets the server recover every dropped tick's
|
||||
# action from a later packet — InputJitterBuffer.ingest() discards
|
||||
# whichever of these the server already applied, so re-sending old
|
||||
# ticks every packet is harmless, not just tolerated.
|
||||
_input_history.push_front(action)
|
||||
if _input_history.size() > NetCodec.MAX_REDUNDANCY:
|
||||
_input_history.resize(NetCodec.MAX_REDUNDANCY)
|
||||
# ticks every packet is harmless, not just tolerated. NetCodec's wire
|
||||
# format has no per-entry seq field — actions[i] is implicitly
|
||||
# "seq - i" — so _input_history must actually BE that many consecutive
|
||||
# ticks, not just "the last few samples taken". A plain push_front on
|
||||
# every tick regardless of delta broke that: an adversarial review
|
||||
# found a lead change silently relabelled older entries (a duplicated
|
||||
# tick shifts everything back by one position without a matching seq
|
||||
# change, and a skip-ahead makes the whole history discontiguous with
|
||||
# the new seq), causing the server to replay already-applied ticks or
|
||||
# apply the wrong redundant copy for a given seq. Handle each case on
|
||||
# its own terms instead of always pushing.
|
||||
if delta == 1:
|
||||
_input_history.push_front(action)
|
||||
if _input_history.size() > NetCodec.MAX_REDUNDANCY:
|
||||
_input_history.resize(NetCodec.MAX_REDUNDANCY)
|
||||
elif delta == 0:
|
||||
# Release: seq didn't advance, so this tick's freshest sample
|
||||
# REPLACES the front entry (still "seq") rather than pushing
|
||||
# everything else back a position under a label that no longer
|
||||
# matches what's actually there.
|
||||
if _input_history.is_empty():
|
||||
_input_history.push_front(action)
|
||||
else:
|
||||
_input_history[0] = action
|
||||
else:
|
||||
# Attack: seq jumped ahead by more than one, so nothing previously
|
||||
# in history is contiguous with the new seq any more — the skipped
|
||||
# range was never sent, by design (that's what "buys more server-
|
||||
# side buffer margin" means). Reset the redundancy window to just
|
||||
# this tick's sample; it rebuilds naturally over the next few
|
||||
# ticks, the same way it does at connection start.
|
||||
_input_history = [action]
|
||||
var bytes := NetCodec.pack_input(_input_seq, _last_received_snapshot_tick, Time.get_ticks_msec(), _input_history)
|
||||
MatchSim.send_input(bytes)
|
||||
|
||||
@@ -464,6 +509,7 @@ func _on_snapshot_received(decoded: Dictionary) -> void:
|
||||
_snapshot_loss_ewma += (sample - _snapshot_loss_ewma) * SNAPSHOT_LOSS_EWMA_ALPHA
|
||||
_expected_next_snapshot_tick = server_tick + 1
|
||||
_last_received_snapshot_tick = server_tick
|
||||
_last_snapshot_wall_ms = Time.get_ticks_msec()
|
||||
# Per-client header (§2.4): unlike the shared body segment, this is
|
||||
# genuinely this recipient's own — input_buffer_depth is THIS client's
|
||||
# own slot's server-side InputJitterBuffer.depth() at send time, which
|
||||
@@ -535,11 +581,18 @@ func get_net_debug_stats() -> Dictionary:
|
||||
if NetworkManager.rtt_ms >= 0.0:
|
||||
var estimated_now_tick := _estimated_tick(NetworkManager.get_server_time_estimate_ms())
|
||||
snapshot_age_ms = (estimated_now_tick - float(_last_received_snapshot_tick)) * NetInterpolator.TICK_MS
|
||||
# _snapshot_loss_ewma only updates on receipt, so during a TOTAL outage
|
||||
# — exactly when this matters most — it would otherwise freeze at
|
||||
# whatever it last read (probably low/healthy) instead of climbing
|
||||
# toward 100%, an adversarial review found. Report honestly once too
|
||||
# long has passed with nothing arriving at all.
|
||||
var is_stale := _last_snapshot_wall_ms >= 0 and Time.get_ticks_msec() - _last_snapshot_wall_ms > SNAPSHOT_STALE_MS
|
||||
var snapshot_loss_pct := 100.0 if is_stale else _snapshot_loss_ewma * 100.0
|
||||
return {
|
||||
"input_buffer_depth": _last_known_input_buffer_depth,
|
||||
"input_lead": _input_lead_controller.lead,
|
||||
"snapshot_age_ms": snapshot_age_ms,
|
||||
"snapshot_loss_pct": _snapshot_loss_ewma * 100.0,
|
||||
"snapshot_loss_pct": snapshot_loss_pct,
|
||||
}
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user