feat(multiplayer): Phase 4 prediction correctness + two input-death fixes

Closes Phase 4's outstanding action-sequence-correctness invariant, then
fixes two server-side bugs an adversarial review of that work uncovered.
Server simulation, bot observations, collision resources and tick rate are
unchanged: the server_physics_parity trace is byte-for-byte identical to
HEAD across 360 ticks including both ships' full observation vectors.

4.11 - prediction history filed under the ISSUING sequence

_send_local_input filed each post-step predicted state under the timeline's
estimate of the sequence the server would consume this tick, trailing
issuance by input_lead. The body had integrated the intent issued under
_input_seq, so predicted[S] held "state after the intent from now" while
the server's authority for S is "state after action(S)". They agree only
while the stick is still. Filing under _input_seq costs nothing: which
action the ship uses is decided in LocalNetShipController.get_action() and
is untouched.

Every prior Phase 4 gate held its input steady, and a steady input cannot
falsify a sequence label - the 60s runs honestly reported marker=0/3784.
New --exercise-input-transitions role toggles thrust every 6 ticks; it is
the only gate that can catch a label regression. Verified non-vacuous: the
old label fails it at 50%.

4.12 - issued-but-unsimulated sequences, and the release path

An attack (delta > 1) issues and sends several sequences for one local
physics step. Those gap sequences had no recorded prediction, so a server
ack of one reported missing_not_recorded - indistinguishable from ring
loss, costing a teleport and resync suppression several times a minute.
They are now recorded stateless via record_unsimulated() and answered with
a new "skip" decision mode. Free-flight hard snaps: 25/8/4 -> 0/0/0.

A release (delta == 0) re-recorded at the unchanged _input_seq, filing the
current intent under a sequence that went out carrying a different action;
LocalInputTimeline deliberately refuses to mutate an issued sequence, so
the ring contradicted the wire. Recording is now skipped on release ticks.

4.13 - two Phase 3 bugs silently killing player input

(a) InputJitterBuffer.consume() advanced last_applied_seq on every tick
including a starve. Since ingest() discards seq <= last_applied_seq, one
starve on a sequence the client had not sent yet stranded the stream one
ahead of arrivals permanently - both sides advancing in lockstep, every
honest packet discarded on arrival. The client's own input_lead release is
enough to trigger it, so input died for ~30 ticks roughly every 6.5s on a
clean LAN. Now only gives up on a sequence once strictly newer data proves
it lost. Silent-client stall and ring-overflow resync are unchanged.

(b) The seq-range guard bounded incoming seq against highest_ingested_seq,
which only advances inside ingest(), which that guard gates. After a ~2s
host hitch every packet was rejected forever with no diagnostic (600+
consecutive rejections reproduced via SIGSTOP). Third iteration of this
guard; each previous version bounded against a value only the accepted
path could advance. Adds an escape after 10 consecutive rejections, which
grants an attacker nothing the rate limiter does not already bound.

(c) The transitions gate reported PASS at 3.76% while input was completely
dead, because suppression stops _record_metrics - a worse outage yields
fewer samples and a LOWER rate. Now scales the required sample count with
run length and asserts the wire's server_stalled bit. Reverting both fixes
makes it fail at samples 292/600, server_stalled=true, input_lead=12.

Fixing (a) also explained a residual the review had already traced: 151 of
151 action-marker mismatches were the server repeating a stale action on a
starve, not a prediction defect. Marker is now 0.00% in all three
conditions (was 1.7-2.5%), and free-flight p99 improved to
0.141/0.168/0.154m from 0.170/0.176/0.184m.

Two pre-existing test defects fixed alongside: the ball gate asserted
RTT-masking on a link with no RTT (flaked 2 in 5; now asserted only at
rtt >= 20ms, 5/5 under latency), and the two-bot CI compared scores across
a 3-5s window (now polls the scores the server actually held; note
score_changed is emitted only on the client path).

QA: 72 unit tests; 60s free-flight at LAN/80+-20ms/5% loss; transition
gate in all three; 2.0s and 3.5s host-freeze recovery; ball contact x5;
two-bot CI x3; all three abuse roles; net/match_net/clock/lobby smokes.

Phase 4 sign-off still pending a human playtest at ~100ms RTT - the
milestone asks how it feels, which no gate here answers.
This commit is contained in:
Josh Creek
2026-08-21 09:17:19 +01:00
parent 3d3024ae8a
commit 75f485667b
70 changed files with 2212 additions and 272 deletions
+137 -16
View File
@@ -42,20 +42,31 @@ const NetBodyState = preload("res://scripts/net_body_state.gd")
# caller wanting to reject "matched but ancient" needs to separately
# check newest_recorded_seq - seq itself.
#
# 2. record() can be called twice for the same seq with a DIFFERENT action,
# when input_lead_controller's release path resends a duplicated seq
# (delta == 0) the later call silently overwrites the ring slot, so
# the stored action becomes whichever of the two calls happened last.
# This matches what the WIRE ends up sending for that seq (the resend
# replaces the redundancy history's front entry — see
# networked_match.gd's _send_local_input), but if the SERVER had
# already consumed the seq from the first packet before the resend
# arrived, the server's applied action and this ring's stored action for
# that same seq can disagree. Narrow (release only fires after 120 ticks
# of sustained surplus depth, when the server is least likely to be
# right on the edge of consuming that exact seq) but real; a future
# replay-based catch-up (task 4.5) built on this history should not
# assume the stored action is provably what the server actually applied.
# 2. HISTORICAL, now fixed — kept because the reasoning still constrains
# callers. record() used to be called twice for the same seq with a
# DIFFERENT action on the release path (delta == 0), the later call
# silently overwriting the slot. That was wrong, not merely imprecise:
# LocalInputTimeline.issue() deliberately does NOT mutate _actions[seq]
# for an already-issued sequence ("may be in flight or consumed"), so
# the overwrite made this ring contradict the wire — it claimed an
# action for S that was never sent for S. networked_match.gd now skips
# recording entirely on a release tick, leaving the original (correct)
# predicted[S] in place. Callers must keep it that way: an already-
# recorded sequence's ACTION is immutable here, exactly as it is in the
# timeline. Only overwrite_state()/rebase_state_range() may revise an
# entry, and only its state.
#
# 3. A sequence can be ISSUED without ever being locally SIMULATED. The
# input_lead controller's attack path (delta > 1) skips sequence numbers
# to buy server-side buffer margin: those gap sequences are filled with
# repeat-last actions and sent, but the client took exactly ONE physics
# step that tick, so no post-step state exists for them. They are
# recorded via record_unsimulated() and report "unsimulated_gap" rather
# than "missing_not_recorded" — a routine consequence of this client's
# own lead control, NOT evidence of history loss, and specifically not a
# hard-snap condition. Distinguishing them matters: treating them as
# missing history teleported the ship and armed resync suppression
# several times a minute during ordinary play.
const RING_SIZE := 128
@@ -76,11 +87,23 @@ func _init() -> void:
_ring_seq[i] = -1
# A reset starts a new authoritative epoch. Retained inputs/states describe
# the old world and must never be compared to the new kickoff state.
func begin_epoch() -> void:
for i in RING_SIZE:
_ring_seq[i] = -1
_ring_entry[i] = null
_has_recorded = false
newest_recorded_seq = -1
last_acknowledged_seq = 0
resync_required = false
# Stores a private copy of both action and state. Returns true when this
# record crossed the unacknowledged-capacity boundary; the caller does not
# need that return today, but it makes the eviction event observable rather
# than silent when reconciliation starts applying corrections in Phase 4.3.
func record(seq: int, action: ShipAction, state: NetBodyState) -> bool:
func record(seq: int, action: ShipAction, state: NetBodyState, contact_window: bool = false) -> bool:
var overflowed_now := false
if not _has_recorded or seq > newest_recorded_seq:
if seq - last_acknowledged_seq > RING_SIZE:
@@ -100,6 +123,37 @@ func record(seq: int, action: ShipAction, state: NetBodyState) -> bool:
_ring_entry[idx] = {
"action": action.copy(),
"state": state.copy(),
"contact_window": contact_window,
"unsimulated": false,
}
return overflowed_now
# Records a sequence that was issued and sent but never locally simulated —
# an attack's skipped sequence numbers (see note 3 in this file's header).
# It advances the same newest/overflow bookkeeping record() does, because the
# sequence genuinely is outstanding and the server will genuinely acknowledge
# it; only the post-step state is absent, because the client never computed
# one. Deliberately carries the action anyway: it is what went on the wire, so
# a caller diagnosing an acknowledgement still has the honest command, and
# nothing here has to invent a state to keep the ring dense.
func record_unsimulated(seq: int, action: ShipAction) -> bool:
var overflowed_now := false
if not _has_recorded or seq > newest_recorded_seq:
if seq - last_acknowledged_seq > RING_SIZE:
overflowed_now = not resync_required
resync_required = true
if overflowed_now:
overflow_count += 1
newest_recorded_seq = seq
_has_recorded = true
var idx := posmod(seq, RING_SIZE)
_ring_seq[idx] = seq
_ring_entry[idx] = {
"action": action.copy(),
"state": null,
"contact_window": false,
"unsimulated": true,
}
return overflowed_now
@@ -111,13 +165,68 @@ func get_prediction(seq: int) -> Dictionary:
if _ring_seq[idx] != seq:
return {}
var entry: Dictionary = _ring_entry[idx]
if bool(entry.get("unsimulated", false)):
# No state to hand back — see note 3. Callers must check this flag
# before touching "state"; it is null, not a zeroed NetBodyState,
# specifically so a caller that forgets fails loudly instead of
# silently comparing against the origin.
return {
"seq": seq,
"action": (entry["action"] as ShipAction).copy(),
"state": null,
"contact_window": false,
"unsimulated": true,
}
return {
"seq": seq,
"action": (entry["action"] as ShipAction).copy(),
"state": (entry["state"] as NetBodyState).copy(),
"contact_window": bool(entry.get("contact_window", false)),
"unsimulated": false,
}
# Reconciliation changes the state paired with already-sent input, never the
# input itself. This is deliberately a no-op for an absent/skipped sequence:
# input-lead control permits sparse sequence numbers, so there is no honest
# action to invent for such a slot.
func overwrite_state(seq: int, state: NetBodyState) -> bool:
var idx := posmod(seq, RING_SIZE)
if _ring_seq[idx] != seq:
return false
var entry: Dictionary = _ring_entry[idx]
if bool(entry.get("unsimulated", false)):
# Writing a state here would manufacture a local prediction for a
# sequence this client never simulated, which is exactly the fabricated
# history §4.4 forbids. The slot stays stateless.
return false
entry["state"] = state.copy()
return true
func overwrite_state_range(from_seq: int, to_seq: int, state: NetBodyState) -> void:
for seq in range(from_seq, to_seq + 1):
overwrite_state(seq, state)
# Carries an authoritative same-sequence correction through the retained
# future. This is intentionally a transport operation, not a synthetic
# physics replay: the live Jolt body has already advanced through the real
# contact world, and a soft correction must not leave its later comparisons
# describing the old trajectory.
func rebase_state_range(from_seq: int, to_seq: int, position_delta: Vector3, rotation_delta: Quaternion, linear_velocity_delta: Vector3, angular_velocity_delta: Vector3) -> void:
for seq in range(from_seq, to_seq + 1):
var prediction := get_prediction(seq)
if prediction.is_empty() or bool(prediction.get("unsimulated", false)):
continue
var state: NetBodyState = prediction["state"]
state.position += position_delta
state.rotation = (rotation_delta * state.rotation).normalized()
state.linear_velocity += linear_velocity_delta
state.angular_velocity += angular_velocity_delta
overwrite_state(seq, state)
# Produces comparison data only. Applying a snap, teleport, velocity delta,
# or visual offset belongs to later Phase 4 tasks.
func compare_authoritative(seq: int, authoritative: NetBodyState) -> Dictionary:
@@ -148,6 +257,18 @@ func compare_authoritative(seq: int, authoritative: NetBodyState) -> Dictionary:
if newest_recorded_seq - last_acknowledged_seq <= RING_SIZE:
resync_required = false
if bool(prediction.get("unsimulated", false)):
# Reaching this sequence at all proves the acknowledgement clock is
# healthy — the entry is present and correctly tagged — so the
# resync_required clear above still applies. There is simply nothing
# to compare, because the client never simulated this sequence.
return {
"status": "unsimulated_gap",
"seq": seq,
"action": prediction["action"],
"authoritative_state": authoritative.copy(),
}
var predicted_state: NetBodyState = prediction["state"]
var position_error := authoritative.position - predicted_state.position
var rotation_error_radians := predicted_state.rotation.angle_to(authoritative.rotation)
@@ -163,6 +284,7 @@ func compare_authoritative(seq: int, authoritative: NetBodyState) -> Dictionary:
"rotation_error_degrees": rad_to_deg(rotation_error_radians),
"linear_velocity_error": authoritative.linear_velocity - predicted_state.linear_velocity,
"angular_velocity_error": authoritative.angular_velocity - predicted_state.angular_velocity,
"contact_window": bool(prediction.get("contact_window", false)),
}
@@ -170,4 +292,3 @@ func _missing_status(seq: int) -> String:
if _has_recorded and seq <= newest_recorded_seq - RING_SIZE:
return "missing_evicted"
return "missing_not_recorded"