Files
CosmicClash/multiplayer-next.md
T
Josh Creek 939b7a9584 docs(multiplayer): record WorkloadVerify closed via self-issued token
Updates §8.10 and §8.28's cross-reference in multiplayer-next.md to reflect
the previous commit: the WorkloadVerify blocker both rows named as the actual
next thing standing in the way of a working server registration/result route
is closed, via a control-plane-self-issued signed token rather than the
Kubernetes-JWT approach originally assumed necessary. Records precisely what
remains: the real delivery channel (an Agones annotation carrying a minted
token, and the supervisor reading it) and fleet.yaml's still-unaddressed
manifest wiring.
2026-09-01 14:52:07 +01:00

318 KiB
Raw Blame History

Online multiplayer — architecture and task breakdown

The single tracking document for the online multiplayer effort: architecture decisions, the wire format, current progress, and a numbered task breakdown with checkboxes, all in one place. TODO.md points here for anything multiplayer-related.

How to use this doc: start at §0 for what's outstanding right now. Pick up a single numbered task, do it, verify it against its stated acceptance criterion, mark it [x] DONE, and stop. Sections 16 are the decisions those tasks assume; read them before picking up work in Phase 2 or later. §9 is a running gotchas list — check it before debugging something that looks like a Godot/Jolt engine quirk, and add to it when you find a new one.

Status: every task in Phases 06 is implemented and verified locally. Both multiplayer milestones still need human verification — a playtest at ~100 ms RTT (Phase 4) and a real 3v3 session (Phase 5). Phase 6's public-internet gate is deliberately blocked by the display-name reclaim defect until Phase 7 identity work lands; its export, Docker, rotation/drain, and CI work are complete. Phase 7's Steam foundation is in progress. Phase 8 — matchmaking, ranked, and per-match server autoscaling — remains a 1.0 launch blocker and is partially implemented: the Go domain policy, store boundaries, migration, supervisor, hardened Fleet baseline, testkit and offline end-to-end path are in place, while production API/DB/Redis/Steam/Agones wiring and runtime gates remain. It is the first phase to add a component outside the Godot project, and its design lives in docs/MATCHMAKING.md. The client has local-ship delta-rebase reconciliation, client-only ball touch prediction, adaptive input-depth signalling, and experimental remote present-time visuals; all server simulation, bot action/observation behaviour, collision resources, and tick rate remain unchanged. See the outstanding list immediately below for what is left and why, §7 for the implemented work and its evidence, and §11 for what is deliberately deferred.


0. Outstanding work — the short list

The one place to look before planning. Everything here is also written up where it belongs; this is the index, not the detail. Phases 05 contain no unfinished tasks.

Phase 8 (matchmaking, ranked, per-match server autoscaling) is a 1.0 launch blocker and is in progress. It is larger than anything below and adds a backend service outside the Godot project. Tasks 8.18.53 are in §7; the design is in docs/MATCHMAKING.md. Three findings would break a naive implementation:

# Finding Why it bites
Task 8.28 Godot's stdout is block-buffered off a TTY — a detached container logs nothing, so server_started never appears Fixed: deploy/cosmic-clash-server now wraps the exec in stdbuf -oL -eL. Verified live — a real docker run -d container showed zero log output for 20+ seconds, including the startup line, and docker stop's SIGTERM lost it permanently rather than delaying it (Godot has no SIGTERM hook); the wrapped launcher shows the startup line within 3s of the same scenario. This affected the already-shipped community server (Docker and native systemd both route through this script), not only the not-yet-built Agones path Process-ready must be an explicit Agones call after static validation/listen, independent of this fix — the API/registration boundary never depended on log output either way, so this was a real operational bug (silent docker logs/journalctl), not a correctness gap in the process-ready design
Task 8.29 --port defaults to 7777 and the Dockerfile hardcodes EXPOSE 7777/udp Several matches need Agones dynamic UDP/SDR ports; L7 ingress does not route this traffic
Task 8.48 compose.phase6-smoke.yml hardcodes the port, first-come slots and --max-matches=2 The allocated flow needs its own fixture so Phase 6 behavior and invocations stay unchanged

Blocking sign-off — the work exists, the verification does not

# What Why it is not done Detail
A Phase 4 human playtest at ~100 ms RTT. Does the ship feel local? Does the ball? Do contact corrections read as bumps or as glitches? Needs hands on a controller. Every numeric gate is green; feel is the milestone's actual subject and no percentile can answer it. Phase 4 gate
B Phase 5 3v3 gate: a full start-to-finish match with 6 players, a mid-match disconnect, and a late joiner. Needs a real multi-client session. Every scenario is verified at 1v1 plus a two-bot CI match; nothing has run at 3v3. Phase 5 gate

These two are independent and can be done in either order, but B is the cheaper of the two to arrange and would also exercise A's conditions incidentally.

Known defects

# What Severity Detail
C Slot reservation and takeover are keyed on display name alone. Any peer connecting with a departed player's name inside the 30 s window claims their slot, ship and team. Real, demonstrated. Bounded by needing a genuine disconnect to race. §11
D Input is still lost at the transport layer during a long server stall, variably — 7 of 8 runs measured 0.00 % of the sequence stream missing, the eighth 23.54 %. Low. Distinct from the rate-limiter cause, which is fixed. The seq-guard resync visibly recovers it. Phase 5 notes
E A second Unable to send packet on channel N stderr race, in _broadcast_snapshot rather than the fixed site in _remove_player. Fixed. Server-side abuse disconnects invalidate the peer before closing it, and snapshot sends re-check that invalidation at the transport boundary. §11

C is the one to plan around: it is fixed for free by task 7.4 (Steam auth tickets in hello), which is why it has not been given a bespoke solution. Anything that ships to strangers before Phase 7 needs it addressed first.

Open architectural question

# What Detail
F A contact-cohort-only shadow world. The remaining known prediction weakness is the contact cohort. Whether it is worth a client-side shadow Jolt world scoped to contacts alone is undecided — and deliberately so until A supplies the felt evidence. Phase 4 notes

Unstarted phases

  • Phase 6 external gate: run the exported Docker server and clients from separate machines over the internet, then play a full match. This is a controlled test only until C is fixed.
  • Phase 7 — Steam transport, browser, identity and production SDR (8 tasks): the optional bootstrap and NetTransport foundation now exist, but custom Steam client/server templates have not yet been supplied. Browser, verified tickets, bans, production credentials and ticketed Hosted Dedicated Server SDR await a project-owned Steamworks App ID and Valve coordination. Carries the fix for C and is the hard prerequisite for Phase 8.

Phase 6 has no dependency on Phase 7 and now turns a two-terminal game into something another person can host. Phase 7 is the next block because Steam identity is required before public exposure.

Deferred by choice, not forgotten

120 Hz simulation, the latency-gap measurement (task 4.9's acceptance criterion), audio hooks, split-screen — all in §11 with what each would buy and cost.


1. Architecture decisions

1.1 Locked decisions

# Decision Why
1 Server-authoritative simulation, with client-side prediction of the local ship and ball. No world rollback / resimulation. Jolt is not bit-deterministic across platforms or across differing contact orderings, and Godot exposes no world snapshot/restore API. Rollback netcode would be a research project.
2 Dedicated servers only. Headless Godot export; the server is never a player. Fair for every player, no host advantage. Self-hostable community servers first, so nothing is blocked on paid infrastructure.
3 ENet first, GodotSteam later, behind a boundary. ENet works in-editor, headless, on LAN, and in CI with no Steam client. Direct-IP connect stays permanently supported and must never become the degraded path.
4 Community discovery uses no custom backend; superseded for queued play by Phase 8. Steam's server APIs remain enough for the community browser. Casual/ranked queues, durable ratings, allocation and authoritative results require the project-owned Go control plane specified in docs/MATCHMAKING.md; it does not replace the browser or direct-IP path.

1.2 Rejected alternatives

  • Peer-authoritative ships (each client owns its own transform). Easiest to build, feels perfect locally, and is trivially cheatable — it directly contradicts README.md's stated anti-cheat position. Ship-vs-ship collisions also become ambiguous with no arbiter.

  • Deterministic lockstep / rollback. See decision 1.

  • MultiplayerSynchronizer / MultiplayerSpawner. The decisive objection is not bandwidth. It is that last_processed_input_seq must arrive in the same packet as the state it describes, or reconciliation is off by a snapshot — and a synchroniser gives you nowhere to put it. It also writes replicated properties directly onto the node, which is exactly wrong for a RigidBody3D under prediction: incoming state has to enter a compare-against-history pipeline, not be stamped onto global_transform. You would end up building the correction pipeline anyway, with the synchroniser as pure overhead. Secondary objections: one packet per body (7 bodies × ~50 B of UDP/IP/ENet framing versus one coalesced snapshot), no per-field quantisation, and no client-side interpolation.

    MultiplayerSpawner is unnecessary for a separate reason: the roster is fixed at match start and fully described by the match_config message, and no ship is ever despawned (§6.4).

  • Seeded RNG for kickoff jitter. Shared-seed determinism requires both sides to consume the RNG stream in exactly the same order forever. The first randf() anyone later adds anywhere in the reset path — a spawn VFX variation, a cosmetic, a commentary line — silently desyncs kickoff positions with no error message. The server broadcasts the resulting transforms instead: 336 bytes, once per kickoff, cannot rot.

1.3 Derived decisions

All hot-path RPCs live on autoloads. /root/NetworkManager and /root/MatchNet exist at identical paths on every peer regardless of which scene is loaded, which side is headless, or whether a client is mid-scene-transition. This deletes the entire "NodePaths must match across peers" class of bugs, kills a family of late-join races where an RPC arrives before its target node exists, and warms Godot's RPC path cache once at connect so it never re-sends a full path on scene change.

Entities are addressed by integer slot, never by path. The snapshot is [slot 0..N-1] in a fixed order established by match_config. MatchNet holds an Array[Node] _slots populated at spawn.

One server process hosts exactly one match. This is forced, not chosen: ship.gd:162 resolves the arena boundary via get_tree().get_first_node_in_group("arena_boundary") and ai_ship_controller.gd discovers its roster via get_tree().get_nodes_in_group("ship"). Both are tree-global, so two matches in one scene tree would cross-wire instantly. It is recorded here because it determines the RAM figure in §1.4.

1.4 Server sizing — bandwidth and CPU are not the constraint

Worth establishing up front, because §2 and §3 repeatedly trade bandwidth for latency and somebody will eventually want to trade back.

ArenaBoundary.bake_colliders() generates roughly 168 box colliders (corner fillets, base wrap, ceiling, end walls) plus the scene's own slabs, 2 goal backstops, 2 Area3D sensors, and 7 dynamic bodies (the ball with continuous_cd). Estimated per-tick cost:

Component ms/tick
Jolt step 0.15 0.4
Godot headless main loop 0.1 0.3
Bot inference, amortised (see task 0.8) ~0.3
Total, of a 16.7 ms budget 0.6 1.1

~610 concurrent matches per modern core, ~150250 MB RSS per process. 100 concurrent matches ≈ 1216 cores and ~20 GB — a single mid-tier VPS. Upstream bandwidth for a full 6-player match is ~630 kbit/s (§2.4).

Neither CPU nor bandwidth is scarce. Latency is. Optimise accordingly.


2. Wire format

Two peers must agree byte-for-byte, so this is specified rather than sketched.

2.1 Channels

Channel Transfer mode Contents
0 reliable handshake, match_config, kickoff, goal, clock, state changes, chat, admin
1 unreliable-ordered client → server input
2 unreliable-ordered server → client snapshots

Unreliable-ordered (ENet sequenced-unreliable, drops stale) rather than plain unreliable for both hot paths: we carry explicit sequence numbers, and a reordered late packet is worthless work. Separating them stops a large reliable match_config from head-of-line-blocking state on a lossy link.

Verify at implementation time. Godot's ENetMultiplayerPeer reserves low ENet channels for its own system messages and offsets transfer_channel on top. The intent above is "three logically distinct channels"; the concrete indices may need an offset. Confirm empirically, don't assume.

Set ENetMultiplayerPeer.server_relay = false. It defaults to true, which lets any client rpc() any other client through your server. With it off, clients can only talk to peer 1. Single highest-value one-line security change in this document.

2.2 Packet header

Every hot-path packet opens with a 1-byte type + version. A capture then decodes standalone, and a mismatched build fails loudly instead of decoding garbage straight into state.transform.

Hot paths carry a single PackedByteArray RPC argument (≈14 B of Godot RPC framing once the path cache is warm). Control messages on channel 0 use normal typed arguments — they are rare and readability beats bytes.

2.3 Input packet — client → server, channel 1, 60 Hz

u8   type_version
u32  seq                 server-tick-space sequence of the NEWEST action
u8   count               1..4 (MAX_REDUNDANCY)
u32  ack_snapshot_tick   newest snapshot tick this client has processed
u16  client_send_ms      wrapping ms clock, echoed back for RTT
--- repeated `count` times, newest first ---
i8   thrust_x, thrust_y, thrust_z    value = clamp(round(v*127), -127, 127)
i8   rot_x, rot_y, rot_z
u8   flags                           bit0 = turbo

12 + 7×4 = 40 B payload, ~90 B on the wire with UDP/IP/ENet framing → ~43 kbit/s up per client.

  • Redundancy 4 is what makes an unreliable input channel safe: starvation requires four consecutive losses (~66 ms).
  • i8 per axis, not 3-bit bins. Bins matching ShipActionCodec.HEADS would cut an action to 3 bytes, but they permanently foreclose analog gamepad sticks, which this game will want. round(v*127)/127 round-trips -1/0/+1 exactly, so today's digital input (player_ship_controller.gd is is_action_pressed-only) is lossless.
  • The encoding is itself a validator. i8/127 cannot express NaN, Inf, or a value outside [-1.008, 1.008]. Half of "sanitise untrusted client input" is solved by not using Variant encoding.

2.4 Snapshot — server → client, channel 2, 60 Hz default

Per-client header built per peer; body buffer built once per tick and reused across peers.

--- per-client header (7 B) ---
u32  last_input_seq        newest input from THIS client the server has applied
i8   input_buffer_depth    jitter-buffer occupancy; negative = starved
u16  echo_client_send_ms   from that input packet, for RTT

--- shared body header (8 B) ---
u8   type_version
u32  server_tick           Engine.get_physics_frames() on the server
u8   match_state           see §6.1
u8   reset_gen             increments on every authoritative teleport
u8   body_count

--- repeated body_count times, slot order fixed by match_config (22 B each) ---
i16  pos_x, pos_y, pos_z         range ±64 m   -> 1.95 mm
i16  quat_x, quat_y, quat_z      w = ±sqrt(1-x²-y²-z²), sign in flags
i16  vel_x, vel_y, vel_z         range ±64 m/s -> 1.95 mm/s
i8   avel_x, avel_y, avel_z      ships ±4 rad/s; ball ±32 rad/s
u8   flags                       bit0 frozen, bit1 turbo, bits2-4 thrust_z bin,
                                 bit5 stalled, bit6 quat_w sign

7 bodies → 8 + 7 + 7×22 = 169 B payload, ~219 B on the wire.

per client down server up, 6 clients + 10 spectators
60 Hz 105 kbit/s 631 kbit/s 1.68 Mbit/s

MTU headroom is ~6× (ENet fragments above ~1400 B); a hypothetical 10v10 at 21 bodies is 477 B and still fits. This format does not need delta compression.

Plain i16 quaternion components, not smallest-three. Smallest-three saves 4 B/body and is the textbook answer. It is also exactly where a hand-rolled codec goes subtly wrong — off-by-one in the 2-bit index, sign of the dropped component, renormalisation drift — in a project that has no test framework yet. Three i16s plus a sign bit give ~3e-5 rad with no bit-shifting, for 2 B/body (≈3 kbit/s). Take the bytes.

Quantisation ranges derive from constants, not from prose. ArenaBoundary.INNER_HALF_X = 18.0, INNER_HALF_Z = 27.0, INNER_HEIGHT = 18.0 (arena_boundary.gd:8-10) plus GameMode.ESCAPE_MARGIN = 15.0; Ship.max_speed = 35.0 (ship.gd:16); Ball.MAX_SPEED = 32.0 (ball.gd:17).

CLAUDE.md's Architecture section states the play volume as "inner x ±12, z ±18, height 12, goal lines z ±17". That is stale — see the real constants above. Task 0.13 fixes the doc.

The flags byte must carry turbo and a 3-bit thrust_z bin. _integrate_forces is not called on frozen bodies, so remote ships on a client never pull get_action(), and Ship._update_movement_vfx() (ship.gd:293) reads _current_action.thrust.z and turbo. Without those bits, every remote ship flies with dead engines.

2.5 Reliable control messages, channel 0

hello · welcome · player_joined · player_left · ready_state · match_config · scene_ready · kickoff · state_change · goal_scored · clock_state · match_ended · chat · server_shutdown.


3. Server-side input handling

Per-player server state:

class PlayerSlot:
    var peer_id: int
    var slot: int                   # snapshot index
    var ring: Array[ShipAction]     # FIXED 32 entries, indexed seq % 32
    var ring_seq: PackedInt32Array  # 32 entries, seq stored at each index (-1 = empty)
    var last_applied_seq: int
    var last_action: ShipAction
    var starved_ticks: int
    var packets_this_second: int
    var remote_controller: RLShipController   # see §7 task 5.7 — null on takeover

3.1 Ingestion

@rpc("any_peer", "unreliable_ordered", channel = 1), in order:

  1. multiplayer.get_remote_sender_id() → look up slot. Unknown sender → drop and count.
  2. Rate limit. packets_this_second > 110 (60 Hz × 1.5 + 20) → drop. Three consecutive seconds over budget → disconnect with RATE_LIMIT. Same for a byte budget.
  3. Framing. count > 4 or payload_size != 12 + count*7 → drop, count malformed. 20 malformed → disconnect.
  4. Sequence range. seq > server_tick + 20 → drop. (Not 120: input_lead is clamped to 12, so anything above ~20 is broken or hostile.) This is why the ring is fixed-size and indexed seq % 32a client can never make the server allocate.
  5. For each action, newest first at descending seq: seq <= last_applied_seq → discard (already consumed); else write ring[seq % 32].
  6. Decode with per-axis clamp only:
    action.thrust = Vector3(b[0]/127.0, b[1]/127.0, b[2]/127.0).clampf(-1.0, 1.0)
    

Never normalise the thrust vector. A player holding W+A+E legitimately produces thrust = (1,1,1), length 1.73, and each axis uses a different power constant — thrust_power 150, maneuvering_thrust 75, vertical_thrust 120 (ship.gd:12-14). Normalising would silently change the flight model for honest players. Per-axis clamp combined with the i8 encoding is complete validation: the reachable value space is exactly what a legitimate client can produce.

3.2 Consumption — once per server physics tick, before the step

expected = last_applied_seq + 1
if ring holds expected:
    action = ring[expected % 32];  starved_ticks = 0
else:
    action = last_action           # REPEAT — do not zero
    starved_ticks += 1
    if starved_ticks > 30:         # 500 ms
        action = ZERO_ACTION;  flags.stalled = true
last_applied_seq = expected
last_action = action
remote_controller.action = action

Repeat-last, not zero. Player inputs are heavily autocorrelated at 60 Hz — the odds that a held thrust was released on exactly the dropped tick are low, and the client predicted with the real input either way, so repeating minimises expected divergence. It is also consistent with AIShipController, which already holds its action between decisions. Zeroing after 500 ms stops a disconnecting player's ship flying into a wall at full throttle forever.

3.3 Jitter buffer — one control loop, not three

An earlier draft had the server adapting target_depth, the server fast-forward-dropping queued actions, and the client slewing input_lead. Three integrators acting on one plant (buffer occupancy) with different time constants is a textbook oscillation; on a jittery link it hunts, and it presents to the player as intermittent sticky controls that are nearly impossible to attribute.

The server reports input_buffer_depth in every snapshot and does nothing else adaptive. The client owns input_lead exclusively.

  • target_depth = 1 (16.7 ms), not 2. With redundancy-4 you have already bought the insurance depth 2 provides; depth 2 is 16.7 ms of pure input latency for nothing.
  • Client input_lead clamp [1, 12], fast attack / slow release: on any starve, increase by up to 3 immediately; decrease by 1 per 60 ticks only after 2 s of clean surplus. A symmetric ±1-per-500 ms slew takes two seconds to absorb a wifi spike, during which the player steers and the ship does not turn — the most rage-inducing failure mode in any netcode.
  • Changing input_lead means skipping or duplicating one tick's sequence number. Never change it more than once per 30 ticks.

Enforce input_lead server-side from observed arrival times. A client that fakes starvation to drive input_lead to 1 gets its inputs applied with less server-side buffering than honest players — a small but real responsiveness edge. The i8 encoding does nothing about this; only observing actual arrival timing does.


4. Prediction and reconciliation

4.1 Two clocks for remote entities — the load-bearing correction

The obvious design runs remote ships and the ball as frozen kinematic proxies at server_time_est - INTERP_DELAY while predicting the local ship to now. That is wrong, and it is wrong in a way that only shows up over real latency:

  • Two ships closing at 50 m/s put the opponent's collider 3.5 m from truth. The hull is a BoxShape3D of (1.6, 0.6, 4) (ship.tscn:12) — that is most of a ship length of positional lie.
  • A fast ball is 2.2 m off against a 0.5 m radius — four ball diameters.
  • ship.tscn:16 has collision_mask = 7: ships collide with ships, the ball, and the arena. Ship-vs-ship contact is constant in vehicle soccer, not incidental.

So prediction would not diverge occasionally due to timing noise. It would diverge deterministically and in the same direction on essentially every contact, and the hard-snap threshold would become the steady state rather than a backstop.

Fix: separate the collider clock from the render clock.

runs at why
remote body collider server_time_est, extrapolated forward from the newest snapshot by ~one-way + half a snapshot interval Extrapolation error over ~45 ms at real accelerations (thrust_power 150 / mass 5 = 30 m/s², 75 m/s² on turbo — ship.gd:12,15, ship.tscn:17) is ~0.030.08 m. Two orders of magnitude better than 3.5 m.
remote $Visual server_time_est - INTERP_DELAY Smooth, jitter-free rendering.

This is the same trick applied to the local ship, pointed the other way. It costs one extra transform write per remote body per tick.

4.2 Where each piece lives

Concern Location
sample + send input LocalNetShipController._physics_process — runs before the physics step, guarantees exactly one sample/tick
record predicted state same, at top of tick N (state = result of N1)
apply velocity / teleport correction Ship._integrate_forces, ~15 guarded lines — the only Jolt-safe place to write state.transform / state.linear_velocity
visual smoothing Ship/$Visual.global_transform, set in _physics_process
snap-vs-blend decision net_ship_predictor.gd (child node)
remote bodies net_interpolator.gd

4.3 Per-tick, own ship

  1. predicted[current_tick - 1] = {transform, linear_velocity, angular_velocity} — ring of 128.
  2. var a := _player.get_action().copy()must copy. player_ship_controller.gd reuses a single ShipAction across ticks (its own header warns about this); buffering it aliases every history entry to the same object. See task 0.1.
  3. _action = a, returned by get_action() this tick so Ship._integrate_forces samples input exactly once.
  4. input_history[seq] = a, seq = predicted_server_tick + input_lead.
  5. Build and send the packet with the last 4 entries.

Ship._integrate_forces then runs completely unchanged.

4.4 On snapshot arrival

A = last_input_seq
if reset_gen changed OR predicted[A] missing OR flags.frozen != local frozen:
    HARD SNAP
else if pos_err > 2.0 m OR rot_err > 60°:
    HARD SNAP
else:
    SOFT CORRECT

Comparing server state at tick A against predicted[A] — the client's own state at that same tick — makes the delta latency-free by construction. That is the entire reason for keeping the prediction ring, and it is why this works acceptably without resimulation: never blend current state toward stale state.

SOFT CORRECT

  • Velocity: applied in full, immediately. net_vel_correction += (srv.linvel - predicted[A].linvel), consumed once in _integrate_forces. Velocity error is invisible to the player but is the cause of future position error; blending it just prolongs divergence.
  • Position/rotation: physics moves in full, rendering does not. Queue the body teleport, and simultaneously offset $Visual by the negation. Net visual movement at the instant of correction: zero. The body is where the server says; the rendered ship catches up.
  • Decay each physics tick, reusing the existing convention at ship.gd:450:
    var k := _tick_scaled(0.88, delta)   # 63% gone in ~130 ms, 95% in ~280 ms
    
  • MAX_VISUAL_OFFSET = 0.4 m, not 2.0. The hull is 4 m long; a 2 m offset means being rendered half a ship-length from your own collider for ~280 ms, so you clip walls you visibly cleared — a felt bug in a game built around wall-riding. Beyond 0.4 m, show the correction. A visible correction is honest; an invisible 2 m lie is not.

HARD CORRECT

  • Apply the same sequence-matched authoritative pose and velocity delta to the current local body, reset body and $Visual interpolation, and clear the visual offset. It is physically the same correction as soft correction; only its presentation differs.

Settled Phase 4 decision — delta transport, not one-body replay. For every matched snapshot, overwrite predicted[A] with authority, transport its pose and linear/angular-velocity delta through each retained state A+1..current, and apply that same delta once to the live local Jolt body. This keeps retained history coherent, so a later snapshot does not correct an already-corrected pre-delta trajectory a second time.

Do not analytically replay stored actions. That approximation cannot reproduce Jolt integration or contact manifolds (friction, restitution, walls, ships, and ball), therefore it becomes least trustworthy exactly where reconciliation is most noticeable. This is still neither whole-world rollback nor a change to server physics: it is client-only state transport around a server-authoritative simulation.

For reset generation changes, place exact authority, begin a new history epoch, and do not consume pre-reset actions. For missing or overflowed history, place authority once and suppress stale acknowledgements until a new matched sequence is recorded; never manufacture future history by filling it with one stale authority state.

Same-sequence pre-correction residual remains diagnostic telemetry. With a server input jitter buffer, it is not by itself a presentation-quality gate: the server may have integrated an action at a different physical instant from the client. Acceptance must report it separately by free-flight/contact/reset/resync cohort, while gating post-correction/presentation error and hard-snap behaviour.

That the two sides integrate the same action for a given sequence is a separate claim, and a checkable one — it is what the action marker and task 4.11's --exercise-input-transitions gate exist for. Keep the two apart: "right action, different instant" is expected here; "wrong action" is a bug, and was one.

4.5 Camera and visuals

The camera must follow $Visual, not the body. ship_camera.gd:115, :149, :150 read target.global_transform directly. Left as-is, every soft correct makes the camera jump the full error while the mesh smoothly lags — strictly worse than snapping, because the world lurches around a player whose ship slides inside the frame.

And it must read $Visual.get_global_transform_interpolated() from _process, not global_transform from _physics_process (task 0.16, rationale in §5.4). Node3D.get_global_transform_interpolated() exists precisely for a camera tracking a physics-interpolated body; global_transform returns the last physics tick's pose, so a _process camera reading it would chase a 60 Hz staircase at 240 fps.

Ordering hazard, straight from the engine docs: get_global_transform_interpolated() "creates an interpolation pump on the Node3D the first time it is called, which can respond to physics interpolation resets… be sure to call it at least once before resetting the Node3D physics interpolation." Every hard snap calls reset_physics_interpolation() on $Visual. Prime the pump when the camera's target is assigned, not lazily on the first frame, or the first snap of the match streaks the camera.

project.godot has physics_interpolation=true, and $Visual's own local transform is interpolated too — so reset_physics_interpolation() must be called on $Visual as well as the body, or every snap smears the mesh for a frame. (This is the same artefact game_mode.gd:263 already exists to prevent.)

4.6 Remote bodies on the client

  • freeze = true, freeze_mode = FREEZE_MODE_KINEMATICnot STATIC, or Jolt cannot derive contact velocity from the per-tick transform delta and your predicted ship hits a static wall instead of a moving ship.
  • net_interpolator.gd samples the snapshot buffer (last 8 per body); collider at server_time_est (§4.1), $Visual at server_time_est - INTERP_DELAY.
  • The two samples run on different clocks and different callbacks. The collider is a physics concern: _physics_process, 60 Hz. $Visual is a render concern: _process, sampled at true render time with physics_interpolation_mode = OFF so Godot does not interpolate an already-per-frame transform. On a 240 Hz client this is 240 distinct remote-ship positions per second instead of 60, and one fewer tick of lag, for no extra cost — the buffer lerp is happening either way (§5.4).
  • INTERP_DELAY = one_way_ms + snapshot_interval * 1.5 + 2.5 * jitter_ewma, clamped [25, 200] ms. At 60 ms RTT / 60 Hz / 5 ms jitter that is 30 + 25 + 12.5 ≈ 68 ms.

The one_way_ms term is not optional, and omitting it is a silent architectural failure. server_time_est (§4.7) estimates what the server clock reads right now. The newest snapshot in hand was stamped one_way ago — §4.1 says exactly this when it extrapolates the collider forward "by ~one-way + half a snapshot interval". So rendering $Visual at server_time_est - INTERP_DELAY only interpolates if INTERP_DELAY ≥ one_way. Set it to the buffer alone (~38 ms at 60 Hz) and the render cursor lands on or past the newest sample: the bullet below about extrapolating past the newest snapshot becomes the steady state rather than the exception, and every remote entity is permanently dead-reckoned. The 25 ms clamp floor is reachable on LAN only.

  • Past the newest snapshot, extrapolate on last known velocity for at most 150 ms, then hold. Never extrapolate indefinitely — a stuck ship reads better than one flying through a wall.
  • Never write linear_velocity to a frozen body. Godot/Jolt zeroes and holds velocity on frozen bodies, so ball.gd:35's linear_velocity.length() trail driver will not work that way. Add Ball.set_visual_speed(speed) mirroring the Ship.set_visual_action(thrust_z, turbo) pattern. Don't route presentation data through a property the physics server owns.
  • Call reset_physics_interpolation() on remote bodies at every kickoff.

4.7 Clock

server_time_est = local_ms + clock_offset, clock_offset from ping/pong on channel 0 every 1 s using the minimum-RTT sample in a rolling 5 s window (the min-RTT sample has the least queueing error).

Freeze tick_offset at match start. Seed it exactly from the handshake (server_tick + round(one_way / tick_ms)) and absorb all subsequent drift into input_lead alone. The prediction ring is indexed in server-tick space, so slewing tick_offset during play silently reinterprets every historical entry and produces sporadic, unreproducible false snaps. Re-seed only across a kickoff boundary.


5. Latency and frame-rate budget

Three of the largest terms are invisible to a netcode document that only counts network hops. Record the budget so future changes are argued against a number.

Client at 60 Hz physics, 60 ms RTT, 5 ms jitter, 60 Hz snapshots. Display at 60 Hz with vsync on — the Godot default, and the worst case. §5.4 redoes the display-dependent rows for 120/144/165/240/360 Hz.

5.1 Own ship (predicted) — input to pixel

Stage ms scales with fps?
OS input → Input.is_action_pressed 10 0.5 × frame interval + device polling partly — see below
wait for next physics tick 8 avg of 016.7 no — 60 Hz physics
physics step applies force 0
Godot physics interpolation 8 physics_interpolation=true; mean, worst case 16.7 no — 60 Hz physics
render + vsync present 25 1.5 refresh intervals, vsync defaults on yes
Total ≈52

This is the existing single-player floor, unchanged by netcode — and ~43 of those 52 ms are things no netcode document discusses. A low-latency present would take it to ~35 ms (§5.4).

Two notes on the model, both corrected from an earlier draft that read ≈45:

  • Input freshness is 0.5 of a frame interval, not 0.25. Godot pumps OS input once per main-loop iteration and Ship._integrate_forces (ship.gd:347) consumes it once per physics tick; for arrivals distributed uniformly between pumps the mean staleness at the pump is half the interval. On top sits device polling, which does not scale with fps at all: ~1 ms at a 1000 Hz mouse or gamepad, ~8 ms at a 125 Hz USB device. The table assumes ~2 ms.
  • Physics interpolation's 8 ms is a mean. Rendering happens between the two most recent completed ticks, so displayed pose lags the newest state by (1 fraction) of a tick — 0 to 16.7 ms, averaging 8.3. The worst case matters for §5.4's discussion of frame-time variance.

Note the right-hand column: 16 of the 52 ms do not move no matter how many frames the client draws. That is the price of a 60 Hz simulation.

5.2 World response — the number that decides whether this ships

Stage ms
input freshness 10 0.5 × frame interval + ~2 ms device polling
wait for next physics tick 8
manual multiplayer flush ~0 ~8 with default idle-frame poll — see §7 task 1.3
client → server transit 30 RTT/2
jitter buffer, target_depth = 1 17
server tick + flush 8
server → client transit 30 RTT/2 — the return leg
interpolation buffer beyond arrival 38 interval × 1.5 + 2.5 × jitter; the one_way half of INTERP_DELAY is the row above
client physics interpolation 8
render + present 25 vsync on, 60 Hz display
World response, opponents ≈174
Ball, with local prediction ≈52 same as own ship
Both, at 144 Hz + low-latency present 148 / 26 §5.4

Correction — this table previously read ≈138 ms and omitted the server→client transit row entirely. INTERP_DELAY was quoted as 38 ms, which is the interpolation buffer measured from snapshot arrival, while §4.6 defines the render cursor relative to server_time_est — server-now. The 30 ms return leg fell between the two definitions and was never counted. §4.6's formula is corrected to include one_way; this table keeps the two terms on separate rows because that is clearer to budget against.

For reference, Rocket League runs 120 Hz physics and predicts both car and ball locally; its equivalent at 60 ms RTT is roughly 90110 ms.

≈174 ms as designed here is not competitive, and this document should not pretend otherwise. It is also not the end state: §5.6 gets to ≈127 ms with two changes that touch no graphics setting and require no bot retrain, and to ≈103 ms with 120 Hz simulation — inside the reference band. Read §5.6 before treating this table as a verdict.

What is settled is the shape of the design: a locally-predicted ball and own ship at ≈52 ms is the difference between this being playable and not, and a 30 Hz / default-poll / interpolated-ball design would land near ≈250.

5.3 Why 60 Hz snapshots, not 30

  • Interpolation buffer: the interval × 1.5 term is 50 ms at 30 Hz vs 25 at 60, on top of the one-way term both share (§4.6), plus a half-interval of cadence quantisation.
  • Interpolation fidelity: at MAX_SPEED = 32 the ball moves 1.07 m between samples at 30 Hz — more than its own diameter, so any wall bounce landing between two samples gets lerped as a straight line through the wall. At 60 Hz it is 0.53 m.
  • Cost: 300 kbit/s. Per §1.4, bandwidth is not the constraint.

Keep --snapshot-hz 30 as an explicit degraded mode.

5.4 High-refresh-rate clients — 120 / 144 / 165 / 240 / 360 Hz

Players on high-refresh displays are the ones most sensitive to everything in this document, and the current code has three places where the client draws 240 frames but only 60 of them contain new information. Those are bugs, not tuning.

What frame rate actually buys

Modelling present as ~1.5 refresh intervals with vsync on (§5.1), and input freshness as 0.5 of a frame interval plus ~2 ms of device polling:

Display present own ship / ball (§5.1) world response (§5.2) with low-latency present
60 Hz 25.0 52 174 35 / 157
120 Hz 12.5 35 158 27 / 149
144 Hz 10.4 33 155 26 / 148
165 Hz 9.1 31 153 25 / 147
240 Hz 6.3 27 149 23 / 145
360 Hz 4.2 24 146 21 / 144

This table assumes the client can actually produce those frames. It cannot — see §5.5. As configured today the project runs SDFGI, SSIL, SSAO, a 5-level glow pyramid, five shadow-casting lights, MSAA 4× and FXAA, and an unconditional full-screen backbuffer pass, none of which any player can switch off. Read §5.5 before treating any row below 60 Hz's as reachable.

Three conclusions to design around:

  1. 60 → 144 Hz is worth ~19 ms on own-ship feel. 144 → 360 Hz is worth ~9. The curve flattens hard, because 16 ms of the remaining budget is the 60 Hz physics tick plus its interpolation and does not move.
  2. A low-latency present is worth more at 60 Hz (17 ms) than the entire jump from 144 to 360 Hz. It costs one settings dropdown.
  3. Frame rate barely moves world response — 174 → 146 across the whole 60360 range, because that budget is dominated by RTT and the interpolation buffer. Frame rate is an own-ship feel lever, not a netcode one. Say this to players plainly; someone who buys a 360 Hz monitor to see opponents sooner has been mis-sold.

Three things that must run per rendered frame, not per physics tick

a. The camera rig. ship_camera.gd:86 runs the entire rig in _physics_process. Global physics_interpolation=true smooths the resulting camera transform, so this is not visible as judder — but it costs an extra tick of camera latency on top of the ship's, and two things it does are not transforms and therefore not interpolated: camera.fov (:182) and the PostFX shader parameters (:186-187). At 240 fps those step at 60 Hz, which reads as a faint pulse in the turbo FOV kick.

The rig moves to _process, reading target.get_global_transform_interpolated() (and $Visual's, post-task 0.2) instead of target.global_transform, with physics_interpolation_mode = PHYSICS_INTERPOLATION_MODE_OFF on the rig itself so Godot does not re-interpolate an already-per-frame transform.

The move is cheap but it is not tuning-neutral. Cost first: one call is ~15 engine-bound operations (2 × get_noise_1d, 2 × set_shader_parameter, Basis.looking_at, slerp, orthonormalized, signed_angle_to, rotated, several global_basis accesses) plus ~60100 bytecode ops — call it 515 µs. At 360 Hz that is 1.85.4 ms/s, under 0.5% of a core. Negligible, but negligible because the absolute work is tiny; 1-exp(-k·delta) is a correctness property, not a cost argument, and it does not license moving arbitrarily expensive code into _process.

The impact shake must be re-tuned, and in the opposite direction to what you would guess. ship_camera.gd:204 advances the noise coordinate by delta * 60.0, and :64 sets frequency = 2.5, so each sample steps delta × 150 noise units. At 60 fps that is 2.5 units per sample — simplex noise decorrelates over roughly 1 unit, so the shake is currently white noise, and physics interpolation is lerping between independent samples. At 360 fps in _process it becomes 0.42 units per sample, which is strongly correlated: the shake turns into a slow, smooth wobble that gets softer the better your monitor is. Re-derive frequency (or the * 60.0) for constant noise-units-per-second, then re-check amplitude by eye at 60 and 240 fps.

Everything else in the rig genuinely is rate-independent and needs no attention: 1.0 - exp(-k * delta) at :126, 137, 156, 172, 177 and move_toward(…, shake_decay * delta) at :212.

Two pre-existing bugs sit in the code this task touches, so fix them here rather than discovering them in Phase 5:

  • The rig has no snap path. camera.global_position is smoothed at camera_smoothing = 10.0 (:14, 137, 156) with no reset anywhere in the file. At a kickoff teleport (game_mode.gd:256-263, becoming an _integrate_forces write under task 0.15) the camera lerps across the arena over ~300 ms. Add snap_to_target() — set global_position/global_basis directly, zero _last_shake_offset — and call it from the kickoff path.
  • Shake decay stalls during a goal cut. :94-96 returns before _apply_shake, so _shake_strength's move_toward decay never runs for the length of the cinematic. Task 0.12 proposes building goal feel on exactly this system.

b. Remote-entity visuals. §4.6's interpolator samples a snapshot buffer between two known states. Driving that from _physics_process quantises every remote ship and the ball to 60 distinct positions per second and then leans on Godot to interpolate between them — an extra tick of lag for no benefit, since we are already interpolating. Sample the buffer at true render time in _process instead: 240 distinct positions per second and one fewer tick of lag.

The split is clean because the two consumers want different times anyway (§4.1): the collider is a physics concern and stays in _physics_process at server_time_est; $Visual is a render concern and moves to _process at server_time_est - INTERP_DELAY, with physics_interpolation_mode = OFF. Setting it OFF is coherent precisely because the node's global_transform is overwritten every rendered frame — there is nothing left for the engine to interpolate. Note this is the opposite of §4.5's rule for the local ship's $Visual, which is written per physics tick and therefore must stay interpolated and must be reset on snap. Same node name, two different regimes; task 0.16 lands in Phase 0 against local-ship semantics, task 2.4 adds the remote case.

It is not free, though it is cheap: per body per frame you bracket-search a ring of 8, run two Vector3.lerps and a Quaternion.slerp, build a Transform3D, and assign global_transform (which dirties and propagates to children). Estimate 36 µs per body → ~2142 µs/frame for 7 bodies, ~1.5% of a core at 360 Hz. That is 46× the work of sampling at 60 Hz. Measure it in task 0.15b rather than asserting it.

c. Receive polling. Task 1.3 already flushes sends from _physics_process. Receiving is the other half: with (b) in place, a snapshot that lands 2 ms after a physics tick can be rendered 2 ms later at 240 fps instead of waiting 14 ms for the next tick. Poll for receive unconditionally at the top of both _process and _physics_process — no rate limiter. A zero-timeout enet_host_service on an empty socket is one non-blocking recvfrom returning EWOULDBLOCK, on the order of 1 µs; 360 of those per second costs ~0.36 ms/s. An earlier draft proposed a 2 ms limiter, which is worse than useless: at 240 fps the frame interval is already 4.17 ms so it never fires, and it only engages above ~500 fps where polling was already cheaper than the limiter.

Manual polling relocates the connection signals. With set_multiplayer_poll(false), peer_connected / peer_disconnected now fire from inside your poll() call — mid-_process, during a render frame — rather than on the idle-frame boundary. Any handler that mutates the scene tree must defer.

Frame-time variance, not mean frame rate, is the real target

At 240 fps the frame budget is 4.17 ms, and physics runs at 60 Hz — so one frame in four carries the entire physics tick and must still fit in 4.17 ms. On that frame the client pays, in one go: the Jolt step over 7 dynamic bodies against a 172-shape compound; 7 × Ship._integrate_forces (ship.gd:346-357), each running apply_thruster_forces, a full ArenaBoundary.get_surface_pull with five _falloff calls (arena_boundary.gd:183-198), apply_rotation_forces, apply_righting_torque and apply_drag_and_limits with two pow() calls via _tick_scaled (:450); 6 × _update_movement_vfx (:296-315, writing two material params and two OmniLight3D energies per ship); and on decision ticks, bot inference — policy_network.gd is a pure-GDScript MLP at 31→64→64→7 ≈ 6.5k multiply-accumulates per bot, so five bots landing together is ~33k GDScript float ops in one frame.

Task 0.8's decision stagger is framed above as a cosmetic hitch. It is not — the physics tick sets a floor on 1%-low frame time that no graphics setting can lower. A game that averages 240 fps but drops one frame in four to 8 ms is not a 240 fps game. Profile p99, not mean (task 0.15b).

The same term matters at the bottom of the range, where most players actually are: see gotcha 22 and task 0.22 for the client-side Engine.max_physics_steps_per_frame cap that stops a hitching client from spiralling.

What frame rate does not buy, so nobody optimises the wrong thing

Input sampling does not improve. player_ship_controller.gd:15-38 reads seven Input.is_action_pressed calls — all digital, all held-state — and Ship._integrate_forces pulls them once per physics tick. The state read at the tick is the freshest state; sampling it 240 times a second returns the same value 4 times in a row. The only thing lost is a press-and-release entirely inside one 16.7 ms tick, which is below human tap duration. Do not build a sub-tick input accumulator. If analog stick support is added later this changes, and the right answer is then a time-weighted average over the tick, not a higher sample rate.

Physics interpolation stays on. It costs ~8 ms (§5.1) and is the single largest fps-independent term after the tick wait, so it will look like a target. It is not: without it a 60 Hz simulation presents 60 distinct world states per second regardless of frame rate, which is precisely the stepping a 240 Hz display was bought to avoid. Leave it on; do not expose a toggle.

Why physics stays at 60 Hz, and what a bump would cost

The honest answer to "our players want 240 fps responsiveness" is that simulation rate, not frame rate, is the binding constraint — 16 ms of own-ship latency and ~33 ms of world response sit behind it, and §5.2 shows frame rate alone cannot get world response under ~146 ms. Doubling to 120 Hz (Rocket League's rate, with snapshots raised alongside) would take world response from ≈174 to ≈141 ms and own-ship from 52 to ≈44, at 60 Hz display — or ≈115 ms combined with a 144 Hz display and a low-latency present:

Term 60 Hz sim 120 Hz sim
wait for next tick 8.3 4.2
physics interpolation 8.3 4.2
jitter buffer, depth 1 16.7 8.3
server tick + flush 8 4
interpolation buffer 37.5 25.0 only the interval × 1.5 term halves; the jitter term does not
client ↔ server transit 60 60 does not move

That is a bigger win than every tuning parameter in §3 and §4 combined. It is nonetheless out of scope for v1, for reasons that are about the project rather than the netcode:

  • Every policy in Game/bots/ is invalidated. ship.gd:450's _tick_scaled is defined against a 60 Hz reference and ai_ship_controller.gd's reaction_ticks counts ticks. A bump means a full retrain — and per TODO.md the generation-5 curriculum is still running.
  • Server density halves, ~610 matches per core to ~35 (§1.4).
  • Bandwidth roughly doubles: input 43 → 86 kbit/s up, snapshots 105 → 210 kbit/s per client, 631 kbit/s → 1.26 Mbit/s per 6-player match. Still not the constraint, but 100 concurrent matches becomes ~126 Mbit/s of server uplink, which is a hosting-plan question rather than a rounding error.

The consequence for this plan is a hard rule: 60 is a constant named NetCodec.TICK_HZ, never a literal. Ring sizes, INTERP_DELAY, input_lead clamps, seq-window bounds, snapshot cadence and the timeout constants all derive from it. Task 1.4's handshake already gates on physics_ticks_per_second, so a mismatched client is rejected rather than silently desynced. Done this way, a later bump is a config change plus a retrain — not a protocol rewrite. Done the other way, the literal 60 ends up in twelve files and the bump never happens.

Client display settings

project.godot sets neither display/window/vsync_mode (defaults to enabled/FIFO) nor application/run/max_fps (uncapped). video_settings.gd:14-16 persists only AA, glow and brightness, and settings_menu.gd exposes only those three. Task 0.17 adds:

VSync: Enabled (FIFO) · Adaptive (default) · Mailbox · Disabled.

  • Adaptive (FIFO_RELAXED) is FIFO while the renderer keeps up and tears only on a missed vblank. That is the right default for a game that will sometimes drop below refresh, because it avoids FIFO's half-rate cliff — miss 144 Hz by one millisecond under strict FIFO and you are pinned to 72.
  • Mailbox only lowers latency when the renderer sustains above the refresh rate; below it there is never a second frame to replace the queued one, so it degenerates to FIFO latency at Mailbox power draw. Per §5.5 this build will not sustain above 144 Hz on typical hardware today, which makes Mailbox an opt-in for players with headroom, not a default. Defaulting to it would be a thermal regression for most players in exchange for nothing.

FPS cap: derived from the display, not a fixed list. Query DisplayServer.screen_get_refresh_rate(DisplayServer.window_get_current_screen()) and offer "Match display" (default), the integer divisors of that rate, then Unlimited — 144 Hz → 144/72/48, 165 Hz → 165/82/55, 240 Hz → 240/120/80/60.

Non-divisor caps beat against scanout. A fixed 60/75/90/…/360 list is wrong on every panel that is not 60 or 120 Hz. Cap at 100 on a 144 Hz display and gcd(100,144) = 4: the pattern repeats every 25 frames across 36 refreshes, with frames held for one or two intervals in an irregular sequence — visible micro-stutter. 120 on a 165 Hz panel is 8 frames per 11 refreshes, same failure. Offer the free-form list only behind an Advanced toggle with a warning.

Three implementation constraints, all of which an earlier draft got wrong:

  • Engine.max_fps is a throttle, not a pacer. It pads each frame with a post-frame sleep to hit 1/max_fps; it has no knowledge of scanout and never phase-locks to a vblank. (Sleep-granularity jitter of roughly ±0.51 ms is inferred, not measured — verify on target platforms. The absence of phase locking is structural.)
  • Grey out the FPS cap whenever VSync is not Disabled. With both active, FIFO clamps presents to vblanks while max_fps pushes some frames past the next one and not others — frame pacing worse than either setting alone. The menu must not permit the combination.
  • Godot cannot report the negotiated present mode. DisplayServer.window_get_vsync_mode() echoes back the mode you stored, not the VkPresentModeKHR the driver granted, and there is no GDScript API that exposes the latter. An earlier draft's "report what was actually applied" is not implementable, and neither is an in-engine present-latency measurement (that needs LDAT or a high-speed camera). Instead put a live Performance.get_monitor(Performance.TIME_FPS) readout next to the dropdown: whether the player is above or below their refresh rate is the fact every one of these settings depends on.

The renderer is Forward+ (project.godot:21, config/features=PackedStringArray("4.7", "Forward Plus")), so the usual "Mailbox is unavailable on Compatibility" caveat does not apply as written — but rendering/renderer/rendering_method is not pinned in project.godot, so a --rendering-method gl_compatibility launch or a driver fallback loses it silently. Mailbox is also commonly unavailable on macOS/MoltenVK. (Needs empirical verification on target OS versions.)

5.5 Can this build produce frames at all?

§5.4's table describes a machine this project is not. Nothing in the repo has ever been profiled, and the render configuration is a showcase build, not a competitive one. Every item below is on by default and none is reachable from video_settings.gd, which persists exactly three values (:14-16: aa_mode, glow_scale, brightness).

From scenes/arena_base.tscn, the Environment every arena inherits:

arena_base.tscn Setting Note
:47-50 sdfgi_enabled, sdfgi_use_occlusion, sdfgi_bounce_feedback = 0.5 Godot 4's most expensive GI path; cascades re-voxelise as the camera moves, and this camera never stops (ship_camera.gd:126,137,156)
:42-46 ssil_enabled, ssil_radius = 4.0 A full-resolution screen-space pass on top of SSAO
:34-41 ssao_enabled, ssao_radius = 2.5, ssao_detail = 0.75
:18-29 glow_enabled, 5 levels Mip pyramid built and resolved every frame
:61, 78, 87, 96, 105 1 directional + 4 shadow-casting OmniLight3Ds Omni shadows are cubemaps: 24 shadow-map faces per frame before the directional

Plus project.godot [rendering]: msaa_3d=2 (4×) and screen_space_aa=1 (FXAA) and use_debanding=true — mirrored by video_settings.gd:14 defaulting to MSAA_FXAA. Stacking FXAA on resolved MSAA is redundant blur, and the menu (settings_menu.gd) offers no 2× rung between "off" and "4×".

Plus shaders/post_process.gdshader:4, uniform sampler2D screen_texture : hint_screen_texture — a full-screen backbuffer copy every frame, unconditionally. The shader's comment notes that non-turbo frames skip two texture taps, but the copy and the full-screen pass happen regardless because vignette_strength never reaches zero (ship_camera.gd:187 writes 0.22 + …, :243 restores 0.22).

What is not the problem, so nobody optimises the wrong thing:

  • The 168 colliders (§1.4) cost zero frame time. They are CollisionShape3Ds on a StaticBody3D — no draw calls, no vertices. The count is confirmed correct (168 generated + 4 authored slabs = 172 in objects/arena_boundary.tscn).
  • The scene is not geometry- or draw-call-bound. arena_boundary.gd's visual shell is ~1450 triangles in two surfaces of one MeshInstance3D; the whole match is on the order of 100150 draw calls and well under 50k vertices. That is nothing.

The project is bound entirely by full-screen passes the player cannot switch off. That inverts §5.4's conclusion about where the leverage is: the largest win per line of code is not a vsync dropdown, it is a graphics preset that gates SDFGI/SSIL/SSAO/omni shadows. Task 0.15b blocks 0.16 and 0.17 for exactly this reason — every number in §5.4 is a priori, and the first measurement may invalidate the fps list entirely.

One mitigating subtlety, which cuts both ways: project.godot [display] sets window/stretch/mode="viewport" with a 1920×1080 base and aspect="expand", so the 3D renders at a fixed ~1080p and is blitted to the window. A 1440p or 4K player therefore does not pay more for any of the above — but also cannot render at native resolution, and a 1080p player cannot render lower. Task 0.17c owns that decision; it interacts directly with render scaling (0.17b) and cannot be left implicit.

5.5.1 Measured (task 0.15b, 2026-08-18)

6-ship Match, 1080p, non-headless. Hardware: Apple M4 (Metal), 10-core — a development laptop, not a dedicated gaming reference machine; treat absolute fps as directional, not a promise to players on other hardware.

p50 p99 fps (p50 / p99)
All effects on (project defaults) 17.93 ms 20.39 ms 55.8 / 49.0
All effects off ~17.2 ms ~58

This invalidates the a priori §5.4/§5.5 fps list exactly as flagged. Default settings cannot sustain even 60 fps on this hardware, let alone 144 — and the surprising part is why: turning every toggleable effect off (SDFGI, SSIL, SSAO, glow, all 5 shadow casters, MSAA, FXAA, PostFX) only recovers the difference between ~56 and ~58 fps. The ~17 ms floor is not made of the full-screen passes this section blamed — something else (base forward-clustered shading, the ~150 draw calls, per-ship VFX materials, or fixed engine/CPU overhead at 6 ships) dominates, and 5.4's framing ("the project is bound entirely by full-screen passes") is wrong as measured on this hardware.

Per-effect isolated cost (each toggled off individually against a fixed baseline sample), for reference — treat these as low-confidence: they cluster tightly at 2.93.8 ms each with no clear outlier, which is consistent with most of that spread being sampling noise from a ~1 ms-jittery baseline rather than real per-effect attribution:

Setting Cost (ms)
SSAO 3.77
PostFX 3.82
Omni shadows (×4) 3.69
SSIL 3.44
FXAA 3.37
Directional shadow 3.30
SDFGI 3.24
MSAA 4× 3.12
Glow 2.89

Consequence for 0.17/0.26/0.28: a graphics preset alone will not reach a 144 fps target on hardware in this class — Low-preset gets to only ~58 fps by this measurement, not the 2×+ jump §5.4 assumed. 0.26 (bake GI) and 0.28 (separate physics thread) need to re-justify their expected win against this floor before implementation.

Root-cause follow-up, attempted and inconclusive (2026-08-18). Three further remote-automated profiling passes (via godot-mcp game_eval sampling Performance.get_monitor() against a live instance, no human at the editor) were run to find what the ~17 ms floor actually is. They did not converge:

Pass Setup Result
1 (above) 6-ship 3v3, sustained 17.93 / 20.39 ms (p50/p99), all-off floor ~17.2 ms
2 Reportedly 6-ship, actually 1v1 (misconfigured) CPU 17.64 ms + frame 10.75 ms — internally inconsistent (CPU time exceeding frame time from non-atomic sampling); agent also reported the game becoming unresponsive mid-run
3 6-ship 3v3, atomic single-eval sampling, retried after pass 2's failures 8.710.2 ms (98115 fps), reported CPU time 0.013 ms — implausibly low for a frame running Jolt physics + GDScript bot inference across 6 ships, so not trusted either

Passes 1 and 3 supposedly measured the same scenario and differ by ~2×. The likely explanation is the measurement method itself, not the game: each game_eval round-trip through the MCP bridge has its own latency and can perturb the very frame timing it's sampling, and nothing here confirms the scene state (ship count, bot activity, camera framing) was identical across passes. Read the specific numbers in this subsection as evidence a floor well under 144 fps exists, not as an attributed cause — the SSAO on/off screenshot check in pass 3 did confirm effect toggles are visually real (ruling out "the toggles are no-ops" as an explanation), which is the one finding that survived across passes.

What this needs next, and why an agent can't finish it remotely: a proper frame-time attribution needs either a human at the Godot editor reading the Debugger's built-in Monitors/Visual Profiler (which breaks GPU time down by pass — opaque, shadow, post-process, etc. — instead of one aggregate number), or an external GPU profiler (RenderDoc, Xcode GPU capture on this hardware). Both require eyes on a live UI, not remote eval polling. This is now the concrete blocker for 0.26/0.28, not further scripted measurement passes. 0.15b's original acceptance criterion (write a max-frame-rate number into §5.5) is still met by pass 1 — the floor is real and under both 60 and 144 fps — but the deeper "why" is open and parked here rather than guessed at.

Root cause of the pass-to-pass inconsistency, found (2026-08-18): a Godot editor and an orphaned headless training process had both been running on the profiling machine, untouched, for 11 days (since 2026-08-08) — leftover from earlier local work, unrelated to this investigation. godot-mcp's automated launches were plausibly contending with that stale editor instance rather than getting a clean process every pass, which is a much better explanation for a ~2× swing between "identical" scenarios than genuine frame-time variance. Both processes were killed and a clean re-check was run.

Is it just that we're on a Mac? Partly, but not via the mechanism first suspected. HiDPI/Retina resolution inflation was checked directly and ruled out: the live viewport renders at 2036×1080 against a target of 1920×1080 — about 6% more pixels, non-uniformly (width only; the 2× multiplier a true Retina backbuffer would apply is not happening, display/window/dpi/allow_hidpi=true notwithstanding). A 6% pixel-count difference cannot produce the ~2× frame-time swings seen above, so resolution is not the explanation for this session's inconsistency — that was the stale-process contention above. It's still worth a one-line fix later (0.17c owns display/stretch decisions) since 2036×1080 is a mildly wasteful, non-native render target.

What Mac hardware does plausibly bias is the shape of the result, not the run-to-run noise: Apple Silicon GPUs are tile-based deferred renderers (TBDR), architecturally unlike the immediate-mode AMD/Nvidia GPUs the target "reference hardware" (a Windows/Linux gaming PC) uses. TBDR keeps a frame in on-chip tile memory and is comparatively cheap at MSAA resolve, but any pass needing to read arbitrary neighbouring pixels across the whole frame — SSAO, SSIL, the glow downsample/upsample chain, the PostFX shader's screen_texture read — forces a break out of tile memory into a full system-memory resolve, an overhead that is largely constant per pass rather than proportional to what the pass computes. That lines up with pass 1's finding that SDFGI/SSIL/SSAO/MSAA/FXAA/shadows/PostFX all cost within a tight 2.93.8 ms band regardless of what each one actually does — consistent with a shared TBDR resolve tax dominating over each effect's real cost. Numbers measured on this machine should be treated as informative about relative ordering at best, not as a stand-in for target-platform (desktop GPU) behaviour — confirmed below.

5.5.2 Measured on real reference hardware — RTX 3090, Linux (2026-08-19)

Same 6-ship 3v3 Match, 1080p, via a purpose-built harness (Game/tools/gpu_profile_harness.gd) run directly against a real GPU-bound X session (not Xvfb — an earlier attempt through Xvfb silently fell back to Mesa's llvmpipe software rasterizer, ~35x slower and completely unrepresentative; caught via the harness's own adapter-name check, not assumed). This is the number that matters — an actual discrete immediate-mode GPU, the architecture players will actually have:

p50 p99 fps (p50)
All effects on (project defaults) 1.85 ms 2.98 ms 540
All effects off 0.53 ms 1.53 ms 1883

This overturns §5.5.1's conclusion, not just its numbers. On real hardware, disabling every effect gives a 3.5× speedup — the opposite of the Mac's ~1.03× — and the per-effect breakdown finally makes physical sense instead of clustering suspiciously:

Setting off Frame time Implied cost
(baseline, all on) 1.85 ms
SDFGI 1.49 ms 0.36 ms
SSIL 1.60 ms 0.25 ms
Glow 1.75 ms 0.10 ms
Shadows (all 5 casters) 1.76 ms 0.09 ms
SSAO 1.82 ms 0.03 ms
MSAA 4×, FXAA, PostFX 1.872.12 ms noise-level (see below)

SDFGI and SSIL alone account for over half of the effects' total cost, matching §5.4's original expectation (voxel cone tracing and a full-res screen-space GI pass being the expensive ones) — the Mac's flat, undifferentiated cost profile was the anomaly, not this one. MSAA/FXAA/PostFX show negative "costs" (disabling FXAA measured as slightly slower than leaving it on) — at ~1-2 ms absolute frame times, OS scheduling jitter is larger than the real signal for cheap passes; those three need a longer sampling window or a proper GPU profiler to resolve, not this harness's coarse get_process_delta_time() sampling. Note also that all-off (0.53 ms) is faster than baseline-minus-sum-of-individual-savings (1.85 0.36 0.25 0.10 0.09 0.03 ≈ 1.02 ms) — the combined removal saves more than the parts, consistent with each full-screen pass carrying some fixed per-pass overhead (pipeline barriers, render-target switches) on top of its own work, which compounds when several stack.

Consequence for 0.17/0.26/0.28, revised: at 540 fps p50 with every effect enabled, this scene is nowhere near GPU-bound on reference-class hardware — the entire "must hit 144 fps" framing in §5.4/§5.5 was solving a problem that doesn't exist on the hardware tier it was written for. That reframes the two gated tasks rather than clearing them outright:

  • 0.26 (bake GI, retire SDFGI) — the relative win is real and correctly targeted (SDFGI is the single largest line item, ~19% of the effects-on budget), and the preset design already bets on this being right (Low/Medium turn SDFGI+SSIL off first, matching exactly what this data says to cut). But "largest frame-time reduction of any task here" (its acceptance bar) oversells it on a 3090 — 0.36 ms off an already-tiny budget is not the headline win §5.7 implied. The task is worth doing for lower-end/integrated GPUs, where the same relative cost almost certainly scales to something that matters — but that's now the open question, unmeasured on this pass.
  • 0.28 (physics/3d/run_on_separate_thread) — its whole motivation is smoothing frame-time variance caused by the physics tick sharing the render thread; at a 1.85 ms p50 / 2.98 ms p99 baseline (both far under even a 240 Hz frame budget), there's no variance problem to fix on this hardware. Deprioritize below 0.26 unless a lower-end-hardware pass shows otherwise.
  • The preset ladder itself (task 0.17, done) needs no changes — its bundle choices (drop SDFGI/SSIL first) are now empirically justified rather than just plausible-sounding.

Still open: no low/mid-tier GPU has been profiled. The 3090 result rules out "the game is GPU-bound on reasonable hardware" as a near-term concern, but says nothing about a GTX 1660 or an integrated Iris/Vega part, which is where a real preset ladder earns its keep. Re-run gpu_profile_harness.tscn on weaker hardware before spending more effort on 0.26/0.28.

5.6 Closing the gap to the reference — without lowering settings

§5.2 lands at ≈174 ms against a ~90110 ms reference band. The instinct is that reaching it means trading visual quality for frames. It does not. Decompose the 174:

At 60 ms RTT, 60 ms is transit and irreducible in code. That leaves 114 ms of local overhead, of which frame rate governs only two terms — input freshness (10) and present (25) — and quality settings govern neither directly. Present latency is a function of vsync mode and swapchain depth, not of how many effects are enabled; a 60 fps client with a shallow present queue beats a 240 fps client with a deep one. The entire 60 → 240 fps range is worth ~12 ms once a low-latency present is in place (§5.4). The other ~100 ms is netcode time model and simulation rate.

Four levers, none of which touches a graphics setting:

Lever Saves Risk
L1 Extrapolate remote visuals to present time instead of interpolating the past 30 Mis-prediction pops
L2 120 Hz simulation 21 Bot retrain, ½ server density, 2× bandwidth
L3 Adaptive jitter-buffer depth, 0 on clean links 8 Starvation on jittery links
L4 Shallow present queue + Adaptive vsync 17 Throughput loss if GPU-bound

L1 is the big one, and it is nearly free

§4.1 already computes remote entities' present-time state — that was the fatal correction that put the collider at server_time_est. $Visual is then deliberately rendered ~68 ms in the past for smoothness. Render it at present time too and the whole 37.5 ms interpolation buffer disappears, leaving only a residual for error smoothing.

The reason this is safe here is that ships have bounded acceleration and the hull is large. Extrapolating with known velocity, error is ½·a·t² over the full 68 ms horizon:

max accel error @ 38 ms error @ 68 ms
position, cruise 30 m/s² (thrust_power 150 / mass 5) 0.022 m 0.069 m
position, turbo 75 m/s² (turbo_multiplier 2.5) 0.054 m 0.173 m
yaw 20 rad/s² (rotation_power 20 / inertia.y 1) 0.8° 2.6°
pitch / roll 2.9 rad/s² (inertia.x/z 7) 0.1° 0.4°

0.17 m and 2.6° worst case, against a 4 m hull. That is well under the width of the ship and an order of magnitude smaller than the 3.5 m staleness §4.1 was written to eliminate. Feed the residual through the same soft-correct pipeline already specified for the local ship (§4.4) and remote ships are visually at present time with a sub-decimetre wobble.

Two bonuses: it collapses §4.1's dual clock back into one — collider and visual both at server_time_est, so §5.4b's _process/_physics_process split and the two-regimes-for-one-node-name hazard both go away — and it applies to the ball, which is near-ballistic between contacts and therefore extrapolates better than ships do.

The cost is real but narrow: a remote ship that reverses input at the moment you sample it mispredicts by the numbers above and then visibly corrects. Interpolation never mispredicts; it is just always late. This is the genuine trade, and it is the one the reference class makes.

The reachable budget

Term today L1 + L4 (v1) + L2 + L3 at 144 fps
input freshness 10 10 10 5.5
wait for next tick 8.3 8.3 4.2 4.2
client → server 30 30 30 30
jitter buffer 16.7 16.7 4.2 4.2
server tick + flush 8 8 4 4
server → client 30 30 30 30
interp buffer → extrapolation residual 37.5 8 8 8
client physics interpolation 8.3 8.3 4.2 4.2
present 25 8.3 8.3 3.5
World response ≈174 ≈127 ≈103 ≈94

≈103 ms at 60 fps with every effect enabled, and ≈94 at 144 fps. That is inside the reference band, reached without disabling SDFGI, SSIL, SSAO or shadows. Even a client struggling at 30 fps on maximum settings lands near ≈120 ms.

Sequencing follows ms-per-unit-of-risk: L4 then L1 for v1 (≈127 ms, no bot retrain, no protocol change); L2 and L3 after, when a retrain is affordable. §5.5's preset system remains worth building — but for frame rate and thermals, which is what it actually buys, not for latency.

The largest lever is not on this list. All of the above assumes 60 ms RTT. Regional server siting that puts most players on a 30 ms RTT takes ≈127 to ≈97 and ≈103 to ≈73 with no code at all. Phase 6 owns it, and it should be argued against these numbers.

Perspective on where this matters. Own ship and ball are already at ≈52 ms and are unaffected by every lever here — they are predicted locally. World response governs opponent ships. In a game whose subject is a ball, that ordering is favourable: the two objects a player tracks most closely are the two already at single-digit-tick latency.

5.7 The next tier — and where it stops paying

§5.5 and §5.6 are the first-order work. This section is what remains after them, and it is deliberately honest about the point where further effort stops being worth it.

Frame rate: SDFGI is the wrong tool for this arena

The single largest available win, and it costs no visual quality. arena.gd and goal.gd have no _process, no _physics_process, no AnimationPlayer and no Tween — the floor, walls, ceiling, goals and every light are static for the entire match. The only things that move are 6 ships and a ball, all small and all self-lit.

SDFGI exists to light dynamic worlds, and it pays for that by re-voxelising cascades as the camera moves — and this camera never stops moving (ship_camera.gd:126, 137, 156). It is the most expensive thing in the frame, doing continuous work to solve a problem this project does not have.

  • Replace sdfgi_enabled with baked GILightmapGI for the static shell, or VoxelGI if bounce onto moving ships matters. Bake cost is offline; runtime cost is a texture fetch. The look is preserved or improved (baked bounce is higher quality than SDFGI's cascades), and it survives on the High preset rather than being the first thing a preset has to switch off.
  • ssil_enabled becomes largely redundant once bounce is baked. It is a full-resolution screen-space pass duplicating information the lightmap already has.

This is the answer to "lowest lag and highest fps without lowering settings": the expensive setting was solving the wrong problem.

Frame rate: expensive defaults that project.godot never overrides

[rendering] contains exactly three keys (msaa_3d, screen_space_aa, use_debanding). Everything else runs at engine defaults, including:

Setting Default Note
lights_and_shadows/positional_shadow/atlas_size 4096 Shared by all shadowed positional lights; 2048 is usually indistinguishable here
lights_and_shadows/directional_shadow/size 4096
lights_and_shadows/directional_shadow/soft_shadow_filter_quality high
occlusion_culling/use_occlusion_culling off Low value in an enclosed arena — measure before adding bake time
mesh_lod/lod_change/threshold Irrelevant: the scene is ~1450 triangles of arena plus low-poly ships (§5.5)

Also worth counting: _build_movement_vfx creates two OmniLight3Ds per ship (ship.gd:270-278), so a 3v3 has 12 dynamic lights on top of the arena's 5. They are correctly shadow_enabled = false and omni_range = 3.5, so they are cheap — noted so nobody "discovers" them and disables engine glow for nothing.

Frame rate: the CPU side, which §5.5 does not cover

§5.5 establishes the project is GPU-bound on full-screen passes. Once those are fixed it becomes CPU-bound, and §5.4's frame-time variance becomes the ceiling. Three levers:

  • physics/3d/run_on_separate_thread (not set; defaults off). This decouples the physics step from the render thread and directly attacks "one frame in four carries the whole tick." It is the highest-leverage item here and the riskiest — it changes when _integrate_forces runs relative to script code, and this project puts real logic there (ship.gd:346-357) plus an RL training path. Prototype and measure; do not enable on faith.
  • ArenaBoundary.get_surface_pull has no early-out. It runs a to_local() plus five _falloff calls for every dynamic body every tick, including for a ball sitting in the middle of the arena where every term is zero. A single bounds check against wall_range/ceiling_range skips almost all of it in open play — 7 bodies × 120 Hz once L2 lands.
  • Bot inference is ~6.5k GDScript multiply-accumulates per bot (policy_network.gd). Task 0.8 staggers them; beyond that, the lever is network width, which is a training decision, not a rendering one.

Latency: what is actually left

After L1L4 and 120 Hz simulation, at 144 fps, the budget is ≈94 ms — and 60 of that is RTT. The remaining 34 ms of local overhead breaks down as input freshness 5.5, tick wait 4.2, jitter 4.2, server 4, extrapolation residual 8, physics interpolation 4.2, present 3.5. Every one of those is at or near a floor set by physics rate or hardware.

Two code ideas remain, both small and both with a cost:

  • Forward-extrapolate the local $Visual instead of interpolating between the last two ticks — render the predicted ship at present time rather than up to one tick behind. Worth ~4 ms. Risk: overshoot at the moment of a collision, which is the most visually sensitive moment in the game.
  • Tighten the extrapolation-error smoothing (§5.6's 8 ms residual). Worth ~4 ms, paid for in more visible correction pops.

That is the whole remaining code budget: ~8 ms, both items trading visual stability for it. Meanwhile:

  • Regional server siting takes a 60 ms RTT to 30 for most players: 30 ms, four times the remaining code budget, no code at all.
  • Ping-weighted matchmaking and a server browser sorted by measured ping convert that into something players actually experience rather than something that is true on average.
  • Steam Datagram Relay (Phase 7) is planned for NAT traversal and DDoS protection, but Valve's backbone frequently routes better than raw BGP paths — for some player pairs SDR is a latency reduction, not a tax. Measure it both ways rather than assuming it costs.

Where this stops paying

Two limits worth writing down before someone spends a month on the last 5 ms:

  1. Past ~100 ms, you are optimising 34 ms at a time against a 60 ms constant. The ratio of engineering effort to felt improvement collapses. Server siting and matchmaking dominate everything else from that point on.
  2. "Lowest lag" and "best feel" diverge at the end. Both remaining code levers, and L1 itself, buy milliseconds by predicting further ahead and correcting harder. Past a point that makes the game feel worse — twitchier, less stable, more prone to visible snapping — while the latency number keeps improving. The number is a proxy, not the goal. Task 4.7's tuning pass, with a human in the seat, is the authority; the budget table is not.

6. Match lifecycle

6.1 State machine

LOBBY -> LOADING -> WARMUP -> PLAYING -> GOAL_PAUSE -> WARMUP -> ...
                                      -> FULL_TIME -> OVERTIME_WARMUP -> OVERTIME -> ...
                                      -> RESULTS -> LOBBY

Broadcast as the match_state byte in every snapshot, and on transition via state_change(state, at_tick).

6.2 Sequence

  1. Connect. Client sends hello(protocol_version, physics_ticks_per_second, display_name, auth_ticket). Server rejects a mismatch on either version or tick rate, with a reason string, then disconnect_peer. (A client at 30 Hz advances its sequence numbers at half rate and confuses every control loop.) auth_ticket is an empty PackedByteArray until Phase 7 — reserve the field now.
  2. Welcome. Server assigns player_id, balances teams, replies welcome(player_id, server_info, roster, match_state, server_tick, score, end_tick), broadcasts player_joined.
  3. Lobby. ready_toggle(); start when all ready, or --auto-start after --min-players plus a countdown.
  4. Config. match_config(match_id, arena_path, team_size, match_length_ticks, roster[], seed). roster[i] = {slot, team, spawn_index, player_id, name, is_bot}slot order here is the snapshot's body order for the whole match. The client validates arena_path against ArenaRegistry.ARENAS before load(); a malicious or buggy server must not be able to make a client load an arbitrary res:// path.
  5. Load. Both sides load networked_match.tscn. Each peer loads the arena and spawns the roster in slot order. Client additionally spawns a camera rig on its own ship and adds HUD.tscn in codenetworked_match.tscn must have no HUD child, because GameMode._ready() (game_mode.gd:44-45) would pick it up server-side. Client sends scene_ready(match_id).
  6. Kickoff. Server waits for all scene_ready (10 s timeout → proceed). Broadcasts kickoff(reset_transforms[], countdown_start_tick, reset_gen). Both sides freeze bodies. HUD counts down from server_tick, not a local Timer. At countdown_start_tick + 180 the server unfreezes and broadcasts state_change(PLAYING).
  7. Play. Inputs up, snapshots down.
  8. Goal. Server's Goal sensor fires → _handle_goal_scored debounce → goal_scored(scoring_team, score, goal_tick, resume_tick). Bodies freeze. Clients play the cinematic within [goal_tick, resume_tick]. At resume_tick: kickoff(...).
  9. Clock. Tick-derived: remaining_ticks = end_tick - current_server_tick. end_tick and a running flag ship in match_config and in clock_state(running, end_tick, at_tick).
  10. Full time / overtime / results. RESULTS holds, then state_change(LOBBY) and both sides load lobby.tscn. Clients return to the lobby, not the main menu — a community server that empties every 2.5 minutes is dead on arrival.

Every lifecycle message carries absolute ticks, never durations. That is what makes reliable-channel latency harmless: on a lossy link ENet's RTO can stretch a goal_scoredkickoffstate_change burst to ~600 ms. Specify the late-arrival case explicitly: a kickoff that lands after its own resume_tick must apply the reset immediately and skip the countdown, not schedule it into the past.

NetworkedMatch must declare all five signals HUDController duck-types on (HUDController.gd:65, 88, 100, 103, 106) — timer_updated, score_changed, match_ended, kickoff_countdown, overtime_started — and emit them from RPC handlers instead of from local logic. Otherwise the HUD silently omits rows.

6.3 Late joiners and spectators

welcome carries full state, so a late joiner reconstructs immediately.

  • Free slot and state is LOBBY/WARMUP → join as a player now.
  • Free slot mid-match → spectate now, take the slot at the next kickoff. Swapping a controller at a kickoff boundary is free; mid-play it is not.
  • No free slot → spectator. A spectator receives identical snapshots (the snapshot is already a broadcast — zero extra server work), spawns no ship, and points a camera rig at a chosen ship or the ball. Cap with --max-spectators.

HUDController._initialize_hud() push_errors and bails when ship is null (HUDController.gd:41-46). Spectators need a path through that.

6.4 Disconnects — no ship is ever despawned

On peer_disconnected the server keeps the ship and swaps its controller:

  1. --fill-bots: replace with an AIShipController on the server's configured model.
  2. --no-fill-bots (default for public servers, see §1.4): swap to the base ShipController — inert but simulated, exactly the placeholder game_mode.gd:216 already uses.

Set flags.stalled so clients can grey out the nameplate. Reserve the slot for 30 s keyed by identity so a reconnect gets its ship back. If the last human leaves, abort to LOBBY.

Justification is wire-format simplicity, not the bot cache. Fixed slot order means the snapshot needs no add/remove machinery, no MultiplayerSpawner, and no re-indexing. That reason stands on its own.

ai_ship_controller.gd currently caches teammate/opponent lists once with the comment "rosters never change mid-match (no despawn path exists anywhere in this codebase)". Do not let that be the justification — protecting a bot's implementation detail is tail-wagging-dog, and taken as an architectural constraint it permanently forecloses 3v3→2v2 shrink, mid-match rebalancing, and join-onto-a-new-slot. Fix the cache anyway (task 0.9): filter(is_instance_valid) plus a roster_changed signal, ~5 lines of cheap insurance.


7. Phase and task breakdown

[P] parallelisable within its phase · [D:x.y] hard dependency

Phase 0 — Non-networked refactors

Every task lands on master independently, is verifiable in single-player today, and cannot break anything. Near-total parallelism.

# Task Files Acceptance
0.1 [P] DONE. Added ShipAction.copy(). Audit: the only get_action() call site (ship.gd:347) reassigns _current_action fresh each tick rather than buffering it, so no aliasing bug exists yet — copy() is a no-op today, ready for Phase 4's prediction ring ship_action.gd, player_ship_controller.gd Free Play unchanged; copy() returns a distinct object with equal fields
0.2 [P] DONE. Inserted Visual (Node3D) into ship.tscn, reparented Nose/TailFin under it, redirected all four code-driven add_child calls onto $Visual (now a public @onready var visual), resolved _apply_team_color's lookup to "Visual/" + mesh_name objects/ship.tscn, scripts/ship.gd Child-type assertion holds; ship looks identical in Free Play; team colours still apply on both teams
0.3 [D:0.2] DONE. ship_camera.gd's three target.global_transform reads (ball cam, ship cam ×2) now read target.visual.global_transform scripts/ship_camera.gd Camera behaviour unchanged in Free Play and Match — visual has identity transform relative to the body until Phase 4 writes an offset, so this is a no-op today
0.4 [P] DONE. can_sleep = false on Ship and Ball objects/ship.tscn, objects/ball.tscn No behaviour change
0.5 [P] DONE. continuous_cd = true on Ship (Ball already had it) objects/ship.tscn No tunnelling at max speed into the ball or walls
0.6 [P] DONE. Spawned ships renamed to Ship_T%d_S%d game_mode.gd Names are (team, spawn_index)-derived, not insertion-order
0.7 [P] DONE. _jittered now uses an owned RandomNumberGenerator, self-randomized in _ready() unless kickoff_rng_seed is set explicitly (a fresh RandomNumberGenerator defaults to a fixed internal state, unlike the global randf_range Godot auto-randomizes at startup — call this out for whoever reads the diff and expects .new() alone to be enough) game_mode.gd Kickoff jitter unchanged in feel; a fixed seed reproduces kickoffs exactly
0.8 [P] DONE. _ticks_until_decision = randi_range(1, reaction_ticks) at spawn, after load_policy() (which still resets to 0 on later calls, e.g. league opponent swaps — harmless, those land at reset boundaries) ai_ship_controller.gd Six-bot Spectate shows no periodic frame spike
0.9 [P] DONE. Roster validity checked (Array.any()) once per decision tick, not every physics tick; filter(is_instance_valid) + roster_changed signal only fire on an actual stale reference ai_ship_controller.gd Bots behave identically; freeing a ship mid-match no longer corrupts observations
0.10 [D:0.12] [P] Add virtuals _owns_goal_logic(), _allows_time_scale_effects(), _goal_pause_seconds(), _owns_world_simulation() DONE, narrower than drafted. _allows_time_scale_effects() dropped: 0.12 deletes Engine.time_scale from the file entirely, so there is nothing left for it to gate. Implemented _owns_goal_logic(), _owns_world_simulation(), _goal_pause_seconds(), all behaviour-preserving (default true/GOAL_CELEBRATION_SECONDS), gating the goal-signal connection and _respawn_escaped_bodies() game_mode.gd Free Play, Match, Spectate and Training all behave identically — verified no other virtual was load-bearing today; these exist for a future networked-client mode
0.11 [P] DONE. _handle_goal_scored checks is_inside_tree() after each await and bails before touching arena/hud state game_mode.gd A scene change mid-celebration cannot strand the flag
0.12 [P] Replace Engine.time_scale hit-stop and goal slow-mo with camera-only effects DONE. Added ShipCameraRig's "Impact Punch" group (punch_fov_kick/punch_vignette_kick/punch_chroma_kick/punch_decay, applied additively after _update_speed_feel each tick, decaying via move_toward over real delta) triggered from the existing _on_target_ball_contact; goal moments now rely on the pre-existing begin_goal_cut/end_goal_cut cinematic cut alone, no separate slow-mo effect needed. All Engine.time_scale fields/methods deleted from game_mode.gd (_hit_stop_*, _goal_slowmo_active, _restore_hit_stop, _run_hit_stop, GOAL_SLOWMO_SCALE) game_mode.gd, ship_camera.gd Goal and impact feel is at least as good; Engine.time_scale is never written — confirmed via grep -rn time_scale scripts/
0.13 [P] DONE. physics_jitter_fix = 0.0 set. CLAUDE.md's architecture section had stale prose dimensions ("inner x ±12, z ±18, height 12, goal lines z ±17") — corrected to reference the actual named constants (INNER_HALF_X 18, INNER_HALF_Z 27, INNER_HEIGHT 18, GOAL_LINE_Z = INNER_HALF_Z) instead of restating numbers that can drift out of sync again project.godot, CLAUDE.md Flight feel unchanged; CLAUDE.md matches arena_boundary.gd:8-14
0.14 [D:0.2] DONE. Added Ship.set_visual_action(thrust_z, turbo), Ball.set_visual_speed(speed) (with a _visual_speed_override field the trail prefers when ≥0), and Ship.net_vel_correction/net_visual_offset fields plus the guarded hook at the top of _integrate_forces (decays net_visual_offset via _tick_scaled, writes it to visual.position) ship.gd, ball.gd No-op until Phase 4; single-player unchanged — nothing calls any of these yet
0.15 [P] DONE. Ship/Ball gained queue_teleport(to); _integrate_forces applies it via state.transform + zeroed velocities + reset_physics_interpolation(). GameMode._reset_body now calls body.call("queue_teleport", to) (dynamic dispatch — RigidBody3D itself has no such method) instead of set_deferred game_mode.gd, ship.gd, ball.gd Kickoff resets in Match are visually identical, with no interpolation smear
0.15b DONE, superseded by §5.5.2 — read that, not the Mac numbers below. First pass measured a live 6-ship Match, 1080p, on an Apple M4 dev laptop (§5.5.1): all-on p50 17.93 ms, all-off floor ~17.2 ms, with per-effect costs clustered suspiciously flat (2.93.8 ms each). That data turned out to be a poor stand-in for the target platform — Apple's tile-based GPU architecture, not a real bottleneck — and was superseded by a same-scenario re-run on real reference hardware (RTX 3090, §5.5.2): all-on p50 1.85 ms / all-off 0.53 ms, SDFGI+SSIL clearly dominant as originally expected, everything else cheap. Keep §5.5.1 for the record of what was tried and why it was distrusted, not as a performance reference scenes/arena_base.tscn, shaders/post_process.gdshader, Game/tools/gpu_profile_harness.gd Measured max frame rate written into §5.5.2 from real reference hardware. At 540 fps p50 with everything on, this scene is nowhere near GPU-bound on a 3090-class GPU — the a priori §5.4 fps list was solving for a constraint that doesn't hold at that hardware tier. 0.17 (done) needed no changes: its preset bundle choices are now empirically validated. 0.26 stays open (real but smaller win than assumed); 0.28 closed (no variance problem exists to fix)
0.16 [D:0.3] DONE. Camera rig moved _physics_process_process; reads target.visual.get_global_transform_interpolated() in both ball-cam and ship-cam; rig itself has physics_interpolation_mode = OFF (it writes its own transform every rendered frame now, so Godot's built-in interpolation would just fight the manual smoothing). target setter primes interpolation (target.visual.reset_physics_interpolation()) and calls the new snap_to_target() so a freshly-assigned target (or a Spectate switch) doesn't lerp in from wherever the rig was previously. Shake re-derivation, implemented differently than drafted: rather than rescale frequency, _apply_shake now quantizes the noise-domain input to whole 60Hz ticks (floori(_shake_time * SHAKE_UPDATE_HZ)) — every render frame within one 1/60s window reuses the identical noise sample, so consecutive distinct samples stay exactly frequency (2.5) domain-units apart at any render frame rate, reproducing 60fps's original jitter character everywhere instead of smoothing out at high fps. snap_to_target() is called from game_mode.gd's reset_ships(), not directly from ship_camera.gd's own kickoff-adjacent code — reset_ships() is now async and awaits one get_tree().physics_frame before snapping, because _reset_body's queue_teleport (task 0.15) defers the actual transform write to the ship's next _integrate_forces; snapping immediately would read the pre-teleport position. Goal-cut shake decay extracted into _decay_shake(), called from the _goal_cut_active branch. Validated: scripts compile, Free Play renders correctly non-headless, reset produces no camera jump, all three headless scenes exit clean scripts/ship_camera.gd, scripts/game_mode.gd:reset_ships Turbo FOV kick and post-process are smooth at an uncapped frame rate; shake reads the same at 60 and 240 fps; a kickoff cuts the camera rather than lerping it across the arena
0.17 [D:0.15b] DONE. VideoSettings gains Preset (Low/Medium/High/Custom) driving a bundle (sdfgi_enabled, ssil_enabled, ssao_enabled, shadows_enabled, glow_enabled, aa_mode, resolution_scale) via apply_preset(); a settings_changed signal lets an already-loaded arena re-apply live (arena.gd connects in _ready()) rather than only affecting the next arena load — meets "settings persist and apply without a restart" without needing a scene reload. Shadow gating targets the actual Light3D nodes (found once at load via find_children, cached, re-applied on every settings change — deliberately not re-derived from current state each time, since a light this code just turned off would otherwise become indistinguishable from FillLight, which is authored shadow_enabled = false on purpose and must never be turned on by the preset ladder). vsync_mode (Disabled/Enabled/Adaptive, Adaptive default) and fps_cap_divisor (0 = uncapped, else divides the live refresh rate at apply time rather than storing a raw fps number, so the same preference re-derives correctly on a different display) added to the settings menu; FPS cap dropdown is disabled (greyed) unless VSync is Disabled; refresh-rate query ≤0 falls back to "Uncapped" only. Live fps readout via _process reading Performance.TIME_FPS. main_menu.gd's _leave_to_gameplay now calls VideoSettings.apply_fps_cap() instead of hardcoding Engine.max_fps = 0, so the player's cap actually reaches gameplay scenes. Acceptance numbers: not run as a literal Low-vs-High preset A/B, but strongly implied by §5.5.2 — real hardware (RTX 3090) runs the High-equivalent (all effects on) at 540 fps p50 already, so Low (which additionally turns off the two dominant costs, SDFGI+SSIL) clearing "≥2×" is close to guaranteed rather than measured directly; the flat-p99-histogram claim genuinely wasn't tested (gpu_profile_harness.gd measures per-toggle cost, not vsync/cap histograms) scripts/video_settings.gd, scripts/settings_menu.gd, scenes/settings.tscn, scripts/arena.gd, scripts/main_menu.gd Low preset ≥2× the frame rate of High on the same hardware; settings persist and apply without a restart; every offered cap gives a flat frame-time histogram (p99p50 < 1 ms) with VSync disabled on a 144 Hz and a 165 Hz display; refresh-rate query returning -1 falls back cleanly
0.17b [D:0.15b] [P] DONE. VideoSettings.resolution_scale (0.51.0, default 1.0) drives Viewport.scaling_3d_mode/scaling_3d_scale/fsr_sharpness via apply_resolution_scale()SCALING_3D_MODE_FSR2 below 1.0 (chosen over bilinear: this project already gave up native resolution at the fixed-1080p blit per 0.17c, so FSR2's sharpening recovers more of that loss than a plain bilinear upscale at the same internal scale), SCALING_3D_MODE_BILINEAR with scale pinned to 1.0 at the top of the range (a no-op scaling mode when the scale is 1:1). Low preset defaults to 0.8. Exposed as a slider in the settings menu; not yet measured against the "0.7 scale gives a large, measurable frame-time drop" bar — same real-hardware caveat as 0.17 scripts/video_settings.gd, settings_menu.gd 0.7 scale gives a large, measurable frame-time drop with acceptable image quality; setting persists
0.17c [D:0.17b] DONE — decided, not changed. Kept stretch/mode="viewport" fixed at 1080p rather than moving to "disabled", documented inline in project.godot [display] with rationale: 0.17b's scaling_3d_scale already covers "render lower than the window" independently of stretch mode (it scales the 3D viewport's internal resolution before this blit, not the window itself), and separately, task 0.15b found an unexplained ~6% non-uniform width scaling on the one machine this was tested on (2036×1080 measured against a 1920×1080 target — see §5.5.1) that needs understanding before stretch mode is touched, not blindly carried into a resolution-dependent change project.godot The decision and its rationale are written into §5.5; render resolution follows the player's setting
0.17d [P] INVESTIGATED — no such lever exists in Godot 4.7. Searched the full project.godot schema (read_project_settings) for rendering/rendering_device/vsync/frame_queue_size and every variant (frame_queue, swapchain, present, present_queue) — none exist as a project-settable parameter in this engine version; the RenderingDevice backend may manage its own present queue internally but doesn't expose it. Adaptive vsync (task 0.17, done) is the only half of "L4" actually achievable through project settings. The §5.6 ~17 ms figure for a shallow present queue is therefore not obtainable as specced — closing this without a code change is correct here, not a shortfall; reaching it would need engine-level (C++/RenderingDevice) changes out of scope for a project-settings task
0.18 [P] DONE, with one discovered GDScript constraint. New scripts/sim_constants.gd (class_name SimConstants, plain const TICK_HZ := 60, not an autoload) is the source of truth for ship.gd's _tick_scaled and training_mode.gd's TICKS_PER_SIM_SECOND — both reference it via const SimConstants = preload("res://scripts/sim_constants.gd") rather than the bare global class_name symbol, because a cross-script const X := f(OtherClass.CONST) initializer needs the reference resolved before the global class table is guaranteed populated. @export_range() upper bounds cannot take even a preloaded reference — export hint arguments must be true literals — so reaction_ticks/bot_*_reaction_ticks (ai_ship_controller.gd, match_mode.gd, spectate_mode.gd ×2) stay at a literal 60; these are editor-inspector slider bounds, not the timing math itself, so this doesn't reopen the bug the task exists to close, but it means the acceptance criterion below is met for tick-rate math and not for export-hint bounds ship.gd, training_mode.gd, new scripts/sim_constants.gd Tick-rate-derived timing math has no bare 60; changing TICK_HZ changes _tick_scaled and TICKS_PER_SIM_SECOND coherently. reaction_ticks export bounds remain literal by GDScript necessity
0.19 [P] DONE. AAMode gained MSAA_2X, appended (not inserted) so existing user://settings.cfg ordinals keep their meaning; default aa_mode changed to FXAA; settings_menu.gd's AA_OPTIONS now lists five entries video_settings.gd, settings_menu.gd Five AA options; default is FXAA; existing saved preferences migrate without resetting
0.20 [P] DONE. New autoload scripts/perf_overlay.gd (PerfOverlay), toggled by a new toggle_perf_overlay input action (F3 default). Headless-guarded; builds its own Label in code rather than touching HUD.tscn new scripts/perf_overlay.gd, project.godot [input] TIME_PROCESS vs total frame time tells the player whether they are CPU- or GPU-bound
0.21 [P] DONE. Shared HudInstrument._throttled_redraw(delta) paces queue_redraw() to ~60/s; value smoothing itself still runs every _process call, only the repaint is throttled scripts/hud_instrument.gd, scripts/hud_gauge.gd, scripts/hud_attitude_indicator.gd, scripts/hud_heading_tape.gd HUD is visually identical; instrument _draw call count is capped at ~60/s regardless of frame rate
0.22 [P] DONE. Engine.max_physics_steps_per_frame = 4 set in GameMode._ready(), applies to every mode including headless Training scripts/game_mode.gd A client throttled to 20 fps degrades smoothly instead of compounding
0.23 [P] DONE. New autoload scripts/background_fps.gd (BackgroundFPS) drops to 30 fps on NOTIFICATION_APPLICATION_FOCUS_OUT / restores on focus-in, independent of scene. main_menu.gd/settings_menu.gd each cap to DisplayServer.screen_get_refresh_rate() in _ready() (falling back to uncapped on a -1 query); leaving the main menu for a gameplay scene uncaps again via a new _leave_to_gameplay() helper, since gameplay has no cap of its own yet (0.17) new scripts/background_fps.gd, main_menu.gd, settings_menu.gd An unfocused window and an idle menu both stop rendering at 900 fps
0.24 [P] DONE. Both guarded with if DisplayServer.get_name() == "headless": returnarena.gd:_ready() skips the whole Environment block, video_settings.gd:_ready() skips apply_aa() scripts/arena.gd, scripts/video_settings.gd --headless allocates no Environment and no AA state
0.25 [P] DONE. _process still calls to_local() every frame (needed for the comparison itself) but skips set_shader_parameter() — the actual GPU-facing cost — below a 0.05 m movement threshold scripts/arena_boundary.gd Field shader behaves identically; the expensive call is skipped on most frames
0.26 [D:0.15b] Bake the arena GI and retire SDFGI (§5.7). arena.gd/goal.gd have no _process, no animation — the arena is fully static, and SDFGI is paying continuously to solve a dynamic-world problem this project does not have. Add UV2 to the arena shell, bake LightmapGI (or VoxelGI if bounce onto ships matters), disable sdfgi_enabled and re-evaluate ssil_enabled scenes/arena_base.tscn, scenes/arena_0*.tscn, scripts/arena_boundary.gd Largest frame-time reduction of any task here, with equal or better image quality; High preset keeps its look; bake is reproducible from a documented step
0.27 [P] DONE. lights_and_shadows/positional_shadow/atlas_size and directional_shadow/size set to 2048 (from the 4096 engine default), soft_shadow_filter_quality=2 project.godot Measurable frame-time reduction; no visible shadow-quality regression at 1080p
0.28 [D:0.15b] CLOSED, not implemented — the problem it targets doesn't exist. Was: prototype physics/3d/run_on_separate_thread (§5.7) to attack frame-time variance from the physics tick sharing the render thread — the riskiest item in this phase, since it changes when _integrate_forces runs relative to script code, and both ship.gd:346-357 and the RL training path depend on that. §5.5.2's real-hardware measurement (RTX 3090) found a 1.85 ms p50 / 2.98 ms p99 baseline with every graphics effect enabled — both comfortably under even a 240 Hz frame budget, with no meaningful p99-over-p50 variance to explain away. Taking on this task's real risk (reordering _integrate_forces relative to script code, with the RL training path depending on today's ordering) for a variance problem that isn't measurably present is a bad trade. Reopen only if a lower-end-hardware pass (§5.5.2's "still open" item) finds real physics-tick-driven variance that 0.26 and the preset ladder don't already cover (closed without a code change; see §5.5.2 for the evidence)
0.29 [P] DONE. Bounds check against wall_range/ceiling_range at the top of get_surface_pull, returning Vector3.ZERO before to_local() and the five _falloff calls whenever every term would be zero mid-arena scripts/arena_boundary.gd Identical flight feel and identical RL observations; measurable tick-time reduction with 7 bodies

These tasks exist because of the high-refresh-rate mandate, and their order matters. 0.15b blocked everything else, and did invalidate the a priori fps list — but not in the direction first assumed (see §5.5.1 vs §5.5.2): on the Mac the game looked GPU-bound and undifferentiated; on real reference hardware (RTX 3090, §5.5.2) it runs at 540 fps p50 with everything on, nowhere near bound by anything. 0.17/0.17b/0.19 (done) are still the right frame-rate levers — SDFGI/SSIL genuinely dominate the optional-effects cost, exactly as originally assumed, just at a much smaller absolute scale than feared on this hardware tier. 0.16 and 0.200.25 are the per-frame hygiene that makes a high frame rate worth having. 0.18 buys nothing today — it is what keeps a future 120 Hz simulation a config change plus a retrain rather than a protocol rewrite. 0.28 closed without a code change (§5.5.2) — the frame-time variance it targeted isn't measurably present on reference hardware.

0.190.29 are all pure single-player wins with no netcode content. If the multiplayer effort is ever paused, they should still land. Within them, 0.26 (bake the GI) is the largest single frame-time win in the document and costs no image quality — the arena is fully static, so SDFGI is paying continuously for a problem this project does not have (§5.7). 0.28 is the riskiest; it is the only Phase 0 task that can plausibly need reverting.

Correction — task 0.2 is wider than an earlier draft claimed. That draft argued the refactor was "narrow" because _build_merged_hull and _build_movement_vfx "only add_child()". That is exactly the problem: they add_child() onto self, the RigidBody3Dship.gd:208 (MergedHull: Hull, Canopy, EngineGlowL/R), :241 (engine cores), :268 (flames), :278 (lights). Leave those and §4.4's soft correct offsets only Nose and TailFin while the hull, canopy, glows, flames and lights stay welded to the corrected collider — every correction visibly tears the ship in half. The old acceptance criterion ("looks identical in Free Play") passes either way, which is why the criterion is now a child-type assertion. ship.gd:218's controller add_child correctly stays on the body; CollisionShape3D stays on the body.

Still true from that draft, and re-verified: ship.gd:44-47 documents why Nose/TailFin remain separate MeshInstance3Ds, and the RL path is untouchedship_observations.gd reads only global_position, basis, velocities and PhysicsServer3D contacts, and training_mode.gd's only get_node is arena.get_node("Boundary").

Phase gate: the game plays identically to master in Free Play, Match, Spectate, and headless Training, with Engine.time_scale never written — and additionally: §5.5 contains a real measured frame-time table (0.15b), the Low preset roughly doubles the frame rate of High (0.17), and the game looks correct uncapped on a high-refresh display with no 60 Hz stepping in FOV, shake or post-process.

Phase 1 — Transport, connection, lobby

# Task Acceptance
1.0 DONE, two real bugs found and fixed after adversarial review. tests/test_runner.tscn + test_runner.gd: discovers every *.gd under tests/cases/, instances it, calls every test_*() method via get_method_list(), aggregates failures, get_tree().quit(1 if failed else 0). tests/test_case.gd is the assertion base (assert_true/assert_eq/assert_almost_eq); case scripts use extends "res://tests/test_case.gd" (path-based) and the runner uses preload(), not a bare class_name reference — the global script-class cache isn't guaranteed populated on a fresh headless run (same class of issue as task 0.18's SimConstants). tests/cases/test_smoke.gd proves discovery/dispatch/aggregation and is the first real case file. An Opus subagent's adversarial review found: (1) GDScript has no exceptions, so a test that hit a runtime error before its first assert_* call left failures empty — exactly like every assertion passing — and was silently counted as a PASS. Fixed: TestCase now tracks assertions_made, incremented by every assert_*; the runner treats zero assertions as a failure in its own right ("made no assertions"). (2) A case file with a parse/compile error hung the whole runner forever — load() on a broken script does not return null here, it returns a non-null but uninstantiable GDScript resource, so a plain null check doesn't catch it; calling .new() on it threw an error severe enough to abort _ready() before ever reaching quit(). Fixed with Script.can_instantiate() as the real guard godot --headless --path Game res://tests/test_runner.tscn runs and exits 0; verified exit 1 with a deliberately-failing assertion, then removed. Re-verified both fixes with scratch case files (not committed): a test that null-derefs before asserting now correctly fails with "made no assertions" (exit 1, not a false pass); an uncompilable case file now fails loudly and promptly (exit 1, not a 124-timeout hang) while the other valid case files in the same run still execute normally
1.1 [D:1.0] [D:0.18] DONE. scripts/net_codec.gd: protocol constants, PacketType enum, channel ids, i16/i8/thrust-z-bin quantisers, pack_input/unpack_input, pack_snapshot_body_segment/pack_snapshot_client_header/pack_snapshot/unpack_snapshot. New scripts/net_body_state.gd is the plain per-body data holder the snapshot functions read/write (not Ship/Ball themselves, so the codec stays callable with no scene tree). NetCodec.TICK_HZ derives from SimConstants.TICK_HZ via preload() (same cache-timing reason as 0.18); ring sizes / seq windows / INTERP_DELAY / timeouts don't exist as constants yet — they land with the tasks that consume them (3.1+), so "derives from TICK_HZ" is satisfied for what exists today scripts/net_codec.gd, scripts/net_body_state.gd, tests/cases/test_net_codec.gd
1.2 [D:1.1] DONE, strengthened after adversarial review. scripts/network_manager.gd autoload (NetworkManager in project.godot [autoload]): host(port, max_clients)/join(address, port)/shutdown(), client_connected/client_disconnected/connected_to_server/connection_failed/disconnected_from_server signals forwarded from multiplayer's own, server_relay = false set the moment a peer exists, is_server/is_client state. Gained a shutting_down() signal, emitted at the top of every shutdown() regardless of role or reason — see task 1.4's row for why An Opus subagent's adversarial review (independently verified by the primary session before applying fixes) found the original tests/net_smoke.gd only proved each process exits cleanly on its own initiative, never that the OTHER peer actually observes the disconnect. Rewrote it: the host now waits for both client_connected and client_disconnected before passing; the client explicitly calls shutdown() mid-test (not just on process exit) and gives it a beat before quitting, same reasoning as §9 gotcha 26 for connects — a clean disconnect notice still needs a few poll() cycles to reach the wire, or the other side falls back to its ~5s peer timeout (gotcha 11) instead of a prompt one. Re-verified passing with both directions actually observed
1.3 [D:1.2] DONE for what exists today. NetworkManager._ready() calls get_tree().set_multiplayer_poll_enabled(false) (Godot 4.7's actual method name — the doc's set_multiplayer_poll(false) was shorthand) and exposes NetworkManager.poll() as the one entry point every caller uses instead. Verified against tests/net_smoke.gd, updated to poll from both _process and _physics_process every frame — connect/disconnect still works cleanly under manual-only polling (§9 gotcha 26 still applies: give a beat after a connect signal before shutdown). The per-call-site placement this task specifies (client: end-of-physics-tick flush after input send, top-of-frame receive; server: tick-start drain, tick-end flush) has no real per-tick caller yet — there is no input/snapshot traffic until tasks 1.4+/Phase 2 exist to send any, so there's nothing to place a flush after. That placement, and the RTT/staleness measurement below, land with the input pipeline, not as a separate task godot --headless two-process test still connects/disconnects cleanly with automatic polling off (verified). RTT/staleness improvement not yet measured — deferred until Phase 2/3's real per-tick traffic exists to measure against, same honesty as task 1.1's "constants that don't fully exist yet"
1.4 [D:1.2] DONE. scripts/match_net.gd autoload (MatchNet): _hello/_welcome/_player_joined/_player_left/_rejected RPCs, protocol_version (NetCodec.PROTOCOL_VERSION) and physics_ticks_per_second (SimConstants.TICK_HZ) checked on the server before a peer is added to roster; on mismatch, server sends _rejected with a readable string then disconnect_peer()s after a 0.3s beat (§9 gotcha 26 applies here too — a bare RPC then immediate disconnect would drop the rejection message). roster: Dictionary[int, PlayerInfo] never contains peer 1 (§1.1 decision 2). A new peer is told about the existing roster via targeted RPCs before the broadcast that tells everyone (including itself) about the new peer, so no client ever observes an unexplained peer_id Verified with a real two/three-process test (tests/match_net_smoke.gd/.tscn): matched client → both sides see player_joined/welcomed; deliberately wrong protocol version → client receives rejected("protocol version mismatch: server=1 client=100") and is disconnected. Caught and fixed one real bug in the process: the server's own roster update in _hello() didn't locally emit player_joined (the broadcast RPC is call_remote, never loops back to the sender)
Two more real bugs found by an Opus subagent's adversarial review, both confirmed independently and fixed. (1) _hello's player_name was completely unvalidated and broadcast verbatim to every peer — a demonstrated DoS: a multi-MB name relayed to all peers head-of-line-blocked the reliable control channel hard enough that a concurrently-joining client's own _welcome never arrived. Fixed with a hard MAX_INPUT_LENGTH = 256 reject (any legitimate client only ever sends local_player_name, which the UI already keeps short — anything past this is a bug or an attacker, not a name to politely truncate) followed by _sanitize_player_name(): strips control/formatting characters, clamps to MAX_PLAYER_NAME_LENGTH = 24, falls back to "Player" if empty. (2) MatchNet.roster was never cleared when a HOST stopped hosting — only the client-side disconnect path cleared it, so Host → Lobby → Leave → Host again left a phantom player in roster permanently, mis-balancing teams and getting broadcast to every future joiner. Fixed via NetworkManager's new shutting_down() signal (task 1.2), which MatchNet now clears roster on unconditionally, regardless of role or reason _sanitize_player_name is static (pure function of its argument) with 5 dedicated unit tests in tests/cases/test_match_net.gd, plus a live rejection test (match_net_smoke.gd --role=client-longname, a 500 KB name, confirmed rejected before ever reaching a broadcast). New regression test match_net_smoke.gd --role=host_recycle: host, client joins (roster.size()==1), host leaves and re-hosts, confirms roster.is_empty() before any new connection — reproduced the bug pre-fix, confirmed fixed post-fix
1.5 [D:1.4] DONE, strengthened after adversarial review. scenes/lobby.tscn + scripts/lobby.gd: roster split into two team columns (dynamically rebuilt Label rows on MatchNet.player_joined/player_left/player_state_changed/welcomed), Switch Team + Ready CheckButton (server process gets a read-only view — never a roster member, §1.1 decision 2), Leave. MatchNet grew team/ready fields on PlayerInfo, a balanced-team auto-assign on join (_pick_balanced_team), and request_set_team/request_set_ready + their server-authoritative RPCs, broadcasting _state_changed the same way _player_joined already did Verified with a real two-process test (tests/lobby_smoke.gd/.tscn) that loads lobby.tscn via change_scene_to_file exactly as main_menu.gd's Host/Join flow (task 1.7) does, then presses the real %SwitchTeamButton/%ReadyButton nodes via a persistent test-only helper (tests/lobby_test_hooks.gd, not a project autoload — parented under get_tree().root so it survives the scene swap, never referenced by production code). An Opus subagent's adversarial review found the original test's host role never actually loaded lobby.tscn at all — it only hosted and waited, so lobby.gd's is_server branch (the read-only view a self-hosting player reaches via main_menu.gd's own Host button — a real, production-reachable path, not a hypothetical) had never run under this task's own suite. Fixed: the host role now loads lobby.tscn too and a new run_host_test() in the shared test helper verifies %ControlsRow is hidden, the roster row renders, and the status text is correct, holding the connection open long enough (MIN_HOST_LIFETIME_SECONDS) for the client's own longer flow to finish against it. Confirmed: roster renders correctly server- and client-side (now genuinely, not just asserted), team switch moves the row to the other column, ready toggle updates the checkbox and the label's ✓ marker, row count matches roster size on both peers
1.6 [D:1.4] [P] DONE. scenes/server_boot.tscn + scripts/server_boot.gd: --port=/--max-clients=/--log-level= from OS.get_cmdline_user_args(), Engine.max_fps = 60, structured [elapsed] LEVEL event key=value… log lines for server_started/peer_connected/player_joined/player_left/peer_disconnected, and a physics-overrun watchdog comparing Engine.get_physics_frames() deltas frame-to-frame. Does not spawn a match yet — that's Phase 2's networked_match.gd — this is just the process shell: listen, log, idle cheaply. Two real bugs caught and fixed while verifying, both in the watchdog: (1) the very first _process() after boot compared against a pre-_ready() baseline and logged a spurious one-time steps=5; skip the first measurement. (2) the initial steps > 1 threshold fired continuously (every 30100ms) on a perfectly idle, healthy server — because §9 gotcha 6 means frames legitimately alternate between 0 and 2 physics ticks under physics_jitter_fix = 0.0, not a flat 1/frame; that's quantisation, not backlog. Raised the threshold to steps > 2 (3+ ticks = the accumulator actually failing to drain), which produced zero false positives over a 4.8s idle run Verified with real headless runs: idle CPU measured via ps -o %cpu at 0.0% (bar is <5%); a real client connect/disconnect via tests/net_smoke.gd --port= produces exactly the expected 4-line log sequence with no spurious warnings
1.7 [D:1.5] [P] DONE. main_menu.tscn gained a Multiplayer section (Host button; Join row with an IP LineEdit, default 127.0.0.1; inline error label) and a full-screen ConnectingOverlay (status label + Cancel). main_menu.gd: _on_host_pressed calls NetworkManager.host() then goes straight to lobby.tscn (synchronous — no overlay needed); _start_join calls NetworkManager.join(), shows the overlay, and starts an app-level CONNECT_TIMEOUT_SECONDS = 6.0 timer; _on_connected_to_server/_on_connection_failed/Cancel/timeout each resolve to the overlay hiding and either lobby.tscn or a visible error, gated by a token counter so a late/stray signal after the attempt was already resolved is a no-op Verified with real multi-process runs of scenes/main_menu.tscn itself (not a wrapper — driven by a temporary-autoload test helper, tests/main_menu_test_hooks.gd, pressing the real HostButton/JoinButton/ConnectingCancelButton) across all four paths: Host → lobby.tscn; Join → connects → lobby.tscn; Join with nothing listening → times out → error shown, stays on menu; Join → Cancel → overlay hidden, stays on menu, is_client false. Two real bugs found and fixed in the process, both pre-existing from earlier Phase 1 tasks, not new to 1.7: (1) NetworkManager's clock ping (task 1.8) gated only on is_client, which turns true the instant join() is called — a slow or refused connect attempt spammed "Trying to call an RPC via a multiplayer peer which is not connected" every frame; fixed by also requiring _peer.get_connection_status() == CONNECTION_CONNECTED. (2) ENet's own connection_failed proved unbounded in practice — verified empirically against a genuinely refused loopback connection, it hadn't fired even 14s in — which would have left a player staring at "Connecting…" indefinitely; task 1.7's own CONNECT_TIMEOUT_SECONDS is what actually satisfies "connection-refused reaches a sane UI state", not the built-in signal alone
1.8 [D:1.2] [P] DONE, strengthened after adversarial review. Folded into network_manager.gd: client pings the server once a second (_ping/_pong RPCs, reliable, channel 0); clock_offset_ms is the min-RTT sample in a rolling 5s window (_clock_samples, pruned by wall time); get_server_time_estimate_ms() is the public API later phases (INTERP_DELAY, tick_offset seeding) will actually call; clock_updated(rtt_ms, offset_ms) signal for observers. New scripts/net_debug_overlay.gd autoload (F4, toggle_net_overlay input action) mirrors perf_overlay.gd's headless-guarded pattern, shows RTT + offset client-side or peer count server-side Verified with a real two-process test (tests/clock_smoke.gd/.tscn) on localhost: first sample at t=0.95s, offset converged to 1534.50ms by t=2.0s (well inside the 2s bar), and stayed within 1.5ms of that value through t=3.96s — comfortably under the ±1 tick (16.67ms) bar. An Opus subagent's adversarial review correctly pointed out this self-consistency check couldn't have caught a systematically-wrong-but-stable offset (e.g. a missing /2 on RTT, or a sign flip — it would converge just as cleanly). Fixed by adding an independent ground-truth cross-check: both host and client compute Time.get_unix_time_from_system()*1000.0 - Time.get_ticks_msec() (each process's own offset from the shared OS wall clock — the same real clock on both, since they're on the same machine), exchanged via a shared temp file written by the host, purely for test orchestration and touching no production code. The true required offset is just the difference of those two numbers; re-run measured the converged offset against it and found 0.99ms of error, comfortably inside a deliberately loose 250ms tolerance (OS wall-clock read resolution and sampling-instant skew, not NetworkManager's own precision, is what sets the tolerance floor here). Note the converged offset value itself is large and arbitrary (~1.5s) because Time.get_ticks_msec() counts from each process's own start, not a shared epoch — expected, and exactly what clock_offset_ms exists to absorb

main_menu.gd gains its first async flow. Every existing handler is GameSettings.x = y; change_scene_to_file(...) — there is no loading screen, no error state, and no back-navigation state machine to extend. Budget for that.

Phase gate: two clients connect to a headless server, appear in a shared lobby, ready up, and disconnect cleanly.

Phase 2 — Server-authoritative simulation, dumb client

No own-ship prediction yet: the client renders everything, including its own ship, from the interpolation buffer. Unplayable over the internet, fine on LAN, and it proves the whole state pipeline before prediction complicates the picture.

This phase is load-bearing, not throwaway — the codec, slot mapping, snapshot pipeline, interpolator and HUD signal surface all survive into Phase 4. Roughly ten lines get discarded.

# Task Acceptance
2.1 [D:1.4] DONE. New MatchSim autoload (scripts/match_sim.gd) carries all Phase 2 hot-path RPCs (match_config, input, snapshot, score_update) per §1.1's "hot RPCs live on autoloads" decision — NetworkedMatch itself (scripts/networked_match.gd + scenes/networked_match.tscn, no HUD child) stays a plain scene node with no networking identity of its own. Server builds deterministic team/spawn-index slots by iterating MatchNet.roster.keys() sorted, loads a random arena via ArenaRegistry.random_path(), spawns ball/ships, then send_match_config()s. Client validates the received arena_path against ArenaRegistry.ARENAS before loading it Both peers spawn an identical tree in real two-process runs (tests/networked_match_smoke.gd/.tscn); an invalid arena path is refused before load
2.2 [D:2.1] DONE. Server reuses RLShipController as the remote-input controller exactly as the architecture doc anticipated — each connected peer's real Ship is driven by one, fed by MatchSim.input_received. _broadcast_snapshot() runs every physics tick (60 Hz), packing NetBodyState for every ship + ball via NetCodec.pack_snapshot_body_segment and sending per-slot, filtered through multiplayer.get_peers() so a disconnected peer doesn't get an RPC send attempt Server-side snapshot cadence confirmed stable at 60 Hz across multiple two-process runs; no "unknown peer ID" spam after the get_peers() filter fix (found via a real disconnect-mid-test case)
2.3 [D:2.2] DONE. New scripts/net_interpolator.gd (class_name NetInterpolator, RefCounted) buffers up to MAX_SAMPLES=16 timestamped NetBodyStates per remote body and produces interpolated (or clamped-extrapolated, MAX_EXTRAPOLATION_MS=150) states at any fractional server tick via sample_at(). Client-side _on_snapshot_received feeds each body's decoded state into its interpolator; ships/ball spawn FREEZE_MODE_KINEMATIC so they never call _integrate_forces/get_action() Client observed 31.43 m of real, physics-verified movement over a 2s held-thrust drive purely from interpolated snapshots, no local simulation
2.4 [D:2.3] DONE — dual-time remote entities (§4.1). Collider updates happen in _physics_process at server_time_est (present-time, correct contact resolution); $Visual updates happen separately in _process at server_time_est - INTERP_DELAY (physics_interpolation_mode = OFF, since the node's transform is overwritten every rendered frame). _current_interp_delay_ms() computes a simplified INTERP_DELAY (one_way + interval*1.5, clamped [25,200] ms) — no jitter term yet, that lands with Phase 3's jitter buffer Verified via the smoke test's separate collider/visual checks; Engine.get_physics_frames()/Time.get_ticks_msec() epoch correlation (NetInterpolator.to_tick()) confirmed working with no extra sync handshake needed
2.5 [D:2.3] [P] DONE. _send_local_input() samples via a stateless, never-added-to-tree PlayerShipController instance (reading real Input state) and sends the resulting ShipAction every physics tick, no redundancy/buffering yet (Phase 3) Input reaches the server and visibly moves the ship — confirmed via a real held move_forward keypress driving 31.43 m of server-authoritative movement
2.6 [D:2.3] [P] DONE, and empirically verified, not just inferred — turned out to already be satisfied as a natural consequence of 2.12.5's implementation (_apply_ship_visual_state already calls set_visual_action for remote ships; _process's ball branch already calls set_visual_speed) Smoke test explicitly reads interpolator.latest().thrust_z mid-drive and asserts >0.5 while move_forward is held (not inferred from movement alone) — measured thrust_z=1.00
2.7 [D:2.3] [P] DONE, also a natural consequence of the above — spawn_camera_rig(_my_slot.ship) and _spawn_hud() are called once the client's own ship is identified in _on_match_config_received Smoke test asserts _camera_rig and hud both is_instance_valid() on the client; confirmed true in every clean run
2.8 [D:1.1] [P] DONE. New NetSim autoload (scripts/net_sim.gd): seeded (--net-sim-seed=, fixed default so a bad run reproduces), CLI-driven (--net-sim-latency=/--net-sim-jitter=/--net-sim-loss=/--net-sim-dup=), a pure passthrough (send() calls the dispatch immediately) unless at least one flag is non-zero — confirmed byte-for-byte inert against every Phase 1/2 regression test with no flags set. Wraps MatchSim.send_input/send_snapshot per this row's original scope, plus NetworkManager's _ping/_pong dispatch — a deliberate scope addition, since that's the only RTT measurement that already exists and is already tested (task 1.8), so it's what makes this task's own acceptance criterion checkable today without waiting on Phase 3's per-peer snapshot echo. "Asymmetric-capable" needs no special-case code: each process reads only its own CLI args and delays only its own outgoing sends, so hosting and joining with different flags is already asymmetric. One correctness subtlety, caught before it shipped: callers that embed a timestamp in a wrapped RPC (_ping/_pong) must capture Time.get_ticks_msec() before calling NetSim.send(), not inside the wrapped Callable — capturing it inside would silently absorb that process's own added delay out of the round-trip measurement instead of adding to it, since the timestamp would then reflect "after my delay" rather than "when I actually tried to send". A second real bug, found by actually running Phase 2's own milestone gate (a full networked_match_smoke run under --net-sim-latency=80 --net-sim-jitter=20, not just the isolated ping/pong test above): a delayed send can outlive the window its target was valid in — the host hit "Attempt to call RPC with unknown peer ID" (the client had disconnected during the ~80-100ms hold, after _broadcast_snapshot's existing get_peers() filter had already passed at schedule time) and the client hit "'_recv_input' on yourself is not allowed by selected mode" (its own shutdown() had already reset multiplayer_peer to a fresh OfflineMultiplayerPeer before a still-pending delayed send fired, so peer id 1 now meant itself). Fixed by having send() accept an optional target_peer_id and re-validating it — plus that this process still has a real (non-Offline) peer at all — at fire time inside a new _fire(), not just at schedule time; the synchronous/inactive path is deliberately left unvalidated so NetSim stays a true no-op when idle Verified with a real two-process test (tests/net_sim_smoke.gd/.tscn): baseline (no flags) observed rtt_ms=7.00 on loopback; --net-sim-latency=80 on the host alone raised the client's observed rtt_ms to 83.00 (want ≥70, confirmed measurably higher than baseline); --net-sim-loss=1.0 on the host produced zero pong samples over 7s (rtt_ms stayed -1, confirmed the drop path actually drops rather than relabels). Phase 2's own milestone gate re-run and passing: networked_match_smoke under --net-sim-latency=80 --net-sim-jitter=20 on both peers — client still observed 26.08m of clean server-authoritative movement via interpolation, thrust_z=1.00 confirmed mid-drive, zero RPC errors (the fire-time-revalidation fix above). Re-ran the full Phase 1 + Phase 2 regression suite (test_runner, net_smoke, match_net_smoke, clock_smoke, lobby_smoke, server_boot, networked_match_smoke) with NetSim present but inactive — all still pass with unchanged behaviour (clock offset converged to the same value, networked_match_smoke still showed clean server-authoritative movement)

| — | An Opus subagent's adversarial review of all of Phase 2 found real, verified bugs the smoke tests couldn't catch, since constant-velocity dead reckoning still moves a ship far enough to pass a moved > 1.0 check. Fixed, all independently re-verified with real two-process runs and temporary instrumentation (removed after confirming):

(1) The interpolator never actually interpolated — every sample_at() call took the extrapolation branch, 100% of the time, LAN or under simulated latency alike. NetInterpolator.to_tick() assumes Time.get_ticks_msec() == physics_frame * TICK_MS on the server; real engine/autoload startup work before the first physics step (plus any dropped tick, which only ever widens it) breaks that by a steady +45-55ms in practice. networked_match.gd now tracks one shared _tick_bias_ms estimate (_update_tick_bias, called from _on_snapshot_received) — the minimum to_tick(server_time_est) - server_tick over a rolling 5s window, same rationale as NetworkManager's own min-RTT filtering: the least-delayed sample best isolates the constant bias from per-packet transit noise, and a rolling (not all-time) window still tracks a real future increase. _estimated_tick() subtracts it before every to_tick() call. Verified: bias converged to ~50-56ms (matching the bug's own measured magnitude exactly) and real interpolation rose from 0% to ~70% of calls (interp=436 extrap=182 out of 618, up from interp=0 extrap=617). A first attempt at this fix was itself broken and made the lead ~30x worse (90+ ticks, ~1.5s): early snapshots arrive before NetworkManager's first clock pong lands (rtt_ms < 0, clock_offset_ms still 0.0), so server_time_est briefly means "my own raw local uptime" — a wildly wrong bias sample that the 5s rolling-min then locked onto for a whole short test, since 5 real seconds never fully elapsed before the test ended. Fixed by skipping bias recording entirely while rtt_ms < 0.

(2) Goals caused a ~27m visual slide. _on_goal_scored bumped _reset_gen immediately, but reset_ball()/reset_ships() only queue teleports (task 0.15, applied on each body's next _integrate_forces) — so the broadcast that same tick carried the NEW gen with the OLD (still-in-goal) position, and the client's buffer-clear-on-reset kept exactly that stale sample and lerped a full-arena slide to the next, genuinely-reset one. The first fix attempt (defer the bump to "the next _physics_process" via a plain boolean) didn't work either — emperically, the goal Area's body_entered signal fires as part of physics tick N's own step, before tick N's _physics_process callback, so a flag set in the handler is already true by the time that same tick checks it: no delay was actually introduced. Fixed by recording the tick the goal was detected on (_pending_reset_gen_bump_tick) and only bumping once Engine.get_physics_frames() > _pending_reset_gen_bump_tick — i.e. strictly on a later tick, which guarantees the queued teleport's _integrate_forces has already run. Verified by forcibly teleporting the ball into a goal mid-test and logging the server's own broadcast stream tick-by-tick: gen change and the already-reset position now land in the identical broadcast, every time.

(3) _local_input_sampler (a PlayerShipController, i.e. a plain Node) was created but never added to the tree and never freed — this was the unexplained "3 resources still in use at exit" warning on every prior Phase 2 test run, confirmed by --verbose naming the exact leaked script chain and by the warning disappearing once a _exit_tree() cleanup was added. Also leaked on the server despite its "client only" comment, since the field initializer is unconditional.

(4) Ball angular velocity decoded 8x too smallNetCodec.rescale_avel() exists specifically to correct a ball's decoded angular_velocity from the ship-range assumption unpack_snapshot() decodes every body with, and was never called. Dormant today (nothing read decoded angular_velocity yet) but silently wrong the moment ball-spin VFX or Phase 4 prediction reads it; now called in _on_snapshot_received.

(5) get_server_time_estimate_ms() was used unguarded before the clock had synced, contradicting its own doc comment — against a long-running dedicated server this freezes every remote body at the oldest buffered pose for the whole first second of every match (clock_offset_ms == 0.0 compares this process's own short uptime against the server's much larger tick count). Both _physics_process and _process now skip their collider/visual update entirely while NetworkManager.rtt_ms < 0.0.

Smaller fixes, all confirmed via the regression suite: net_sim.gd's delayed-send timer now uses process_always = true (a simulated wire shouldn't stop just because the local game pauses) and _fire() also checks get_connection_status() == CONNECTION_CONNECTED, not just non-Offline, before dispatching (a known, accepted residual gap remains: a shutdown-then-reconnect inside one delayed send's hold window isn't fully closed, judged disproportionate to fix for debug-only tooling); _broadcast_snapshot() now appends one body per slot unconditionally (a zeroed placeholder for a momentarily-invalid ship) so the ball's fixed index assumption can't silently break if "no ship is ever despawned" (§6.4) ever stops holding; networked_match.gd now only declares score_changed (the one signal it actually emits) instead of also declaring timer_updated/match_ended/kickoff_countdown/overtime_started, which — despite never being emitted — made HUDController show a permanently frozen timer widget purely because has_signal("timer_updated") was true.

Confirmed fine, not just assumed, via a real hostile-client stress test and a real 3-process multi-client run: a malformed/garbage/oversized _recv_input payload cannot crash the server (Godot's StreamPeerBuffer silently zero-fills past EOF; count is a bounded u8); NetworkedMatch skipping GameMode._ready()'s super() call drops nothing load-bearing; deterministic team/spawn-index slot assignment is correct with 2 simultaneous clients (verified with a real 3-process host+2-client run); RPC authority enforcement on _match_config/_score_update/_snapshot genuinely rejects a forging client server-side | Full Phase 1 + Phase 2 regression suite (test_runner, net_smoke, match_net_smoke, clock_smoke, lobby_smoke, networked_match_smoke baseline and under the --net-sim-latency 80 --net-sim-jitter 20 milestone gate, net_sim_smoke) re-run clean after every fix |

net_sim.gd belongs in this phase, not Phase 3. A LAN-only phase gate passes even with §4.1's flaw fully present, because LAN INTERP_DELAY sits at the clamp floor and closing-speed error is small. Phases 2 and 3 would both go green and Phase 4 would discover the architecture is wrong.

Phase gate — MILESTONE: a real 1v1 at --net-sim-latency 80 --net-sim-jitter 20, not just on LAN. Ships fly, the ball moves, goals detect server-side.

Phase 3 — Input pipeline hardening

# Task Acceptance
3.1 [D:2.5] DONE. Client sends the last NetCodec.MAX_REDUNDANCY (4) ticks' actions per packet, newest-first (the wire format already supported this from Phase 1 — Phase 2 just wasn't using it). Server gains a real per-slot ring buffer, new standalone scripts/input_jitter_buffer.gd (InputJitterBuffer, RefCounted, no scene dependency — same reason net_codec.gd/net_interpolator.gd are pure classes), consuming exactly one sequence number per physics tick Verified both by unit test (test_redundancy_survives_3_packet_burst_loss) and live: 25% random simulated input loss produced zero observed starvation ticks; 100% loss correctly produced zero seeding/consumption (no crash, ship simply never receives a command)
3.2 [D:3.1] DONE. InputJitterBuffer.consume(): repeat-last on starve, zero + stalled=true only after STARVE_ZERO_TICKS (30 = 500ms). input_buffer_depth/last_input_seq/echo_client_send_ms are now genuinely per-peer in every snapshot (_broadcast_snapshot builds them from each slot's own InputJitterBuffer), replacing Phase 2's hardcoded zeros One real bug found wiring this into a live match: the server's ring buffer started counting its own "expected sequence" from a local 0 the instant a slot was created — well before that player's first real packet could possibly arrive (connection/spawn setup takes real time) — so the two numberings never converged and the ship silently never moved. Fixed by seeding last_applied_seq from the client's own numbering on first real ingest(), not assuming a shared from-zero baseline. Verified with real two-process runs before and after the fix
3.3 [D:3.2] [P] DONE. New standalone scripts/input_lead_controller.gd (InputLeadController, unit-tested like InputJitterBuffer): clamp [1,12], fast attack (+3, debounced to once per 30 ticks) on any server-reported starve, slow release (1 per 60 ticks) gated behind a one-time 2s clean-surplus bar. A lead change is realized as extra distance between the client's own outgoing seq and what the server has consumed — attack skips extra seq numbers, release duplicates (re-sends) the current one; the server's ring buffer needs no special handling for either, since a skip is an ordinary drop and a duplicate is a same-seq resend already discarded Verified live: on a clean LAN, one early attack (a momentary connection-setup hiccup) recovered via two releases within ~4s, settling back near minimum; under sustained 30% simulated loss, lead climbed to 7 via repeated attacks and never released while genuine loss continued — confirming debounce, attack, and release gates all fire correctly on real conditions
3.4 [D:3.1] [P] DONE. MatchSim._recv_input validates before decoding: per-peer rolling-1s rate limit (packet count AND byte budget, §3.1's own numbers), disconnect after 3 consecutive over-budget seconds; framing (redundancy count + payload size checked against NetCodec's own layout, since StreamPeerBuffer silently zero-fills past EOF instead of erroring — a Phase 2 adversarial-review finding), disconnect after 20 malformed packets. networked_match.gd additionally rejects seq > server_tick + 20 and counts (rather than silently ignoring) input from a peer with no slot. Server-side input_lead enforcement from arrival times was scoped down to observability rather than active enforcement — see the note below the table Two new permanent regression tests (networked_match_smoke.gd --role=client-abuse-malformed / client-abuse-flood) call MatchSim._recv_input directly with garbage bytes and a legitimate-but-too-frequent flood respectively — bypassing the honest client encoder entirely, i.e. exactly what a hostile custom client sending raw ENet packets looks like. Both confirm a real disconnect, not just tolerance. Found and fixed two smaller bugs getting these to pass cleanly: a GDScript lambda-captures-by-value mistake in the tests themselves (fixed by capturing a single-element Array instead of a plain bool), and a real race where NetworkManager's own ping/pong reply could target a peer a concurrent abuse-disconnect had just removed from the same poll() batch (now guarded); the post-shutdown physics path now also uses the safe NetworkManager lifecycle flag and stays error-free
3.5 [D:3.2] [P] DONE. tests/cases/test_input_jitter_buffer.gd and test_input_lead_controller.gd: sequential consumption, redundancy surviving a 3-packet burst loss (3.1's own acceptance text, verbatim), starvation repeat-then-zero timing, stale/reordered-packet handling, buffered-depth reporting, ring-wraparound slot-tagging safety, and the full attack/debounce/release state machine including a starve mid-release-window forcing a fresh clean-surplus wait 14 new tests, all passing (test_runner.tscn: 33 total, 0 failed)
3.6 [D:2.8] DONE, with one honest scope note. networked_match.gd's client can swap its input sampler for a real AIShipController (--test-bot, optionally --test-bot-model=) instead of PlayerShipController — parented onto the client's own ship via Ship.set_controller() since (unlike the human sampler) it needs real scene context. Known limitation, documented in code: this client's ships are all FREEZE_MODE_KINEMATIC, driven purely by transform writes, so nothing ever writes linear_velocity/angular_velocity onto them — the bot's observations always see every ship as stationary. It still produces well-formed, bounded actions from that degraded input (the policy network's output layer is bounded regardless of input quality), sufficient for this task's actual job (CI traffic generation, not bot skill). New CI driver tests/networked_match_ci.gd/.tscn: headless server + two headless --test-bot clients. This task's own original acceptance text names "p95/p99 prediction error" and "snap count" — both Phase 4 concepts that don't exist yet (no client-side prediction or hard-snap threshold exists before Phase 4); asserting on data that doesn't exist would be fabricated, so those two are explicitly not checked, with the gap called out in the driver's own header comment rather than silently dropped Real 3-process runs: both bots' independently-written final scores agreed after a deterministically forced goal (bot-vs-bot scoring isn't reliable enough within a short run to gate on), both saw 500+ snapshots over an 8s run (well above the 60Hz-scaled floor), all three processes exited 0. "Clean stderr" is the external invocation's job (grep the captured output), same as every other smoke test in this project — verified manually, not self-asserted by the script
3.7 [D:2.8] [P] DONE, with prediction error deliberately omitted (documented, not silently dropped — same Phase 4 gap as 3.6). Extends net_debug_overlay.gd with jitter (new RFC3550-style EWMA in NetworkManager, from raw per-sample RTT — Phase 1's rtt_ms is a min-filtered sample, deliberately jitter-insensitive by design, so it can't answer this on its own), snapshot loss (new EWMA in networked_match.gd over each received snapshot's own server_tick gap — snapshots go out at a steady one-tick cadence, so a gap is direct evidence of a drop or reorder), snapshot age (computed on demand from the same bias-corrected tick estimate the interpolator itself uses), input buffer depth and input_lead (both already tracked client-side for 3.3), and bandwidth (new rolling per-second byte counters in MatchSim, the two 60Hz hot-path channels only) Verified values are live and plausible, not just present, by calling get_net_debug_stats() directly in a real two-process test: bandwidth matched the wire format's own byte math almost exactly (measured ≈2400 B/s sent against a computed 40B×60Hz, ≈3540 B/s received against 59B×60Hz for a 1v1), and buffer depth/lead/loss all moved in the correct direction between a clean LAN run and one under simulated 60ms latency + 10% loss

AIShipController runs a policy in pure GDScript with no Python or ONNX dependency, so 3.6 gets a competent automated player for free.

Server-side input_lead enforcement from arrival times (§3.3's closing paragraph) was scoped down to observability, not built as active enforcement. The concrete, mechanically well-specified parts of task 3.4 (rate limiting, malformed-packet counting, seq-range rejection, disconnect policy) fully close the load-bearing security gaps; the advantage a client gains from claiming a dishonestly low input_lead is explicitly described in the doc itself as "small" (reduced apply latency, not an outright cheat — there's no prediction/reconciliation yet for a bad lead to actually corrupt), and building real arrival-jitter-derived enforcement well — without risking a third, subtly-interacting control loop on top of the two §3.3 already warns against — is a genuine design task in its own right, not a mechanical one. Revisit if Phase 4's prediction work turns "slightly lower latency" into a sharper edge.

Phase gate — MET. Both networked_match_smoke and the CI driver (task 3.6) re-run under the gate's own exact condition, --net-sim-latency 80 --net-sim-loss 0.05, on every peer: the human-driven smoke test still shows clean server-authoritative movement (22.61m over the usual 2s drive), and the two-bot CI run still shows both clients independently agreeing on the final score after a forced goal, 495+/504 snapshots received each, all three processes exiting 0 with clean stderr.

| — | A second adversarial review of the fix commit above found that two of its nine fixes silently cancelled each other out, re-creating the original critical bug at a lower failure threshold — plus four smaller real issues, all re-verified with real two- and three-process runs.

CRITICAL — the seq-range guard fix (round 1's MEDIUM item, gotcha 42) made the ring-overflow resync fix (round 1's CRITICAL item, gotcha 39) unreachable in production. The guard bounded every accepted seq at last_applied_seq + RING_SIZE — the consumer's position — which in turn caps InputJitterBuffer's own highest_ingested_seq at that same ceiling, since nothing above the bound is ever allowed to reach ingest() at all. But consume()'s resync condition needs highest_ingested_seq to reach expected + RING_SIZE, one full ring past that same ceiling — arithmetically impossible on the only call path that exists. The two fixes read as independent (one in the jitter buffer, one in the caller) but shared a variable and quietly defeated each other; the round-1 commit's own new unit test for the resync never caught it because it called ingest() directly, bypassing the guard entirely — the exact composition the bug lived in. Verified failing on the committed code: a 0.6s SIGSTOP host freeze reproduced the original 0.00m death, at a lower threshold than the pre-round-1 bug (~0.6s vs ~0.7s), reachable via ordinary server tick loss with no external trigger at all (Engine.max_physics_steps_per_frame = 4 means a server that falls behind wall-clock time during any stall never catches back up on its own). Fixed by rebinding the guard to highest_ingested_seq (now a public field, matching last_applied_seq's own convention) instead of last_applied_seq — the client's actual send epoch, which ingest() updates once per accepted packet regardless of how far the consumer has fallen behind, rather than the consumer's own lagging position. Re-verified against a real 2-bot CI match with a 1.5s host SIGSTOP freeze injected mid-run (well past the 0.6s failure threshold): both peers kept moving (47.46m / 10.19m and, on a repeat run, 25.62m / 17.96m), stalled=false, sampled while genuinely still connected.

HIGH — the InputLeadController release fix (round 1's HIGH item, gotcha 40) was itself incomplete. Round 1 added a real depth check (input_buffer_depth > TARGET_DEPTH) but left the original gate, and lead > LEAD_MIN, still ANDed onto the same final condition — so a backlog the controller never caused (lead pinned at its own floor) still could never release, since that old clause always failed regardless of what the new depth check found. Confirmed by the round-1 commit's own new unit test, whose assertion text literally read "lead cannot release below its own floor even under large surplus" as if that were the intended behaviour. Fixed by splitting the one gate into two independent decisions: whether to duplicate this tick's seq (the only thing that actually narrows real buffered depth) now follows the depth signal alone; whether to keep decrementing lead's own bookkeeping below its documented [LEAD_MIN, LEAD_MAX] floor is a separate, purely cosmetic choice made inside that branch.

MEDIUM — task 3.6's CI gate (round 1's own fix for gotcha 43) still sampled after both bots had legitimately disconnected. The fix used a run_seconds - 0.5 margin, narrower than the original bug (sampling after the full run) but still not enough: multiplayer.get_peers() at sample time was already empty, and the check was only passing on STARVE_ZERO_TICKS's own ~200ms of residual starvation grace, not because it was genuinely still connected as its own print claimed. Widened the margin to run_seconds - 2.0 and added an explicit slot.peer_id in multiplayer.get_peers() assertion at sample time, so a future regression in either direction fails loudly here instead of silently passing on residual grace.

LOW — the human smoke test's movement bar was beatable by gravity alone. moved > 1.0 measured full 3D distance; a 1.2s window of completely dead input still registered ~1.07m from pure vertical settling (spawn height dropping to the floor) — above the bar, with only the separate thrust_z_ok check actually catching the failure. Forward thrust is a horizontal force, so switched to XZ-only displacement, which gravity alone cannot satisfy regardless of spawn height or timing.

LOW — "clean stderr" wasn't actually clean. Every disconnect logged ERROR: Unable to send packet on channel 0, max channels: 0 from match_net.gd's _remove_player, which broadcasts _player_left to every peer in multiplayer.get_peers() — including, transiently, the peer that just disconnected (whose own ENet connection can still be momentarily present in that set with its channels already torn down), and — found only after the first fix still left an error in the 2-bot CI scenario specifically — including a second still-connecting peer when two clients disconnect within the same poll() batch, since get_peers() hadn't yet been updated for the one not currently being handled. Fixed by deferring the whole notification (call_deferred) to the next idle frame, by which point poll() has fully returned and every disconnect event in the batch has actually settled, then explicitly excluding the peer that left. Re-verified clean (grep for ERROR) across both the basic 2-process smoke test and a real 2-bot CI run with a mid-match host freeze injected.

Noted, not fixed — a related but distinct stderr source in _broadcast_snapshot. The deliberately-adversarial client-abuse-malformed smoke test still logs one Unable to send packet from networked_match.gd's snapshot broadcast, racing a host-forced disconnect_peer() in match_sim.gd's abuse-disconnect path against the same tick's connected_peers.has(slot.peer_id) snapshot — a different call site than the one just fixed, only reachable via the abuse-detection disconnect path rather than a normal client-initiated one, and out of scope for this pass. Left for a dedicated look rather than a rushed fix under this round's time pressure.

Confirmed fully correct, not just re-asserted: the _input_history fix (round 1's LOW-MEDIUM item) was re-verified via a synthetic-marker harness stamping a computable value into every outgoing action and checking it through a real 3-process match under 60ms+40ms jitter+18% loss — 1080 marker checks, 0 mismatches, including real attacks and releases; the leaky-bucket rate limiter (round 1's MEDIUM-HIGH item) cannot false-positive on honest traffic (~1.8x measured margin under real impairment); the resync boundary arithmetic itself is correct under packet reordering and duplication. The lesson that mattered most this round wasn't any single fix — it was that two fixes landed in the same commit, each individually correct in isolation, that silently cancelled each other out (see gotcha 45) | Full regression suite (35 unit tests, the basic 2-process smoke test, the malformed/rate-limit/duty-cycle abuse roles, and a real 2-bot CI run with a 1.5s host SIGSTOP freeze injected mid-match) re-run clean after every fix in this round

| — | An Opus subagent's adversarial review of all of Phase 3 found a critical, silent, permanent bug plus eight smaller real issues — all empirically verified with real two- and three-process runs, not just code reading.

CRITICAL — InputJitterBuffer's 32-entry ring permanently bricked a player's input on any backlog bigger than the ring. consume() advanced last_applied_seq by exactly 1 per tick with no resync; once the un-consumed backlog exceeded RING_SIZE, a fresh arrival would land in the exact slot consume() was still waiting on, and since both counters only ever advance, the gap never closed — the affected player's ship silently went to zero thrust for the rest of the match. The reviewer reproduced this with a real SIGSTOP/SIGCONT host freeze (a faithful stand-in for a GC/IO/scheduler hitch on a listen-server host): client movement dropped from ~26m to a flat 0.00m at ~0.7s of freeze, reproducible 4/4 times, and found the cliff got worse under real network conditions (a lossy link that had already pushed input_lead up lowered the fatal threshold to ~400ms) and could be reached with no external trigger at all via ordinary client/server clock drift (~1.7% faster client death-spiraled within ~60s). Fixed with a real resync mechanism: ingest() now tracks the highest seq ever seen regardless of ring capacity, and consume() detects when the gap to that value exceeds RING_SIZE and jumps directly to what the ring can still actually provide, instead of starving through an unrecoverable span. Re-verified with the reviewer's own reproduction: a 3-second SIGSTOP freeze mid-drive now fully recovers (27m+ movement), both via the human smoke test and a real 2-bot CI match. New unit test test_ring_overflow_resyncs_to_fresh_data_instead_of_starving_forever covers the exact under-tested direction the reviewer flagged (the original suite only exercised the under-full ring case).

HIGH — InputLeadController's release logic couldn't drain a backlog it didn't itself create. Release was gated on lead > LEAD_MIN — this controller's own memory of past attacks — so a backlog from an external cause (a server hitch, persistent clock drift) left input_buffer_depth elevated indefinitely while lead (and the release gate) never moved, since the controller never itself attacked. Fixed by gating release on the actual server-reported input_buffer_depth > TARGET_DEPTH (§3.3's own target_depth = 1), not on self-tracked state. New unit test test_release_drains_a_backlog_it_never_caused_itself reproduces the scenario directly.

MEDIUM-HIGH — the rate limiter was trivially evaded by a duty-cycled flood. The original design tracked "N consecutive over-budget seconds" and hard-reset that streak to 0 on any single clean window, so a burst-then-idle attacker (flood hard, one clean window, repeat) evaded it indefinitely — the reviewer sustained ~33x the packet budget for 28.5s with zero disconnect warnings against the real MatchSim._recv_input. Replaced with a leaky-bucket excess accumulator (grows by each window's actual total, drains by exactly one window's worth of budget every window, regardless of how the excess is distributed in time) — immune to the same evasion by construction. New permanent regression test client-abuse-flood-dutycycle reproduces the reviewer's exact attack shape (0.35s burst / 3.0s cycle) and confirms it now disconnects.

MEDIUM — the seq > server_tick + 20 guard compared two unrelated epochs. Engine.get_physics_frames() counts from the SERVER PROCESS's own start; a client's _input_seq starts at 0 when ITS match scene loads — input_jitter_buffer.gd's own seeding logic exists specifically because these share no baseline. Bounding against server uptime meant the guard could never fire on a long-running dedicated server (no real protection, despite the comment's claim) and could silently drop an honest client's input forever once enough accumulated server tick loss closed whatever accidental head-start margin existed. Fixed by bounding against the slot's own last_applied_seq + RING_SIZE — the client's actual epoch, using the same capacity the ring-overflow fix itself treats as "unrecoverably far ahead."

MEDIUM — InputJitterBuffer.stalled was computed but never reached the wire. _ship_to_net_body_state never set NetBodyState.stalled even though NetCodec already packed/unpacked the bit — the one signal that would have made the ring-overflow bug visible to the client, the debug overlay, and the CI gate was silently dropped between the buffer and the snapshot builder. Now wired through.

MEDIUM — task 3.6's own CI gate passed with a completely dead input pipeline. Its assertions (snapshot count, a server-forced goal's score agreement) don't depend on client input reaching the server at all; the reviewer confirmed it kept reporting SMOKE PASS with the ring-overflow bug actively triggered mid-run. Fixed by recording each bot's ship position before the run and asserting real server-side movement plus a non-stalled jitter buffer — sampled while clients are still actively connected, not after (an early attempt sampled too late and caught each bot's own legitimate end-of-match disconnect instead of the bug, since a departed peer's buffer starves too — that's correct behaviour, not a regression, just the wrong moment to check it). Re-verified: the fixed CI gate still passes cleanly under a real mid-match host freeze now that the underlying bug is fixed, and (checked by inspection during the fix) would have caught the original bug had it still been present.

LOW-MEDIUM — a lead change silently mislabelled the redundancy history. _input_history was always push_front'd regardless of the seq delta, but the wire format has no per-entry seq field (actions[i] is implicitly seq - i) — a duplicated tick (release) shifted older entries under a label that no longer matched what was actually there, and a skip-ahead (attack) left the whole history discontiguous with the new seq, so the server could replay already-applied input or apply the wrong redundant copy. The original code comment's claim that this "only ever degrades a backup copy, never the real per-tick record" was itself wrong. Fixed by handling each delta case on its own terms: ordinary ticks still push; a release replaces the front entry in place instead of shifting everything back; an attack resets the window to just the current sample, which rebuilds naturally over the next few ticks (the same way it does at connection start).

LOW — bandwidth and snapshot-loss overlay metrics froze at their last value instead of decaying during a total outage — exactly when they matter most. MatchSim.bytes_sent_per_sec/bytes_received_per_sec are now read through get_bytes_sent_per_sec()/get_bytes_received_per_sec(), which report 0 once meaningfully more than one window has passed with nothing tracked; get_net_debug_stats()'s snapshot_loss_pct now reports 100% once more than SNAPSHOT_STALE_MS has passed since the last actual snapshot receipt. Verified live: all three read their honest post-outage values (0, 0, 100%) after a real ~2.5s gap in traffic, not the frozen pre-outage numbers.

LOW — a guard comment on NetworkManager._ping misdescribed what the code actually does, claiming disconnect_peer(..., now=true) when the real call uses the default force=false (an earlier attempt at force=true, tried and reverted elsewhere this session, made Godot's own peer bookkeeping more inconsistent, not less). Comment corrected to match reality.

Confirmed fine, not just assumed: the jitter metric's magnitude is honest (cross-checked against real 60Hz snapshot-stream jitter on the same impaired link, same order of magnitude), just slow to converge from a 1Hz sampling cadence — worth documenting as "steady-state link quality" rather than a live indicator, not worth rebuilding; --test-bot/AIShipController wiring on a frozen kinematic ship is fully safe, no NaN/Inf even under the bot's permanently-zero-velocity observations; both original abuse-detection regression tests are genuine, confirmed via a working control (a continuous flood still disconnects in ~4.5s); redundancy + adaptive lead + real loss alone (no hitch) is solid over long runs | Full regression suite — including the net-sim-latency milestone gate, all three abuse roles, the CI driver, and the reviewer's own SIGSTOP/SIGCONT reproduction at 3s (well past the original ~0.7s failure threshold) — re-run clean after every fix |

Phase 4 — Prediction and reconciliation, ship and ball

# Task Acceptance
4.1 [D:3.1] DONE. Immediate local input with immutable sequence/redundancy bookkeeping; server/training action semantics unchanged 60 unit tests and 60s LAN/jitter/loss runs pass
4.2 [D:4.1] DONE. 128-entry sequence-tagged prediction history and snapshot matching Same-sequence free-flight samples resolve in all 60s runs
4.3 [D:4.2, 0.14] DONE. Atomic staged reconciliation, delta rebase, epoch/reset and missing-history recovery No free-flight hard snaps across LAN, 80±20ms, or 5% loss 60s runs
4.4 [D:4.3, 0.2] DONE. Client-only bounded position and rotation visual offsets/decay; interpolation reset Free-flight p99 raw residual ≤0.207m; exposed visual p99 0m in final matrix
4.5 [D:4.3] [P] REJECTED / SUPERSEDED. Analytic one-body action replay was removed in favour of same-sequence delta transport Jolt/contact nondeterminism makes replay unsuitable; see §4.4
4.6 [D:4.3] DONE. Client-only dynamic proxy, authoritative shadow, RTT-limited reveal, 150 ms blend and 3 m handoff Final contact run: same-frame reveal, 4 blends, max 152ms, no hard handoff
4.7 [D:4.4] [P] DONE. Client-only debug keys tune thresholds, decay, visual offset and remote-present A/B Defaults remain 2m/60°/0.4m and render-only tuning never reaches server/training
4.8 [D:4.4] [P] DONE. p50/p95/p99 residual telemetry, elapsed-time snap rate, reason/cohort counters Final free-flight p99: LAN .150m; 80±20ms .149m; 5% loss .207m; zero hard snaps
4.9 [D:4.4] DONE. Present-time remote visual extrapolation, angular integration, and render-only residual correction; delayed interpolation remains an A/B debug mode Final two-bot present-time p99 ≤.208m / 3.146°, below .3m / 5° gate
4.10 [D:4.9] [P] DONE. Signed starvation sentinel and client hysteresis/cooldown; headless --test-bot remains target depth 1 Jitter run observed starvation fallback; stable runs preserve safe target behavior

Ball prediction is not optional and not deferrable to a later phase. With §4.1 in place the touch registers correctly on the server, but the ball still renders a third of a beat late — your ship visibly passes through it before it moves. In a game whose entire point is hitting a ball, that is the difference between "networked" and "broken", and it is the same machinery as own-ship prediction applied to one more body. Do it while the prediction code is warm. Buffering server ball state into a shadow copy (rather than discarding it) is what lets you measure disagreement continuously instead of discovering a 3 m error at window end.

| 4.11 [D:4.2] | DONE. Prediction history is filed under the issuing sequence, and a forced-input-transition trace gates the label | Marker mismatch 0.001.3% (was 9.3% LAN / 24% at 80±20ms); control run at the old label fails the same gate at 50% | | 4.12 [D:4.11] | DONE. Issued-but-unsimulated (attack-gap) sequences are recorded and skipped rather than diagnosed as history loss; the release path no longer re-files an already-issued sequence | Free-flight hard snaps 0 across all three 60 s conditions, down from 25/8/4 missing_not_recorded | | 4.13 [D:4.12] | DONE — two server-side input-death bugs found by adversarial review, both reproduced and fixed with controls. A starve no longer advances past a sequence the client has not sent; the seq-range guard can no longer latch shut permanently | Marker 0.00% in all three conditions (was 1.72.5%); 2.0 s and 3.5 s host freezes now recover; control runs with each fix reverted fail the gate | | 4.14 [D:4.3,4.8] | DONE. Prediction startup distinguishes the server's pre-history sequence-0 acknowledgement from genuine missing/evicted history, so warm-up cannot arm hard-snap recovery | 143 Godot tests pass; two-process ENet match passes 173 prediction samples with 0 hard snaps, 0% snapshot loss and authoritative movement; the 80±20 ms impaired-link run passes the near-surface gate with p95 0.682 m / p99 0.717 m and no free-flight hard snap; the 5% loss run passes with 222 samples, 7.1% observed snapshot loss, p99 0.716 m and 0 hard snaps |

Phase gate — correctness gates MET; the milestone's felt-quality half remains untested. The action-sequence-correctness gap is closed and permanently gated (4.11), the two seq-delta paths it exposed are fixed (4.12), and an adversarial review's two server-side input-death bugs are fixed with controls (4.13). What has not happened is the original milestone's actual subject: nobody has played this with hands on a controller at ~100 ms RTT to judge whether ship and ball feel local and whether contact corrections read as bumps. Numbers cannot answer that, and the contact cohort is where the remaining known weakness lives (see the shadow-world note below). Sign off after a human playtest, not before — item A of §0.

Read 4.13 before trusting any earlier Phase 4 evidence. Until this session the server was silently discarding a connected player's input for ~30 ticks roughly every 6.5 seconds on a clean LAN, and permanently after any ~2 s host hitch. Every Phase 4 number recorded before 4.13 was measured through that, and the gates reported green throughout — for the same reason they missed the label bug in 4.11: a steady input cannot distinguish "the server repeated my last action" from "the server applied my real action".

The mislabelled prediction history, and why every earlier gate missed it. _send_local_input filed each post-step predicted state under _local_net_controller.last_applied_seq — the timeline's estimate of the sequence the server would consume this tick, which trails issuance by input_lead. The body had actually integrated the current raw intent, issued under _input_seq. So predicted[S] held "state after integrating the intent from now" while the server's authority for S is "state after integrating action(S)", sampled input_lead ticks earlier. The two agree only while the commanded action is constant — and every Phase 4 acceptance trace held its input steady (move_forward held, or the free-flight hover alternating on a 0.7 s/0.3 s period). A steady input cannot falsify a sequence label: the marker reads 0/N under a correct and an incorrect label alike. The 60-second free-flight runs genuinely reported marker=0/3784; the instrument was fine, the trace was blind.

Filing the state under _input_seq fixes it and costs nothing. The code comment that had rejected this ("avoids turning client prediction into an input-delay queue") conflated which action the ship uses — decided in LocalNetShipController.get_action(), still the raw current intent, still immediate, untouched by this change — with which sequence its resulting state is filed under. Measured with --exercise-input-transitions (below):

condition input_lead old label filed under _input_seq
LAN 1 35/376 (9.3%) 06/456582 (01.3%)
LAN, adversarial toggle phase 1 289/576 (50.2%)
80±20 ms 3 97/404 (24%) 0/424 (0%)

Mismatch scales with input_lead, exactly as the mechanism predicts. It also cut pre-existing missing_not_recorded hard snaps 4× on the 60 s LAN free-flight run (25 → 6, 24.8/min → 6.0/min): the old label's per-tick +1 cursor was an estimate that could drift off the sequence the server actually acknowledged, while an issued sequence is by construction the thing the server acknowledges.

Task 4.12 — the two seq-delta paths, and what is left. Relabelling exposed two further places where the history disagreed with the wire, both now fixed:

  • Attack gaps (delta > 1). The lead controller skips sequence numbers to buy server buffer margin. Those sequences are filled with repeat-last actions and genuinely sent, and the server genuinely acknowledges them — but the client took exactly one physics step that tick, so no post-step state exists for them. They were simply absent from the ring, which compare_authoritative could only report as missing_not_recorded: indistinguishable from real ring loss, and therefore a hard snap, a full authority teleport, and armed resync suppression several times a minute during ordinary play. They are now recorded stateless via record_unsimulated() and report their own unsimulated_gap status, which NetShipPredictor.decide() answers with a new "skip" mode — no correction, no teleport, no suppression, no snap counted, its own metrics cohort. The next simulated sequence (a tick or two later) reconciles normally. Result: free-flight hard snaps went from 25 / 8 / 4 to 0 / 0 / 0 across the LAN, 80±20 ms and 5%-loss 60-second runs; the doc's own long-standing target was <1/min and LAN was measuring 24.8/min.
  • Release (delta == 0). _send_local_input re-recorded at the unchanged _input_seq, filing the current intent under a sequence that had already gone out carrying a different action. LocalInputTimeline.issue() deliberately refuses to mutate an already-issued sequence ("may be in flight or consumed"), so the ring was contradicting the wire outright. Recording is now skipped entirely on a release tick; the existing predicted[S] is already correct, and the extra unlabelled local step is precisely the tick of latency the release exists to recover.

The residual is solved — it was not a prediction bug at all. An adversarial review intersected every sequence the server starved on against every sequence the marker flagged, across five two-process runs: 151 of 151 mismatches were the server repeating a stale action on a starve, zero unexplained. When the server starves on seq S it repeats action(S-k) but still acks S, so the snapshot's thrust_z honestly describes a different action than predicted[S] — the marker was correctly reporting a real client/server disagreement that prediction did not cause and could not fix. The apparent correlation with input_lead was a confound: the conditions that raise the lead are the conditions that produce starves. Fixing the starvation cause (task 4.13 below) took the marker to 0.00% in all three conditions, including 80±20 ms and 5% loss where it had been 1.72.5%.

Two sub-findings from that investigation, recorded because both are counter-intuitive: dequantize_thrust_z_bin(quantize_thrust_z_bin(0.0)) returns 0.142857, not 0.0 (7 bins over [-1,1], roundi(3.5) == 4), so a server-reported thrust_z of 0.14 literally means "exactly zero" — the 0.26 threshold absorbs it, as designed. And _pending_local_reconciliation keeps only the newest snapshot, so acks are dropped whenever two snapshots land in one physics tick: the marker under-samples, and the true action-disagreement rate is higher than it reports.

The client-only shadow Jolt world is still the open question (item F of §0), but it is now scoped to the contact cohort alone. Even perfectly labelled, the client predicts contacts against remote ships and the ball sitting at interpolated-delayed positions, so a contact-cohort prediction cannot be sequence-correct in the live world — no amount of bookkeeping fixes that, and a shadow world is the only thing that does. It is a large subsystem and effectively the whole-world rollback §1's locked decisions set out to avoid, so do not build it before a playtest says the contact cohort actually reads badly to a human. Free flight no longer needs it.

New smoke role — --exercise-input-transitions. Toggles forward thrust every 6 physics ticks (~100 ms) with alternating yaw, and asserts the action marker stays under 5% mismatch over ≥200 samples. This is the only gate here that can catch a sequence-label regression, for the reason above, so it must not be folded into the steady-input free-flight run:

godot --headless --path Game res://tests/networked_match_smoke.tscn -- --role=host --drive-seconds=8
godot --headless --path Game res://tests/networked_match_smoke.tscn -- --role=client --drive-seconds=8 --exercise-input-transitions

Verified against a working control: reverting the one-line label makes this gate fail at 50.2% mismatch, so it is not vacuous.

Task 4.13 — two server-side input-death bugs the Phase 4 gates could not see

Both are Phase 3 code, both predate this session, and both were found by an adversarial review of the Phase 4 changes rather than by any gate. Neither was caused by 4.11/4.12; both are squarely in the way of Phase 4's feel milestone, so they are fixed here.

(a) A starve stranded the input stream one sequence ahead of arrivals — permanently. InputJitterBuffer.consume() set last_applied_seq = expected on every tick, including a starve. Because ingest() discards anything seq <= last_applied_seq, a single starve on a sequence the client had not sent yet left the server permanently one ahead: both sides then advance one per tick, the gap never closes, and every honest packet is discarded on arrival. The client's own input_lead RELEASE (delta == 0, which deliberately issues no new sequence for one tick) is sufficient to trigger it — so this fired roughly every 6.5 seconds of ordinary play on a clean LAN, blacking out input for 30 ticks until the lead controller's MIN_CHANGE_INTERVAL_TICKS debounce permitted a +3 attack to jump the client clear. The reviewer measured 2 blackouts in a 23 s run and 2 in a 30 s run, with the host applying the same repeated action for 30 consecutive ticks while the wire carried fresh input every one of them. Fixed by only giving up on expected when strictly newer data has arrived, which proves it lost rather than merely late. Both escape paths are untouched: a silent client still zeroes and stalls on STARVE_ZERO_TICKS, and a far-behind consumer still hits the ring-overflow resync.

(b) The seq-range guard was a one-way door. _on_input_received bounded incoming seq against jb.highest_ingested_seq + RING_SIZE — but highest_ingested_seq only ever advances inside ingest(), which that same guard gates. Once a client's live sequence got more than 32 ahead (a host stall drops the intervening packets wholesale, since input is unreliable), every subsequent packet was rejected, the bound could never move again, and that player's input was dead for the rest of the match with no diagnostic. Reproduced with a 2 s SIGSTOP host freeze: 600+ consecutive rejections, the server applying zero thrust across 1300 sequences while the client's wire carried full thrust throughout. This is the third iteration of this guard, and the structural lesson is that each previous version bounded against a value only the accepted path could advance. Fixed by keeping the bound but adding an escape: after SEQ_REJECT_RESYNC_LIMIT (10) consecutive rejections, accept and let the existing resync machinery re-establish the baseline. This grants an attacker nothing — walking the epoch forward by sustained rejection costs the same packets as walking it forward by acceptance, and §3.4's rate limiter already bounds that rate.

(c) The gate printed PASS while input was permanently dead. The --exercise-input-transitions gate reported SMOKE PASS at 3.76% mismatch on a run where input was completely dead, because suppressed reconciliation stops calling _record_metrics — so the worse the outage, the fewer marker samples and the lower the reported mismatch rate. Every other assertion in that path (local_prediction_ok, moved > 1.0) reads the client's own action and position, which a client flying purely on prediction satisfies perfectly. Fixed by scaling the required sample count with run length (max(200, drive_seconds * 30), half of nominal 60 Hz) and asserting the wire's server_stalled bit. Verified non-vacuous: reverting both fixes and re-running the 3.5 s freeze fails at samples 292/600 with server_stalled=true and input_lead=12 (LEAD_MAX) — while reporting marker=1/292 = 0.34%, which the old gate would have passed.

QA matrix, re-run in full after 4.11 + 4.12 + 4.13 (all green): 72 unit tests; 60 s free-flight at LAN / 80±20 ms / 5% loss — p99 raw 0.141 / 0.168 / 0.154 m, exposed visual p99 0.000 m, 0 hard snaps in every condition, marker 0/3484, 0/3049, 0/3397; forced-input-transition gate at LAN, 80±20 ms and 5% loss, all 0.00%; 2.0 s and 3.5 s SIGSTOP host-freeze recovery; ball contact ×5; two-bot CI ×3; all three abuse roles; net_smoke, match_net_smoke (incl. host_recycle), clock_smoke, lobby_smoke.

Note the free-flight p99 improved (0.170/0.176/0.184 → 0.141/0.168/0.154) and input_lead now sits at 1 on LAN instead of oscillating to 34. Both are downstream of 4.13(a): the periodic blackouts were degrading prediction accuracy and driving the lead controller.

Two test defects fixed alongside, both pre-existing and both surfaced by 4.13(a):

  • Ball-contact gate flaked 2 in 5. ball_proxy_moved_before_authority_count requires the predicted proxy to have visibly moved before the next authoritative ball state arrives — but snapshots land every ~16.7 ms at 60 Hz, so on a loopback LAN the entire pre-authority window is about one physics tick and observing it is a coin flip. It is also the least interesting case: the counter measures RTT-masking, and LAN has no RTT to mask. Measured 5/5 passes (count 23) at --net-sim-latency=80. Now asserted only when NetworkManager.rtt_ms >= 20, with the same-frame reveal — the test's real claim, correct in every run either way — carrying the gate on LAN. Runs asserting the masking behaviour should pass --net-sim-latency.
  • Two-bot CI compared scores across a 35 s window. The host checked each client's recorded score against its own score at read time, but clients write theirs several seconds earlier; any goal in between failed the run with both bots agreeing perfectly with each other. Latent until 4.13(a) made the bots effective enough to reliably score a second goal — then it failed 2 of 3 runs, every failure server=2 vs both clients=1. The host now polls and records every score it actually holds, and asserts both clients agree with each other and that what they saw is a state the server genuinely passed through. 3/3 green, including a run ending 11 where the clients had recorded 01. (Polling, not score_changed: that signal is emitted only in _on_score_update_received, the client path — the server mutates score directly in _record_goal and never emits. Connecting to it recorded nothing but the initial 00.)

Follow-up, not done: LocalNetShipController.last_applied_seq is now write-only and LocalInputTimeline.consume() is vestigial to the reconciler (still unit-tested, still advancing _last_applied_action, but nothing reads the result). Left in place rather than removed as unreviewed scope — but it now looks load-bearing and is not.

Phase 5 — Match lifecycle

# Task Acceptance
5.1 [D:2.1] DONE. scripts/match_state.gd (enum + validated transition table, pure/unit-testable), server-driven machine in NetworkedMatch, state_change RPC on reliable channel 0 carrying an absolute at_tick, and the snapshot match_state byte populated for real Client observed LOADING -> WARMUP -> PLAYING -> GOAL_PAUSE -> WARMUP with monotonic ticks in a real two-process run; every consecutive pair legal; wire byte asserted independently of the RPC
5.2 [D:5.1] DONE. _end_tick/_clock_running, clock_state RPC, timer_updated emitted from absolute ticks on both peers; goal pause shifts end_tick rather than pausing anything No Timer and no _process polling remain in the networked path; both peers derive remaining = end_tick - now from the same server-tick estimate
5.3 [D:5.1] DONE. kickoff RPC carrying resulting transforms (never a seed, per §1), deferred freeze, reset_gen bump, countdown from server_tick, late-arrival skip Real two-process run: LOADING -> WARMUP -> PLAYING, countdown ticks match WARMUP_TICKS exactly; a kickoff past its own resume tick unfreezes immediately and emits 0
5.4 [D:5.1] DONE. goal_scored(scoring_team, score, goal_tick, resume_tick), freeze on the goal tick, reset moved out of the sensor path into the kickoff at resume_tick; cinematic is presentation-only PLAYING -> GOAL_PAUSE -> WARMUP -> PLAYING observed on the client; bodies stay where the goal left them for the whole window; Engine.time_scale untouched
5.5 [D:5.1] [P] DONE. Clock expiry -> FULL_TIME -> sudden death on a draw or RESULTS, golden goal in overtime, then LOBBY on both peers. get_tree().paused is never used in the networked path Full run observed end to end: LOADING -> WARMUP -> PLAYING -> FULL_TIME -> OVERTIME_WARMUP -> OVERTIME -> GOAL_PAUSE -> RESULTS -> LOBBY, both peers returning to the lobby scene
5.6 [D:5.1] [P] DONE. Controller swap on disconnect (ship never despawned), 30 s identity-keyed reservation, reclaim on reconnect, --fill-bots/--no-fill-bots, stalled set immediately for the nameplate Real 3-process run: ship survived, controller valid, slot reserved, reclaimed by name, same ship instance
5.7 [D:5.6] DONE. _swap_slot_controller() rebinds in the same transaction; slot.controller retyped to the base ShipController; every use is_instance_valid-guarded The disconnect test caught the real bug: the narrower RLShipController type made the swap assignment fail, leaving a freed reference
5.8 [D:5.1] [P] DONE. A slotless peer spectates (no ship spawned, same snapshot stream), HUDController.spectator_mode keeps clock/score/celebration and hides only the ship cluster, camera cycles ships then ball, --max-spectators cap. §6.3's "take the slot at the next kickoff" is now implemented, not just printed — the line claiming it was there from the start while _is_spectator was assigned once and never revisited (see the Phase 5 note below) Spectator path exercised by the mid-match joiner; HUD no longer push_errors and bails with a dead HUD; promotion verified 4/4 from both sides, with a control proving §6.4's reservation outranks the queue
5.9 [D:5.3] [P] DONE. New GameMode._on_bodies_respawned() virtual; NetworkedMatch bumps reset_gen through Phase 2's deferred path so the bump and the respawned pose land in the same broadcast Single-player modes unaffected (base is a no-op)
5.10 [D:5.1] [P] DONE. scripts/replay_log.gd, --replay-log=<path>, storing wire bytes verbatim in both directions — plus, after a review found three recording gaps, REJECTED packets with their reason in the kind byte (capped per window so the log cannot become a remote disk-fill amplifier), a failed write that ends the log instead of desyncing its framing, an explicit close() with a summary, and tools/replay_dump.gd to read one back. The reject recording immediately found a real bug: the server was rate-limiting a stall backlog it had caused itself, losing 8.88% of a player's input Live 6 s match recorded 1115 records (557 inputs / 558 snapshots); a stored snapshot decodes back to server_tick=100 match_state=WARMUP bodies=2; 6 unit tests incl. truncation and foreign-file rejection

Ship.set_controller (ship.gd:213-218) calls queue_free() on the outgoing controller. Task 5.7 exists because the takeover path in 5.6 otherwise leaves MatchNet holding a freed reference — the exact class of bug that surfaces as a random server crash weeks later.

Task 5.10 is the highest-value debuggability investment here. The packets are already flat bytes, so it is ~50 lines. Without it, "my ship snapped" is permanently unreproducible from a field report — the CI gate catches regressions, but it cannot debug a player's bad night.

Task 5.1 notes

scripts/match_state.gd holds the enum and the §6.1 transition table as pure data with no scene/RPC dependency — the same reason net_codec.gd and input_jitter_buffer.gd are standalone — so the table is checked exhaustively (every state reachable, every state has an exit, no self-transitions, abort-to-LOBBY from anywhere, illegal shortcuts rejected) rather than by example. The enum's integer values are the wire format, pinned by a test: match_state has been a u8 in the snapshot header since §2.4, so renumbering an existing state silently reinterprets packets from an older peer. Only append.

The server validates every transition and push_errors an illegal one rather than following it, because the symptom otherwise — clients faithfully following into a state the server's own code never meant to reach — is near-impossible to diagnose from a field report.

Two channels carry the state, deliberately. state_change (reliable, channel 0) is prompt and carries the absolute at_tick; the snapshot's match_state byte is the catch-up path for a client that has not been sent a transition yet — a late joiner (§6.3), or the window between scene load and the first RPC. The byte needs a tick guard: snapshots are unreliable_ordered on channel 2 and ordering holds only within a channel, so a state_change for tick N routinely arrives before an in-flight snapshot from tick N-2. Without the guard the client applies the new state and is immediately dragged back by the older byte, oscillating on every transition — observed directly (LOADING -> WARMUP -> LOBBY -> PLAYING -> LOBBY -> ...) while running a deliberately-broken-byte control. Only a byte at least as new as match_state_since_tick is accepted.

The client deliberately does not enforce the transition table — authoritative state must be accepted, and a late joiner legitimately jumps straight to PLAYING. The table is a server-side invariant. The smoke test asserts legality of what the client observes, seeding its first sample from whatever state the client converged to rather than counting that as a transition, so late-loading clients (seen seeding at WARMUP rather than LOADING) still pass.

5.1 does not gate physics, freezing or input on state. Tasks 5.3 and 5.4 own freeze/unfreeze at kickoff and goal; doing it here would both duplicate that work and change the conditions every Phase 4 prediction gate was measured under. MatchState.is_live() exists for them to use. WARMUP_TICKS/GOAL_PAUSE_TICKS are honest placeholders so 5.1 drives real transitions to verify against — 5.3 replaces the first with the broadcast kickoff (reset transforms + countdown from server_tick), 5.4 the second with _goal_pause_seconds() and the client-cinematic split. The server also leaves LOADING immediately rather than waiting for scene_ready, which does not exist yet (5.3).

New smoke flag --exercise-match-state (pass to both roles — the host forces a goal to drive a GOAL_PAUSE cycle, the client records and validates the sequence):

godot --headless --path Game res://tests/networked_match_smoke.tscn -- --role=host --drive-seconds=6 --exercise-match-state
godot --headless --path Game res://tests/networked_match_smoke.tscn -- --role=client --drive-seconds=6 --exercise-match-state

Verified against a control: hardcoding the snapshot byte back to 0 fails both the byte assertion and the transition-legality assertion. That control is why the gate asserts the wire byte separately from the RPC at all — everything else in the check is RPC-driven and would pass identically with a dead byte, which is exactly how Phase 4's mislabelled history survived every gate (gotcha 47).

Phase 5 notes

Task ordering caught three ordering bugs of the same shape, all found by a failing run rather than by review, and all worth remembering as a class: a value consumed by one per-tick updater and cleared by another is order-dependent. _update_kickoff_countdown() clears the _kickoff_resume_tick that _update_match_state() reads to leave WARMUP (match froze forever); _apply_match_state() resets _state_deadline_tick on every transition, so a GOAL_PAUSE deadline assigned before _set_match_state was wiped (match never resumed); and a set_deferred("freeze", true) landed before the queued kickoff teleport could apply, stranding every body where the goal left it.

Freezing is asymmetric between server and client, and this is not optional. On the server every body is a real dynamic simulation and all of them freeze. On a client, freeze is already load-bearing for something else: remote ships and the ball are permanently FREEZE_MODE_KINEMATIC and driven by transform writes, with only the local ship unfrozen for prediction. Freezing "all bodies" on a client therefore unfreezes the remote ones on the way back out — they fall under gravity while the interpolator fights them for the transform. Measured: 210 hard snaps and an infinite p99. A client freezes only the one body it actually simulates.

Prediction is suspended while the match is not live. During a countdown or goal pause the local ship is frozen on both peers, so there is nothing to predict — but the reconciler still ran delta transport and visual-offset maths over those frozen states and produced a p95 position error of 2.4e10 m while the instantaneous error stayed small. Input keeps flowing so the server's jitter buffer does not starve into stalled.

§6.4's two rules conflict and the reservation has to win. "Reserve a departed player's slot for 30 s" and "abort to the lobby once the last human leaves" applied naively means the abort fires instantly in a 1v1 — the moment the only player drops, the match is torn down and the reservation can never be redeemed, making the reconnect path unreachable exactly when it matters (one player whose connection blipped). Abort now waits until nobody is connected and no reservation is outstanding.

Task 5.7's bug was real and the test found it. SlotInfo.controller was declared RLShipController, but §6.4's takeover swaps in an AIShipController or the base controller — a narrower declared type makes that assignment fail its type check, leaving the field pointing at the controller set_controller() just queue_free()d. It surfaced as controller_valid=false on the first disconnect run. The per-tick slot.controller.action write is now also gated on is RLShipController: a disconnected slot's bot drives itself, and overwriting its action from a permanently-starving buffer would pin it to the departed player's last input.

--check-only --script is the only thing that catches a parse error in networked_match.gd. The unit runner never loads it, so bot_model_path being undefined (and later ReplayLog being unregistered) both passed 81/87 unit tests while breaking every two-process run. Validate touched scripts directly. A newly added class_name also needs godot --headless --path Game --import before anything can resolve it — and the same --import is the fix when a previously working class_name stops resolving, which happens on its own: .godot/global_script_class_cache.cfg silently lost MatchState between sessions, and every two-process run then died with Cannot infer the type of "live" variable at the MatchState.is_live() call, with nothing in git status to explain it. Read that error as "the class cache is stale", not "the code is wrong".

The reviewer's p95 0.688 was real, and the three-process framing was a red herring — mine as much as the reviewer's. The report was "a 3-process run failed the free-flight gate at p95 0.688 (bar 0.5) with roughly a third of snapshots missing", so the first investigation compared process counts: two-process p95/p99 0.084/0.098, idle third process 0.084/0.094, spectator 0.0840.098 / 0.0940.146 over four runs, and 0.0% snapshot loss even under deliberate 2x CPU oversubscription (20 spinners on 10 cores, where only snapshot_age moved, 14ms → 32.3ms). Every one of those runs passed, so the conclusion recorded here was "not reproducible". That conclusion was wrong, and it was wrong because every probe used --exercise-free-flight — the one mode the 0.5 bound was calibrated on.

It reproduces on two processes, on an idle machine, with 0.0% snapshot loss: the plain --role=client drive fails the free-flight gate roughly a third of the time. Eight plain-role runs measured a free-flight cohort of 12257 samples with p95 0.2750.726, failing the 0.5 bound in 3 of 8. The harness's own _run_free_flight_trace comment had already said why — "a straight forward trace reaches the goal/wall in seconds and turns the supposed free-flight QA run into a contact test" — but the plain role went on asserting the open-volume bound against whatever free-flight samples that contact-heavy drive happened to leave behind, sometimes as few as 12.

The underlying difference is not noise. Prediction error near the arena's surface-pull field is genuinely several times higher than in open air: the same build measures 0.0840.111 under --exercise-free-flight and 0.2750.726 on the plain drive. Both are honest numbers about different flight profiles, and one bound cannot serve both. --exercise-free-flight keeps the calibrated 0.5/2.0 gate (~5x margin). The plain role now asserts the all-cohort percentiles instead — always well-sampled (545696, versus a free-flight cohort that can collapse to 12) and much tighter in spread (raw_p95 0.3540.609, raw_p99 0.3620.742) — at 1.2/2.0, ~2x above the worst observed, and prints the free-flight numbers explicitly marked reported, not asserted. free_flight_hard_snaps == 0 is still asserted in both modes, and anything past 2.0m is a hard snap by definition, so a genuine free-flight regression cannot hide behind the looser bound. Verified: 6/6 plain-role runs pass where 3/7 previously failed, all four other modes (free-flight, 80±20ms latency, input transitions, ball contact, match state) still pass, and tightening the new bound to 0.3 makes it fail — the gate is evaluated, not skipped.

The other durable improvement from the first investigation still stands: a percentile alone cannot distinguish "the predictor regressed" from "the client never received the data", so the client gate prints snapshot_loss / snapshot_age / rtt on every run and, on a quality failure with >20% loss, says explicitly that the run was transport-starved — without converting the failure into a pass. Both directions verified non-vacuously. It is also what proved the 0.688 was not transport: every reproduction reported 0.0% loss.

Lesson worth more than the fix: probing only with the purpose-built mode is how a flaky gate stays invisible. The first pass ran eight variations of process count and CPU load and never once ran the plain role that the reviewer had actually run.

Task 5.10's three recording gaps, and the real bug closing them found. The review flagged that the replay log ignored store_* failures, never recorded the packets the server rejected, and had no caller for close(). All three are fixed: a failed write now ends the log permanently rather than desyncing every later record's framing (write_failed, checked via FileAccess.get_error() once per record); close() is called from _exit_tree with a summary line, because letting the RefCounted's destructor do it implicitly never tells anyone whether the log is complete; and rejected packets are recorded with their reason in the kind byte (REJECTED_MALFORMED / REJECTED_RATE_LIMIT / REJECTED_SEQ_GUARD, framing unchanged, FORMAT_VERSION 2 so "no rejects" can be told from "this build never recorded them"). Recording is capped at 8 per peer per rate-limit window — without that cap the diagnostic is a remote disk-fill amplifier, since the attacker chooses the packet rate. Verified end to end: an honest client logs 0 rejects; client-abuse-malformed sends 25 and logs exactly 8; client-abuse-flood sustains ~2400 packets/s and logs exactly 8. Uncapped totals are kept separately (MatchSim.get_reject_totals()) and survive the peer's disconnect — the first version stored them on _PeerInputState, which is erased on disconnect, so every summary printed an empty dictionary.

And the bug the recording immediately found: the server rate-limited a backlog it caused itself. A 2s host stall (SIGSTOP, standing in for a GC/IO/scheduler hitch) has the client sending at 60Hz throughout, and ENet delivers the whole backlog in the first window after resume — 70 of an honest client's input packets rejected as "rate limit exceeded", against a limit that client never came close to violating. Redundancy does not cover it, and that was the assumption worth checking rather than asserting: the dropped packets are contiguous, so each one's redundancy window falls inside the same dropped run. Measured with the new log: 0 of 70 rescued, and 82 of 923 sequences (8.88%, ~1.4s of that player's input) never reached the server at all, versus 0.00% on an otherwise identical run with no stall. Every prediction gate still passed — this is the same class as the Phase 3/4 input-death bugs, invisible to every gate that reads only the client's own state.

Fixed by not policing a backlog the server caused: MatchSim._physics_process watches for a wall-clock gap over STALL_DETECT_MS (a stalled process doesn't run that callback either, so the first frame after the stall sees the whole gap, which is exactly the size of the backlog about to arrive) and grants each already-tracked peer a capped, two-window packet grace. The leaky bucket drains against the same graced budget, or a stall would still accumulate excess toward a disconnect for traffic the server just explicitly allowed. Results: 2s stall, rate-limit rejects 70 → 0, sequences missing 8.88% → 0.00%, and REJECTED_SEQ_GUARD 9 → 0 as a second-order confirmation (the guard was firing partly because the dropped backlog let the client's epoch run away). Across eight stall runs on the fixed build, 7 measured 0.00% missing; the eighth measured 23.54% with zero rate-limit rejects and the seq-guard resync visibly doing its job — a separate, occasional transport-level loss during the stall that this change does not address and does not make worse (item D of §0). The three control runs on the unfixed build lost 4.34%, 7.52% and 7.86%, every time.

Abuse detection is unweakened and this was checked rather than argued: all three abuse roles still disconnect, and no flood induced a server stall in any run, so the grace cannot be farmed by flooding. An attacker who can induce server stalls to earn budget already has a strictly worse capability than sending extra input packets.

§6.3's "spectate now, take the slot at the next kickoff" was a print statement, not a feature. The server logged "joined mid-match; spectating until the next kickoff" and then never did anything about it; on the client, _is_spectator was assigned once during _on_match_config_received and never revisited — and that handler returns early whenever _slots is non-empty, so no rebroadcast could ever promote an in-match spectator. The reconnect path only worked because a returning player is a fresh process that runs _on_match_config_received from scratch.

Implemented on both sides. The server queues late joiners in arrival order and drains the queue from _begin_kickoff() — before the reset transforms are read, so a promoted player's ship is placed by that same kickoff instead of being left wherever its previous owner abandoned it, and the controller swap lands on an already-frozen body, which is the entire reason §6.3 puts this at a kickoff boundary. A slot only becomes available once its player has gone and their 30s reservation has lapsed: §6.4 outranks §6.3, because taking a still-reserved slot would quietly break the reconnect promise. _abort_if_abandoned now counts a waiting spectator as somebody still present, for the same reason it already counts an outstanding reservation — otherwise the one person queued for the slot that just opened gets dumped to the lobby at the exact moment they were about to receive it.

The client gets a new broadcast slot_assigned (reliable, channel 0). Broadcast rather than addressed to the new owner, because every client holds its own slot list and one that names the wrong peer keeps flying somebody else's ship as a remote body; reliable, because unlike match_state there is no per-snapshot field that would re-converge a client that missed it. The promoted client undoes everything that made that body remote — fresh interpolator (its buffered samples describe the previous owner's flight), Godot's own physics interpolation switched back on, visual offsets cleared — and then deliberately does not unfreeze: it clears _local_prediction_ready so the next snapshot teleports it to a genuine authoritative pose and starts prediction there, exactly as a fresh client does. The controller-attach block was factored out of _on_match_config_received into _take_local_ownership() rather than copied, since a copy is a copy that drifts.

New --role=host-latejoin / --role=client-latejoin and --slot-reservation-seconds= (a server-side override in the same shape as --match-length, because the interesting moment is otherwise 30 real seconds away). Verified 4/4 from both sides: the joiner is queued, is not promoted merely because the reservation lapsed, takes the slot at the forced goal's kickoff, keeps the same ship instance, and both peers independently measure ~45.7m of movement under its input — the client's own number and the server's agree, so the promoted seat is real rather than relabelled. Control with a 90s reservation: the kickoff fires and nothing is promoted, the slot still reads the departed player's name, and the joiner stays a spectator. The existing spectator test is a second control — a spectator with no free slot is never promoted.

Two test-side races were fixed while getting there, both worth remembering because they produced confident false failures: sampling predicting at an arbitrary frame reported false for a client that then flew 45m, because unfreezing is queued and applied on the body's next _integrate_forces (task 0.15), so there is a real window where the state is PLAYING and _local_prediction_ready is set but ship.freeze has not flipped yet. Poll the whole condition with a deadline, never a proxy signal, and never one instant.

§6.4's reconnect was only ever graded from the server's side, and the client's side was failing the whole time. run_disconnect_host_check ticked 60 physics frames (1.0s) past the reclaim and then shut the server down — so the reconnecting client, whose wiring check waits a 2.0s settle before it looks at anything, had its peer torn out from under it every single run and reported current_scene is not NetworkedMatch after 2.0s. The host printed PASS throughout, and the host was the side anyone read. The hold is now a real window (default 8s), and the host additionally asserts that the reconnected player's input reaches the server and moves the ship the server owns — every other assertion there is slot bookkeeping that would hold identically for a client whose input pipeline came back dead, which is the exact failure the reservation exists to prevent. Both the position and the connection state are sampled while the peer is still connected, not once at the end of the hold: the client leaves on its own schedule, and an end-of-hold sample reported still_connected=false for a perfectly good run — the same mis-timed sampling a Phase 3 review caught in the CI gate.

New --role=client-reconnect grades the returning player: not a spectator, owns a slot whose peer_id is its own, has a real ship, rejoined a live match with the clock already known (_end_tick >= 0 — §6.2 step 2's bootstrap, since a player who must wait for the next goal to learn the score has not really rejoined), and its input still moves its ship. That set is chosen because a stale _last_match_config once made a reconnecting player a spectator, and that bug was visible in this scenario's own logs while it reported PASS. Verified 3/3 both sides, with a control that rejoins while the slot is still occupied and correctly fails on is_player=false. The first version of that control failed with the generic "lost its ship mid-drive", so the spectator case is now reported before the drive rather than after.

tools/replay_dump.gd reads a log back — record counts by kind, plus how much of the input sequence stream actually reached the server once redundancy is counted. It is committed rather than left in a scratch directory because it is what turned "the server dropped some input" into the numbers above, and a log nobody can read is half a feature.

New/changed test surface: --exercise-match-state (both roles; host forces a goal, client validates the whole observed sequence and the wire byte), --role=host-disconnect for the 5.6/5.7 three-process scenario (paired with --role=client-reconnect, which grades the returning player), --role=host-latejoin/--role=client-latejoin plus --slot-reservation-seconds=<s> for §6.3's kickoff promotion, --match-length=<s> to reach FULL_TIME in a short run, --replay-log=<path>, --fill-bots/--no-fill-bots, --max-spectators=<n>. The ball-contact scenario now steers at the ball with closed-loop real input instead of a hand-tuned fixed-heading burst, which 5.3 broke by adding KICKOFF_YAW_JITTER (0 contacts in 3/3 runs); it thrusts while turning rather than hovering to aim, which took it from 2/3 to 5/5.

Phase gate: a full 3v3 start-to-finish including a mid-match disconnect and a late joiner. Not yet run — every scenario above was verified at 1v1 (plus a two-bot CI match). The 3v3 gate needs a real multi-client session; it is item B of §0, alongside Phase 4's un-run human playtest (item A).

Phase 6 — Dedicated server productionisation

# Task Acceptance
6.1 [P] DONE. Export preset (dedicated_server=true, custom_features="dedicated_server") and run/main_scene.dedicated_server, mirroring the existing run/main_scene.training mechanism Linux Dedicated Server builds
6.2 [D:6.1] DONE. Verify the stripped export boots and scores a goal Docker smoke runs two exported-server matches and observes server-owned goals from two headless clients
6.3 [P] DONE. Full CLI surface plus a config-file fallback Unit tests cover precedence, validation, and --help
6.4 [P] DONE. Structured logging (join, leave, goal, kick, rate-limit, tick overrun) with --log-level Greppable stdout/stderr events exercised in the smoke
6.5 [P] DONE. Arena rotation between matches; --max-matches N drain-and-exit Smoke asserts two different arenas and server_draining
6.6 [P] DONE. systemd unit, Dockerfile, SERVER.md (ports, firewall, sizing per §1.4, and the SIGTERM caveat) A third party can host from the docs alone
6.7 [D:3.6] [P] DONE. CI builds the server export and runs the smoke test against the exported binary, not source .github/workflows/dedicated-server-smoke.yml runs make verify-phase6 on clean checkout

dedicated_server=true enables Godot's strip-visuals export mode, which replaces meshes and textures with placeholders per resource. Every relevant site is already headless-guarded — ship.gd:167, ball.gd:25, goal.gd, arena_boundary.gd — so the code should be safe. Verify it against a real stripped build anyway; this is the kind of thing that fails silently.

Docker/VPS is the primary v1 deployment path. Raw ENet self-hosting needs port forwarding, and SDR is Phase 7 — so Phases 16 ship something that works on LAN or a VPS and nowhere else. That is fine, but say it out loud rather than letting a player discover it.

Godot 4 gives GDScript no SIGTERM hook. SIGTERM/Ctrl-C kills the process immediately and clients see an ENet timeout (~5 s). Acceptable — but document it rather than letting it be discovered. --max-matches N under a process supervisor covers planned drains.

Rcon is deferred past v1. An authenticated remote command channel is a real security surface, and --max-matches plus a supervisor covers most of the need with none of it.

Phase gate: docker run a server, connect from another machine over the internet, play a full match. Precondition, not a footnote: §0 item C — slot reservations keyed on display name alone — is fixed by task 7.4, so exposing this build to strangers is gated on that, not on this phase.

Phase 7 — Steam transport, browser, identity

# Task Acceptance
7.1 [D:1.2] IN PROGRESS. GodotSteam integration and custom export templates — client and headless server Pinned build inputs and the reproducible validation command are documented; awaiting the custom binaries/SDK access
7.2 [D:7.1] IN PROGRESS. NetTransport boundary extracted with ENet and feature-gated steam_transport.gd (SteamMultiplayerPeer, SDR); advertising waits for ISteamGameServer work NetworkManager.host/join(..., transport) selects explicitly; stock builds reject Steam without ENet fallback
7.3 [D:7.2] [P] IN PROGRESS. Server-browser UI and ISteamMatchmakingServers adapter remain intentionally unimplemented until the pinned GodotSteam client API is available; ENet direct-IP remains the supported browser-free path No server_browser.tscn or fake Steam API has been added; implementation must wait for real Steam SDK/API access so Internet/LAN/favourites/history behavior can be verified against the actual service
7.4 [D:7.2] [P] IN PROGRESS. TicketVerifier now supports a synchronized backend ban decision before single-use ticket consumption; auth tickets in helloBeginAuthSession, Steam identity in the roster and persistent ban list remain server/domain/auth.go and adversarial tests reject banned identities without consuming their ticket and allow a later verification after unban; GodotSteam auth integration, server-side VAC state and durable ban storage remain
7.5 [D:7.2] [P] IN PROGRESS. SteamBootstrap gates initialization on the steam feature, SteamMultiplayerPeer class and Steam singleton; explicit Steam selection fails closed, while ENet remains the default and never becomes an implicit fallback test_net_transport.gd proves stock builds keep ENet available and reject unavailable Steam requests without returning an ENet peer; custom Steam client/server export smoke remains blocked on pinned GodotSteam binaries, and full ENet runtime verification remains blocked on the absent Godot executable
7.6 [D:7.4] IN PROGRESS. Pure Go AuthCoordinator models pending/accepted/rejected/cancelled backend auth sessions around the single-use verifier, including maintenance expiry sweeping; store.PostgresSessions persists only token digests; the authenticated API issues sessions only from an injected verified-identity provider; Godot ControlPlaneClient.login_steam() now submits only the Web API ticket, validates the opaque response and stores the session in memory server/domain/auth.go, server/store/session_sql.go, server/api/service.go, control_plane_client.gd and adversarial tests keep identity unavailable while pending, reject cancellation/expiry/wrong-attempt/replay, expire abandoned attempts at the boundary, release identity only after verifier success, reject invalid session inputs/extra identity fields, prove API delegation, validate ticket/session header boundaries and issue opaque sessions; real Steam BeginAuthSession/EndAuthSession adapter, login UI and live PostgreSQL/session integration remain
7.7 [D:7.1] [P] Obtain the production App ID, publisher key, SDR coordinator SDK/signing approval, certificates and hosted-data-centre support from Valve Production prerequisites and rotation owners are recorded; no Spacewar credential or development certificate can reach a release build
7.8 [D:7.6,7.7] Ticketed Hosted Dedicated Server SDR: routing registration, coordinator-issued player→server relay tickets, client ticket installation, reconnect and expiry Two real accounts complete and reconnect to an assigned dedicated match through SDR; server/player IPs are not exposed; ENet gates remain green

The transport interface is written here, not in Phase 1. Eight virtual methods (begin_auth, advertise, get_identity, supports_server_browser…) designed against an API nobody on the project has used will be wrong. Write NetworkManager._make_peer() concretely in Phase 1 and extract the boundary once there are two real implementations. Locked decision 3 guarantees the ENet path is never deleted, so there is no migration risk in waiting.

GodotSteam requires custom engine builds and export templates — including for the headless server. That is the part people discover three weeks in. Budget for it.

Phase 8 — Matchmaking, ranked ladder, per-match server autoscaling

1.0 launch blocker. Full design and reasoning: docs/MATCHMAKING.md. Nothing here is implemented. Unlike Phases 07 this phase adds a component outside the Godot project — a backend service — and that is the largest architectural departure in the project's history, so read the design doc before picking up any task below.

This inverts the server model. Phases 17 build a community server: it runs forever, waits for --min-players, plays a match, rotates arena, repeats, and players find it by IP or (7.3) the server browser. Matchmaking makes the player durable instead — queue, get grouped by rating, and a server is allocated for that one match and destroyed after. Both models ship; they are different playlists, not a replacement.

Hard dependency on 7.6 and 7.8. Slot reclaim is keyed by display name today. A rating attached to a spoofable identity is farmed trivially, so no queue ships before single-use verified identity lands. Production allocation also depends on the ticketed Hosted Dedicated Server SDR route; ENet remains the local/CI/community transport, not a silent production fallback.

8A — Architecture, contracts and data

# Task Acceptance
8.1 DONE. Add an ADR locking Go + PostgreSQL + Redis, provider-portable Kubernetes, Agones, ticketed Hosted Dedicated Server SDR, EU/NA fleets and independently runnable API, matcher, allocator and maintenance roles; keep README.md/docs/TECH_STACK.md consistent docs/ADR-001-matchmaking-platform.md names boundaries/rejected alternatives; current docs name the locked stack and replaceable provider; no application code calls a provider allocation API
8.2 [D:8.1] DONE. Encode the launch SLOs from docs/MATCHMAKING.md: RTT, allocation/connect latency, 99.9% allocation/result success, API latency and tick health docs/MATCHMAKING-SLOs.md defines each metric, denominator, percentile/window, owner, alert threshold and release evidence
8.3 [D:8.1] DONE. Publish versioned OpenAPI + WebSocket contracts for Steam login/session, profile/rating, queue create/heartbeat/cancel/resume, proposal accept/decline, assignment/status, server registration/roster/result/shutdown server/contracts/v1/ contains machine-readable REST/events contracts and dependency-free structural tests; REST resync is specified by the contract; server/api/service.go also exposes the documented /api/v1 route names (including server-assigned idempotent queue ticket IDs and DELETE cancellation) alongside the existing client /v1 routes, covered by TestDocumentedContractRoutesAdaptToServiceAPI
8.4 [D:8.3] DONE. Define opaque IDs, legal queue/match state transitions, revisions and idempotency keys server/contracts/v1/state-transitions.json locks terminal states, legal edges, stale-revision handling and same-key replay/conflict behavior; contract tests cover the invariants
8.5 [D:8.4] IN PROGRESS. Initial PostgreSQL migration now defines durable idempotency keys, queue ownership/active-participation fencing, identities, sessions/revocations, ranked seasons, ratings/events, matches/participants, penalties, results, audits and outbox; follow-up migrations persist server-derived queue probe RTT metadata, the allocator GameServer/allocation registry, matcher-selected proposal region/protocol/team/slot plans, and leased allocating-match claims server/migrations/0001_initial.sql, 0003_queue_probe_metadata.sql, 0004_allocator_registry.sql, 0005_proposal_match_plans.sql, 0006_match_allocation_claims.sql, migrations/runner.go, cmd/migrate and static checks cover the durable tables, uniqueness/check constraints, Redis-as-cache boundary and serialized forward migration recording; opt-in scripts/run_postgres_integration.sh now runs the runner and real queue/assignment ownership, idempotency, revision and expiry checks through pgx. migrations.Rollback now reverses N most-applied migrations via migrations/down/<version>.sql files (one per existing migration, dropping in FK-safe reverse order), wired into cmd/migrate --rollback=N, verified live: roll back to empty and reapply reaches the same schema; remaining serializable adapters and cache-loss repair remain
8.6 [D:8.3,8.4] IN PROGRESS. Add allocated-mode ServerConfig compatibility fields as opt-in defaults ServerConfig now validates allocation mode, match/server IDs, playlist version, client build, future assignment expiry, image digest, transport and EU/NA region; server_boot.gd fails closed for the not-yet-wired Steam SDR transport, constrains allocated processes to one match, and emits allocation identity/transport in server_started; signed-authorisation admission, dynamic endpoint wiring and full manifest/runtime tests remain

8B — Authentication and secure control plane

# Task Acceptance
8.7 [D:7.6,8.3] IN PROGRESS. Pure Go ticket policy binds the expected App ID and verified identity, rejects expiry/replay/wrong app/malformed tickets, and consumes each ticket once. cmd/control-plane now wires SessionIssuer: store.PostgresSessions{DB: db} (same discovery/fix pattern as §8.10's ResultSubmitter: the adapter already correctly implemented Issue, just wasn't wired, so /v1/session/steam 503'd even before considering whether SteamLogin — the real, still-correctly-unwired Steam blocker — was available) server/domain/auth.go covers single-use and binding invariants; real AuthenticateUserTicket backend adapter, bans, publisher secret store and Steam verification remain. A separate test-only server/cmd/testkit-api binary (never referenced by any Dockerfile/K8s manifest) substitutes a fake Steam login accepting any non-empty ticket, enabling §8.40's real Go+Postgres+Godot integration test
8.8 [D:8.7] IN PROGRESS. Pure Go session policy issues opaque short-lived tokens, stores only digests, authenticates by verified player identity and supports revocation server/domain/auth.go covers wrong-token/expiry/revocation behavior; distributed revocation, account/IP limits, request limits and production session persistence remain
8.9 [D:8.4,8.7] IN PROGRESS. Pure Go join policy binds SteamID/player/match/server/team/slot/protocol/expiry, rejects duplicate roster slots, permits same-identity reclaim through backend loss and fences prior server-owned generations server/domain/reconnect.go covers SteamID/server/slot binding, expiry, repeated reclaim, grace boundary and old-generation fencing; signed token issuance/verification, persistent leases, Godot hello transport and production integration remain
8.10 [D:8.5,8.31] IN PROGRESS. Dependency-free workload credential policy validates an adapter-verified signature, issuer/audience/time bounds, namespace/service account, pod UID, GameServer UID, allocation ID, match ID and server ID before result submission; server/workload now parses projected compact JWTs, rejects unsigned/malformed/ambiguous tokens, verifies signing input through an injected trust callback, and delegates exact binding to the domain policy; the API exposes the workload-authenticated server result route. WorkloadVerify is now wired for real -- the blocker this row previously named (cmd/control-plane/main.go never wiring it, so /register and /result 503'd on every real request) is closed by taking a different, equally-valid approach instead of the Kubernetes-JWT one: serverMutation only ever compares WorkloadBinding.ServerID/.MatchID (and AdvanceServerRegistration only additionally needs .AllocationID) -- nothing downstream requires Namespace/ServiceAcct/PodUID/GameServerUID, so a Kubernetes-issued token was never actually required, just the assumption that it was. server/workload/signed_token.go mints and verifies a short-lived HMAC-signed token with a secret only the control plane holds (the same trust model domain.SessionStore already uses for player sessions), needing no cluster JWKS/TokenReview to validate -- signature and expiry are fully self-contained. store.AllocationBindingStillValid cross-checks the claims against the durable allocations table for defense-in-depth (a validly-signed token naming a never-recorded or mismatched allocation is still rejected). api.WorkloadVerifierFromSignedToken combines both and is wired into cmd/control-plane (--workload-secret/COSMIC_CLASH_WORKLOAD_SECRET; a startup warning fires and the route stays 503 if it's left unset) and cmd/testkit-api (fixed test secret) server/domain/workload.go, server/workload/jwt.go, server/api/service.go and adversarial tests reject every binding mutation, missing/unverified signature, none/malformed JWT, ambiguous audience, server/match mismatch and time boundary; server/testkit/pipeline_test.go carries allocation identity through the offline result path; cmd/control-plane now wires ResultSubmitter: store.PostgresResults{DB: db} (a ready-made adapter that had been referenced from nowhere at all, not even a test); server/workload/signed_token_test.go covers round-trip, tampered payload, wrong secret, malformed input and the exact expiry boundary; server/api/workload_verifier_integration_test.go (opt-in, real PostgreSQL) covers acceptance against a real allocation row, rejection of an unrecorded allocation, rejection of a real allocation id paired with a mismatched match id, and the previously-503 Service.WorkloadVerify field itself now succeeding -- all verified clean with -race across multiple runs; cmd/control-plane/main_test.go's TestServerRoutesRequireWorkloadVerifyToBeWired now documents and pins the misconfigured-secret case specifically, not the "always unwired" case. What's still missing: the actual delivery channel. Nothing yet mints a real token at allocation time or hands it to a running pod -- server/agones.Client.Allocate would need to request a third cosmic-clash.io/workload-token annotation (alongside the match-id/allocation-id ones it already requests) computed from the same secret, and the supervisor would need to read it from there instead of (or in addition to) the Kubernetes-projected-token file path it currently reads from disk; live duplicate/conflict alerting also remains
8.11 [D:8.1] DONE. Write the threat model: forged clients/replay/queues/results, floods/bots, pod/insider compromise, gameplay and API DDoS, SDR signing-key theft, dependencies and denial-of-wallet docs/THREAT-MODEL.md records prevention, detection/response, owner and residual risk for every threat; it separates offline CA/online signer, client/game-server/PostgreSQL/Redis trust boundaries and recovery behavior
8.12 [D:8.11] IN PROGRESS. Provider-neutral Kubernetes baseline enforces restricted namespace admission, non-root/read-only/no-capability workloads, separate service accounts, allocator-only RBAC, default-deny networking and explicit edge/data/DNS/Agones flows; application manifests consume externally populated Secret objects; the Go API now has an optional bounded per-replica rate-limit/429 boundary deploy/k8s/base/ plus server/security/test_kubernetes_policies.py, server/api/rate_limit.go and adversarial tests cover static hardening, secret-reference invariants, fixed-window limits and bounded key memory; private-store provisioning, distributed/global quotas, edge DDoS/WAF/origin shielding, WebSocket limits, overload shedding, encrypted backups and live policy/load tests remain
8.13 [D:8.12] IN PROGRESS. Docker/Kubernetes references are digest-pinned, a dependency-free checker rejects mutable tags/plaintext credentials and concrete release overlays can reject template digests; release documentation defines SBOM, dependency/image scanning, signing, admission verification and a 24-hour critical-fix SLA scripts/verify_supply_chain.py, server/security/test_supply_chain.py, docs/SUPPLY-CHAIN.md and .github/workflows/supply-chain.yml cover repository policy and provenance requirements; registry SBOM/scan/sign/admission execution and a concrete production overlay remain

8C — Queueing, matchmaking, playlists and rating

# Task Acceptance
8.14 [D:8.4,8.5,8.8] IN PROGRESS. Pure Go queue domain enforces one active ticket per verified player under concurrent mutation, 10 s heartbeat/30 s expiry, retry-safe create/heartbeat/cancel, owner-only recovery reads and deterministic candidate projection; store layer adds a rebuildable candidate-cache boundary and authenticated HTTP queue adapter with playlist/build/protocol compatibility metadata server/domain/queue.go, server/store/candidates.go, server/store/queue_sql.go, server/store/redis_candidates.go and server/api/service.go cover ownership/expiry/idempotency, concurrent create fencing, candidate/player ownership binding, owner/revision-scoped SQL heartbeat/cancel/recovery, injectable PostgreSQL queue backend selected by the HTTP service, authoritative queue-to-cache rebuild, expired recovery as a terminal error, owner-scoped SQL recovery with authoritative expiry handling, server-owned candidate resolution, strict compatibility metadata, TTL-bound Redis upsert/remove/snapshot, API create/heartbeat/cancel projection hooks, optional control-plane Redis configuration and atomic durable-source repair on partial/malformed cache state; opt-in real PostgreSQL execution now covers create/replay/active-player fencing, owner recovery, revision-fenced heartbeat/cancel and expiry, while miniredis covers Redis behavior and repair-source failure — this suite had never actually been run clean against a live database before: doing so once found CreateQueueTicket passing one extra unbound argument to its insert, which failed every real ticket creation with a param-count mismatch (fixed, re-verified against a real postgres:17-alpine container). A separate opt-in real-Redis suite (server/store/redis_integration_test.go, scripts/run_redis_integration.sh, COSMIC_CLASH_REDIS_ADDR-gated) now covers upsert/snapshot/remove, a real TTL actually waited out, and the "lost keyspace" repair path against a genuine FLUSHALL — including that the repair persists back to Redis, not just returned an in-memory answer. TestPostgreSQLConcurrentQueueHeartbeatIsRevisionFencedUnderRealRace races 5 concurrent same-revision heartbeats against real PostgreSQL: exactly one wins, the durable revision lands at exactly 1; live Redis failover-under-load and worker integration remain
8.15 [D:7.8,8.3] IN PROGRESS. Pure Go probe validation treats Steam location as opaque, requires nonce/freshness/region and server-computed RTT, and implements discrepancy quarantine/release; authenticated HTTP now accepts only opaque location/nonce input through a server-owned probe provider and records validated RTT into the active player's durable queue ticket; durable queue projections have a server-derived RTT JSON field for matcher reads server/domain/probes.go, server/migrations/0003_queue_probe_metadata.sql, adversarial fixtures and server/api/service.go/store/queue_sql.go cover stale/wrong/forged evidence, the 25 ms/30% threshold, three-sample quarantine, five-clean release, authenticated provider arguments, rejection of client RTT fields, player/ticket/expiry binding, persistence failure, playlist-scoped candidate reads and bounded metadata decoding; Steam coordinator, regional probe adapters and multi-region probe population remain
8.16 [D:8.14,8.15] IN PROGRESS. Pure Go candidate/team selection implements the <=100 ms ceiling, pairwise widening tolerance, anchor inclusion, deterministic set/region scoring and balanced team partitioning; queue-backed formation now consumes the server-owned projection, fences duplicate player identities and rejects playlist/build/protocol mixing; the matcher worker performs bounded formation and delegates the final claim to the durable proposal transaction; the runnable matcher now supports explicitly enabled ranked six-player polling with durable verified-Steam identity lookup. Fixed a real crash-loop: Worker.Run treated every RunOnce error as fatal to the whole loop, including "no compatible candidates" (FormFromQueue's completely routine answer when currently-queued players share no verified region) — found building a live two-player integration attempt (see below): two real players with no common region crashed the entire matcher process, taking matchmaking down for every other player in the playlist, and would crash-loop again on restart since the same incompatible candidates stay queued. Now only genuine static misconfiguration (ErrWorkerNotConfigured/ErrUnsupportedPlaylist/ErrInvalidMatcherSize) stops the loop; everything else retries next interval server/domain/matcher.go, teams.go, server/matcher/worker.go, server/store/queue_sql.go and adversarial fixtures cover no-common-region, tolerance boundaries, lexical ties, mean-rating balance, malformed candidates, duplicate identities, compatibility mismatches, incomplete batches, source failure, durable claim failure, queue-backed oldest-anchor formation and incomplete ranked identity metadata; two new tests cover Run (not just RunOnce) surviving a per-pass error via a real concurrent goroutine, and still stopping immediately on a real configuration error, both clean across repeated -race runs. A live two-player Godot proposal integration attempt is on disk but not committed: Game/tests/control_plane_proposal_smoke.gd/.tscn and scripts/verify_control_plane_proposal_integration.sh exist and found the bug above, but the session paused running further concurrent headless Godot processes after discovering they'd been causing native engine crashes (macOS crash reporter, EXC_BAD_ACCESS/SIGBUS) intermittently all session, confirmed by the user; the two-player script was never itself verified to a clean pass. Arena selection and long-running worker integration remain
8.17 [D:8.14,8.16] IN PROGRESS. Pure Go proposal policy sends a 10-second response window to every selected human, requires unanimous acceptance, applies exact decline/timeout cooldowns and ranked escalation; authenticated API exposes revisioned accept/decline mutations; formed matches now pass through a playlist-aware proposal boundary. Fixed a real severe bug: declining a proposal never requeued anyone's ticket — every participant, decliner included, was left stranded at PROPOSED (invisible to the matcher, still blocking a fresh queue_create, renewable forever by an ordinary heartbeat) with no path back into matchmaking. ProposalDeclineRequeueSQL now requeues every participant to QUEUED with a fresh expiry on decline; the not-yet-built decline cooldown mentioned here can later exempt the decliner specifically, but leaving anyone stuck today wasn't that cooldown, it was just broken. The same bug's timeout sibling is fixed too: a proposal that simply expires (no unanimous response inside the window) hit the identical gap in ProposalExpireSQL/ProposalParticipantExpireSQL, reached from both GetProposal (a client recovering after missing the expiry event) and RespondToProposal (a response arriving after the window); ProposalExpireRequeueSQL mirrors the decline fix, guarded on state = 'EXPIRED' so it's safe to call unconditionally. Closed the remaining responsiveness gap too: cancelling a queue ticket directly while it's part of an OPEN proposal used to leave the other participant waiting out the full window instead of being told immediately; CascadeCancelToOpenProposal now declines and requeues the proposal in the same transaction as the cancel server/domain/proposal.go, formation.go and server/api/service.go plus adversarial fixtures cover partial/unanimous response, expiry, replay/conflict, stale API revision, casual lineup preparation and ranked metadata validation; real PostgreSQL integration tests confirm the decline, timeout, and cancel-cascade paths all requeue every participant (decliner/uninvolved participant/cancelling player's partner alike) to QUEUED with a refreshed expiry, visible again to ListQueuedCandidates (the matcher's own read), and that a cancelling player's own ticket correctly stays CANCELLED rather than being swept back up; clean across 5 runs each; queue precedence, allocation integration and the decline cooldown itself remain
8.18 [D:8.5,8.14,8.17] IN PROGRESS. Go store layer defines PostgreSQL SERIALIZABLE whole-transaction retries and queue candidate/proposal claim SQL using FOR UPDATE SKIP LOCKED plus durable uniqueness/revision fences; proposal creation now persists its matcher-selected region/protocol/team/slot plan alongside proposal/participant claims, queue creation has a durable idempotency/owner-read adapter, participant-scoped proposal recovery now expires OPEN proposals and pending participants transactionally at read time, and proposal accept/decline now uses participant/proposal locks, revision fencing and durable idempotency; final unanimous API acceptance invokes replay-safe promotion of the exact stored ALLOCATING match/team/slot topology and claimed tickets to ACCEPTED; runnable matcher polling supports an optional Redis candidate projection that repairs empty/lost cache state from authoritative PostgreSQL before the durable final claim server/store/serializable.go, queue_sql.go, proposal_sql.go, proposal_recovery_sql.go, match_sql.go, redis_candidates.go, server/matcher/worker.go, server/api/service.go and tests cover retry classification, claim-boundary invariants, player/ticket/playlist mapping, durable queue replay/conflict, owner-scoped queue/proposal recovery, expiry at read and mutation boundaries, final-response promotion retry, exact match-promotion replay/conflict, fixed team/slot persistence, zero-row claim aborts, atomic statement ordering, incomplete matcher batches, source failures and empty-index Redis repair; opt-in PostgreSQL execution now covers queue create/replay/fencing, assignment persistence, proposal claim/promotion, accepted-proposal match promotion, participant recovery, unanimous response and rollback of partial claims — actually running this suite live (it had not been before) found ProposalParticipantExpireSQL had no expiry-time condition at all, so every call timed out every pending participant on the spot; the very first accept on any proposal then failed with a false conflict. Fixed with the same expires_at <= gate ProposalExpireSQL already used, re-verified live. A real concurrent-goroutine test now covers the two-matcher race this was missing: two proposals sharing one contested ticket, racing two real Postgres connections under -race, exactly-one-wins/loser-fully-rolls-back including the loser's own uncontested ticket, stable across 8 runs; allocation runtime integration tests remain
8.19 [D:8.18] IN PROGRESS. Pure Go casual lineup requires 26 humans with at least one per team, fills missing slots with explicit bots, permits kickoff-only bot-slot backfill and assigns no backfill penalty/rating update; proposal preparation now derives the lineup from formed teams server/domain/casual.go, formation.go cover both-team minimum, bot shape, live-play rejection, zero-penalty backfill and casual proposal composition; queue candidate selection, opt-in 10 s backfill proposals, reconnect/leave penalties and live integration remain
8.20 [D:8.18] IN PROGRESS. Pure Go ranked admission requires six unique verified solo humans, rejects bots/backfill/parties, and allows only random-enabled non-elevated arenas; proposal preparation requires matching metadata for every formed player server/domain/ranked.go, formation.go cover count, identity, party, bot/backfill, arena eligibility and formed-player metadata rejection; ArenaRegistry integration, allocation wiring and innocent-ticket restoration remain
8.21 [D:8.5,8.20] IN PROGRESS. Pure Go rating core implements canonical Glicko-2, daily inactivity, ranked 1/3 and casual 1/N human-opponent weights, deterministic opponent ordering, and authoritative draw/overtime/abandon scoring; certified result completion now applies per-player updates inside the durable transaction with lexical row locks and revision increments server/domain/rating.go, server/store/result_sql.go and tests cover canonical/inactivity/weight/invalid-input, draw/OT/abandon, ordered participant snapshots, lock/value re-read and rating update SQL; live PostgreSQL rating and seasons execution now covered (§8.23), and one concurrent result transaction case is covered: TestPostgreSQLConcurrentIdenticalResultSubmissionAppliesRatingsExactlyOnce races 5 identical concurrent result submissions and confirms the rating applies exactly once (exact-value match against an independently computed update, not just "some change"); a genuinely conflicting concurrent submission race remains
8.22 [D:8.21] IN PROGRESS. Pure Go ranked profile exposes the first ten games as provisional, derives tiers only through validated backend-owned rating bands, and keeps casual ratings outside the API; authenticated HTTP now returns the authoritative ranked view. "Authoritative ranked view" was durable-adapter-shaped but had no durable adapter: rankedProfile/profile only ever read an in-memory map, so every real GET /v1/profile/ranked 404'd regardless of a player's actual rating. RankedProfileProvider (interface) + store.PostgresRankedProfiles close it, preferred over the map when set so existing tests/literals are unaffected; LastSeasonID/SeasonHistory deliberately left unset (no season pointer on ratings, needs its own query/semantics) server/domain/rating.go, tier_test.go and server/api/service.go cover provisional override, exact band boundaries, malformed policy rejection, session authentication and ranked-only response fields; server/store/ranked_profile_sql.go, verified against real PostgreSQL via curl (a fresh identity correctly 404s through the real adapter) and via §8.40's integration test (control_plane_smoke.gd now asserts this exact 404 round-trips before queueing); persisted tier policy, client UI and reconnect transport remain
8.23 [D:8.21] IN PROGRESS. Pure Go ranked-only season policy compresses 25% toward 1500, clamps RD to 200350, preserves volatility/history, is idempotent by season ID, and defines exact 12-week windows/due detection; migration and Go store now persist a per-player/per-season marker and rating update atomically; cmd/maintenance runs bounded due-season batches with signal-bound shutdown server/domain/rating.go, season_test.go, server/migrations/0001_initial.sql and server/store/maintenance_sql.go cover compression, floor/cap, duplicate replay, window boundary, completed-season idempotence, bounded enumeration, row locking and conflict-safe rollover markers; opt-in PostgreSQL execution now covers the durable rating update, marker creation and duplicate replay without a second compression — re-run live for the first time as part of the wider integration-suite verification below, after fixing a test setup gap (a missing seasons row tripped the ranked_season_rollovers foreign key before the rollover logic itself ran); live maintenance/DB execution remains
8.24 [D:8.9,8.20,8.21] IN PROGRESS. Pure Go ranked connection policy binds match/server/player/team/slot/protocol, supports 60 s reclaim with server-owned generations, fences old connections, and applies the rolling 7-day 5 m/15 m/1 h/24 h abandon ladder; canonical signed authorisation issuance/verification now gates admission server/domain/reconnect.go, join_auth.go and adversarial fixtures cover repeated backend-independent reclaim, all signed claim binding, tampering, failed verification, old-generation fencing, grace boundary and deterministic cooldown audit ordering; persistent lease fencing, join transport and full match/result integration remain
8.25 [D:8.10,8.24] IN PROGRESS. Pure Go result policy binds match/server/workload identity, hashes canonical payloads, makes identical retries idempotent, leaves conflicts inert, separates integrity eligibility, classifies roster/simulation/result/fairness evidence, validates annotation signatures/digests, and exposes 5 m alert/30 m review delivery thresholds; Go store SQL now executes receipt → match lock → certified rating updates → completion → receipt acknowledgment → outbox atomically, and exposes bounded ordered outbox reads plus publish acknowledgements for replayable fan-out; OutboxDispatcher now delivers in order and acknowledges only after successful fan-out; the API validates workload-bound server result submissions and the PostgreSQL adapter repeats domain validation before invoking this durable boundary server/domain/result.go, workload.go, server/workload/jwt.go, server/api/service.go, server/store/result_sql.go and outbox.go plus adversarial fixtures cover credential binding, duplicate/conflict, annotation forgery, server/match mismatch, invalid direct-adapter payloads, delivery-outage-versus-integrity classification, commit ordering, idempotent SQL reconciliation, ordered rating locks, certified-update gating, unpublished-event replay/ack boundaries and delivery-before-ack failure ordering; opt-in PostgreSQL execution now covers result-pending completion, durable receipt/outbox publication, ack removal, identical replay and conflicting replay rejection, and a real concurrent-goroutine identical-submission race confirming exactly-once rating application; production credential verification, Agones annotation persistence/reconciliation and integrity evidence adapters remain

8D — Agones, allocation and regional scaling

# Task Acceptance
8.26 [D:8.1,8.6,8.12] IN PROGRESS. Provider-neutral Kustomize base now defines a restricted Agones Fleet with region/build/protocol/transport labels and UDP game port, plus distinct EU/NA overlays; the base avoids rewriting cross-namespace Agones RBAC deploy/k8s/base/fleet.yaml, overlays/eu, overlays/na and server/security/test_fleet_manifests.py cover labels, replica floor, UDP declaration, pod hardening, overlay distinction and RBAC namespace safety; live Kustomize/Agones rendering, second-provider fixtures, edge/network/DNS/secret and SDR POP/cert/public-UDP overlays remain
8.27 [D:8.26] IN PROGRESS. Go supervisor package provides local-safe Agones REST discovery, validates assigned address/port data, injects dynamic SDR_LISTEN_PORT/SDR_IP, performs explicit process-ready probing and Ready transition; direct mode bypasses Agones server/supervisor/ covers allocated/direct startup, invalid endpoint rejection, dynamic endpoint/Ready ordering and authenticated drain; allocated Godot now supplies a loopback readiness/drain control surface and agones_sdk.gd supplies sidecar Health/Ready/Shutdown/annotation REST operations; metadata watch, real Agones annotation/shutdown confirmation and emulator integration remain
8.28 [D:8.6,8.27] IN PROGRESS. Supervisor separates explicit process-ready from Agones Ready and never scrapes stdout; allocated mode refuses to mark Ready without a configured readiness probe. The durable control-plane counterpart now exists: a workload-authenticated POST /v1/servers/{id}/register (and its /api/v1 contract alias) advances a match's ALLOCATING -> PROCESS_READY -> ASSIGNMENT_READY state, and every participant's queue ticket with it, as one idempotent SERIALIZABLE transaction, gated on every participant already holding a live, unexpired assignment. The supervisor now calls it: once Agones Ready succeeds, it POSTs process-ready (assignment_ready=false) using a workload token read fresh from disk each call (matching kubelet's in-place rotation of a projected service-account token), and a registration failure kills the child rather than leaving an Agones-Ready-but-control-plane-unregistered process running; ControlPlaneURL unset (the default) is a total no-op. It then reports assignment-ready too: server_boot.gd already verifies its mounted roster synchronously before /ready is ever exposed (so process-ready implies the roster was valid), and the API's ASSIGNMENT_READY gate checks only durable assignments rows server-side — so no new Godot-side state was needed, correcting an earlier overcautious note here. The supervisor retries assignment-ready (default 5 attempts, 2s apart, configurable) since those durable rows may lag process-ready slightly; a persistent failure there does not kill the child, unlike process-ready. Per-allocation data (currently match-id) now has a real channel to an already-Ready pod: server/agones.Client.Allocate requests cosmic-clash.io/match-id/cosmic-clash.io/allocation-id as GameServerAllocation.spec.metadata.annotations (Agones applies these to the allocated GameServer's own object_meta — the only channel that exists post-allocation, since env vars are fixed at pod creation), and the supervisor reads them back from its existing /gameserver SDK call, falling back to them only when MatchID isn't explicitly configured. The image now exists: a new Dockerfile game-server target packages the supervisor as ENTRYPOINT alongside the same dedicated-server export server produces server/supervisor/ tests prove Ready follows the probe and direct mode remains functional; server_control.gd, agones_sdk.gd and process-level smokes prove loopback /ready, /health, bearer-protected /drain, sidecar-shaped Health/Ready calls and drain admission fencing; server/api/service.go, server/store/allocation_match_sql.go and adversarial tests cover the registration route, digest/protocol validation, idempotent replay/conflict and the assignment-count gate; server/supervisor/supervisor_test.go covers opt-in registration, the workload-token/body/idempotency-key shape, the kill-on-failure path, the match-ID annotation fallback (both that it's used and that its absence fails closed before any HTTP call), the full process-ready-then-assignment-ready sequence and body shapes, and a retry test that fails assignment-ready twice with 409 before succeeding, asserting Start() still succeeds and the child is never killed; server/agones/allocation_test.go covers the requested annotations. docker build --target game-server verified for real: both binaries present, correct permissions, supervisor prints its usage; server/store/stalled_allocation_sql.go/_test.go and a live TestPostgreSQLStalledAllocationsAreReclaimedWithoutPenalisingPlayers cover the deadline boundary (a recent match must survive untouched), the no-penalty requeue and refreshed expiry, participant release, and idempotence against a second pass. Fixed in passing: Dockerfile's server stage's ubuntu base digest had gone dead on Docker Hub (docker pull returned "not found", verified independently) — make verify-phase6 was silently broken for a clean build before the re-pin; confirmed fixed with a full make verify-phase6 run (arenas rotated, both goals observed, clean teardown). Still not wired into deploy/k8s/base/fleet.yaml: the manifest doesn't reference the game-server image or invoke any supervisor flags yet — the concrete remaining step is deciding and adding the per-deployment values (--control-plane-url, --workload-token-path plus the projected token volume, --server-id-env/--image-digest-env Downward API wiring), deliberately not guessed at here since they're environment-specific. deploy/cosmic-clash-server now wraps its exec in stdbuf -oL -eL (falling back to unwrapped if unavailable), fixing a real, live-confirmed bug: a genuinely detached (docker run -d) container showed zero docker logs output — not even the startup line — for 20+ seconds while the process ran normally, and docker stop's SIGTERM lost that buffered output permanently rather than delaying it; re-verified fixed against the real launcher script, then a full make verify-phase6 re-run confirmed no regression. Health-reclaim now exists: store.ExpireStalledAllocations reclaims a match stuck in ALLOCATING/PROCESS_READY/ASSIGNMENT_READY past a deadline (server crashed, or was reclaimed by Agones as unhealthy, before ever registering) by failing the match and requeuing every participant to QUEUED with a fresh expiry rather than penalising them — task 8.50's own "infrastructure-caused cases cannot penalise affected players" criterion directly settles the requeue-vs-fail design question this had been blocked on. Wired into cmd/maintenance alongside the season-rollover sweep (--stalled-allocation-deadline default 2m, --stalled-allocation-batch). Superseding the fleet.yaml framing above: §8.10's WorkloadVerify blocker is now closed (a control-plane-self-issued signed token, not a Kubernetes JWT — see §8.10), so /register and /result no longer 503 unconditionally once --workload-secret is set. What fleet.yaml still can't reach yet is a real token: nothing mints one at allocation time and hands it to a running pod (§8.10's "what's still missing" — the cosmic-clash.io/workload-token annotation and the supervisor reading it), on top of the manifest itself still not referencing the game-server image or supervisor flags — both deliberately not guessed at here since they're environment-specific
8.29 [D:8.26,8.27] IN PROGRESS. Supervisor discovers and validates the Agones endpoint, propagates the actual dynamic --port, and exports SDR_LISTEN_PORT/SDR_IP only for Hosted-SDR while preserving an isolated ENet path server/supervisor/ tests cover invalid address/port rejection, dynamic port argument/env propagation and SDR-vs-ENet separation; real Agones dynamic/passthrough mapping, POP/cert/firewall/NAT and multi-match fixture remain
8.30 [D:8.18,8.26,8.28,8.29] IN PROGRESS. Pure Go allocator filters Ready servers by region/build/protocol/transport, atomically claims one with idempotent allocation replay, verifies unanimous accepted-proposal/playlist/participant invariants before provider invocation, and now owns the assignment-publication boundary; PostgreSQL adds durable GameServer registration and compatible SKIP LOCKED claims with request-digest fencing, plus a leased ALLOCATING-match claim that derives the immutable compatibility tuple and fences bind/release by deterministic allocation ID; bind atomically attaches only a recorded provider allocation and advances every accepted participant ticket to ALLOCATING; server/agones strictly projects Ready GameServers from Fleet compatibility labels, submits/validates namespaced GameServerAllocation responses and dynamic endpoints; server/allocator reconciles provider success into durable state before exposing the endpoint; cmd/allocator refreshes that Ready projection before driving the lease → provider → durable-record → match-bind sequence and rebinds a recovered durable provider allocation without a second provider call server/domain/allocator.go, server/store/allocator_sql.go, server/store/allocation_match_sql.go, server/store/allocation_match_adapter.go, server/agones/allocation.go, server/allocator/service.go, server/allocator/worker.go, server/cmd/allocator, server/migrations/0004_allocator_registry.sql, 0006_match_allocation_claims.sql and tests cover strict Ready-server projection, stale-projection protection for allocated rows, deterministic compatible selection, exhaustion, conflicting/identical allocation replay, allocation-match lease recovery/bind fencing, atomic participant lifecycle transition, recovered-allocation binding without provider recall, immutable Fleet selector labels, provider ambiguity lease retention, unknown allocations, SQL claim ordering, invalid input, provider error/malformed response/IPv6 endpoint handling, accepted-proposal gating, durable-reconciliation failure isolation and assignment replay/conflict; TestPostgreSQLAllocatorClaimReplayAndCapacityFence and TestPostgreSQLAllocationMatchClaimLeaseAndBindFence cover the live database paths when the disposable database gate is run; TestPostgreSQLConcurrentAllocationClaimNeverDoubleBooksAReadyServer now races real concurrent claims against fewer Ready servers than requesters and proves no double-booking and no stray errors, stable across repeated -race runs; unknown provider-outcome reconciliation, signed roster metadata and live Agones integration remain
8.31 [D:8.9,8.30] IN PROGRESS. Pure Go assignment gate requires Allocated state, exact allocation ID/match/server/region/build/protocol/transport compatibility, non-empty hosted endpoint and verified manifest signature before exposure; allocator publication cannot expose Ready state; durable roster persistence now verifies each canonical join-authorisation signature before publishing player rows; allocator service gates roster publication on allocated state and endpoint presence server/domain/assignment.go, allocator.go, store/assignment_sql.go plus adversarial fixtures cover early-connect, tampered signature/manifest, wrong compatibility, empty endpoint, unknown allocation, forged roster signature, valid signature, premature publication and post-publication mutation rejection; Agones metadata watch, hosted-address registration, production signer and client-ticket publication remain
8.32 [D:8.2,8.26,8.30] IN PROGRESS. Provider-neutral FleetAutoscaler baseline preserves a two-process Ready buffer, caps warm capacity, and leaves Allocated scale-down independent of the Ready floor; Fleet image references remain digest-pinned for current/rollback pre-pull deploy/k8s/base/fleet-autoscaler.yaml and manifest tests cover Fleet ownership, Buffer policy and floor/cap invariants; regional on-demand node pools/failure domains, pre-pull rollout, warm-allocation p95/p99 and N+1 certification remain
8.33 [D:8.26,8.32] IN PROGRESS. Fleet scheduling now requires on-demand capacity and spreads Ready processes across zones with skew 1; the autoscaler preserves the two-process Ready floor deploy/k8s/base/fleet.yaml and manifest tests reject interruptible placement and single-zone concentration structurally; regional node pools, forced node-loss testing and measured N+1 headroom remain
8.34 [D:8.28,8.29] Native x86_64 benchmark of boot-to-process-ready and assignment-ready, p99 CPU/RSS/network and 60 Hz ticks; limits/node cap with 30% headroom Measurements replace old estimates and certify density with no tick backlog
8.35 [D:8.17,8.19,8.20,8.30,8.31] IN PROGRESS. Pure Go initial-connect policy decides ranked 30 s no-show cancellation with abandon ladder and casual 60 s bot start only when each team has a human; empty-team casual allocations cancel server/domain/noshow.go covers wait/deadline boundaries, deterministic no-show/innocent ordering, ranked cooldown history and no pre-live rating action; persistent ticket restoration, allocation shutdown, bot spawn and live integration remain
8.36 [D:8.10,8.25,8.28,8.30] IN PROGRESS. Supervisor exposes an authenticated loopback-only drain request boundary that never places the token in command arguments/logs and rejects remote/partial/query-bearing configurations; the base now includes a two-Ready PodDisruptionBudget; Supervisor.Run and cmd/game-server-supervisor now orchestrate signal-bound drain-before-kill with a bounded grace deadline server/supervisor/, server/cmd/game-server-supervisor/, server_control.gd and deploy/k8s/base/game-server-pdb.yaml cover bearer-token enforcement, loopback URL validation, secret-safe configuration, missing drain credentials, readiness transitions, Ready-floor disruption protection, graceful child exit after drain and force-kill of an unresponsive child; live 300 s/285 s lifecycle, PDB/Fleet drain and infrastructure-abort classification remain
8.37 [D:8.5,8.10,8.25,8.26,8.31] Horizontally scaled primary control plane + warm standby, EU/NA fleets, RPO <=5 m/RTO <=30 m; signed result annotation/reconciliation preserves delivery outages Restore/failover meet targets; live simulation continues; valid delayed result commits exactly once after recovery
8.38 [D:7.7,8.26,8.36,8.37] Provider migration after Valve approves both providers' EU/NA POPs/certs and public UDP: restore, validate coordinator trust, switch allocations, drain old Both geographies complete Hosted-SDR matches on new provider and no old-provider live match is terminated

8E — Client experience and recovery

# Task Acceptance
8.39 [D:8.3,8.14,8.17] IN PROGRESS. MatchmakingState now projects queue → proposal → allocation/process-ready/assignment-ready/connect/live plus terminal failure states; autoload ControlPlaneClient provides authenticated queue create/recovery/heartbeat/cancel and proposal response requests with idempotency/revision headers; matchmaking.tscn/matchmaking.gd expose the state and authoritative actions from the main menu; authenticated proposal recovery now reconciles missed proposal events and expires them at read time; the Go API publishes targeted authenticated revisioned queue/proposal/assignment events and the Godot client consumes state/proposal/assignment events, fetching the authoritative assignment after assignment readiness; the API can now use the durable participant-scoped proposal provider for both recovery and accept/decline mutations test_matchmaking_state.gd, test_control_plane_client.gd, test_matchmaking_ui.gd, TestProposalRecoveryIsParticipantScopedAndExpiresAtReadBoundary, TestAuthenticatedWebSocketDeliversOnlyTargetedRevisionedEvents, TestStateChangingAPIActionsPublishTargetedEvents, TestProposalRecoveryUsesDurableBackendAndRemainsParticipantScoped and store proposal SQL tests reject stale/gapped/conflicting updates, validate endpoint/token/payload normalization, preserve idempotent duplicates, and guarantee visible phase/terminal copy; wiring durable allocation events into the outbox, wait/latency explanations and Godot runtime verification remain
8.40 [D:8.3,8.14] IN PROGRESS. Pure Go revisioned replica reducer rejects gaps for REST resync, makes duplicate/out-of-order events idempotent, and resumes from the authoritative snapshot revision; authenticated queue-ticket recovery now has an owner-checked REST read; Godot client projection persists non-secret ticket/proposal state, forces authoritative recovery after restart, and can replay a lost queue-create response with the original ticket/idempotency key; the Go API now exposes an authenticated /v1/events WebSocket with bounded per-player queues, strict upgrade/vocabulary validation and REST-resync-safe slow-client failure; Godot ControlPlaneClient can connect to the stream using Godot 4.7's handshake-header API, consume validated events, automatically reconnect with bounded backoff, and trigger REST recovery on projection gaps and stream return; OutboxDispatcher now provides ordered at-least-once delivery after durable commit; assignment recovery events carry the durable assignment revision without leaking it into REST JSON; prediction startup now distinguishes the sequence-0 warm-up acknowledgement from genuine missing/evicted history, preventing a false hard-snap during a live match server/domain/sync.go, server/api/events.go, server/api/service.go, server/store/outbox.go, service_test.go, outbox_test.go, matchmaking_state.gd, control_plane_client.gd, local_prediction_history.gd, net_ship_predictor.gd and tests cover gap, snapshot, replay, same-revision conflict, owner-only ticket recovery, expired-ticket terminal handling, malformed restart snapshots, API-level duplicate-create replay/conflict, authenticated handshake/key rejection, targeted event delivery, exactly-once slow-subscriber closure, invalid-event rejection, delivery-before-ack failure ordering, assignment event/response revision separation and prediction warm-up/hard-resync separation; Godot 4.7.1 headless project parse and 143-test unit harness pass with compatibility rendering, and a two-process ENet authoritative match smoke passes spawn, movement, prediction, 0% snapshot loss and 0 hard snaps; cmd/control-plane provides a signal-bound API role, cmd/matcher now supports casual and explicitly enabled ranked roles with durable identity lookup, and cmd/maintenance provides bounded season maintenance. Live multi-process control-plane/game verification now exists: scripts/verify_control_plane_integration.sh runs a real postgres:17-alpine, the real api.Service (via the new test-only server/cmd/testkit-api, wired identically to cmd/control-plane except for a fake Steam login — see §8.7), and a real headless Godot client (control_plane_smoke.gd) round-tripping login → fetch_ranked_profile (expect 404, §8.22) → queue_create → heartbeat → cancel_queue over an actual network connection — the first time this boundary was exercised end to end rather than against a mock on either side. It immediately found a real bug: matchmaking_state.gd's apply_ticket_update treated every same-revision confirmation right after begin_queue() as a conflict (comparing expires_at_unix, a field the client can't know in advance), so a real client would loop on recover_queue forever instead of ever settling into QUEUED — fixed and re-verified stable across 3 consecutive full runs, three times now (once per coverage addition). A two-player proposal round trip (needs a running matcher, not yet wired into testkit-api), allocator, and Redis fan-out live verification remain
8.41 [D:7.8,8.9,8.31,8.40] IN PROGRESS. Authenticated GET /v1/assignments/{matchId} now exposes only a validated, player-scoped assignment view including the hosted endpoint; Godot AssignmentState/ControlPlaneClient.fetch_assignment() bind the response to the authenticated player, recheck expiry, validate and retain the endpoint, and connect_to_assignment() now starts only the validated ENet/Steam transport after assignment readiness; the opaque join authorisation is carried in the MatchNet hello payload rather than the endpoint URL; allocated Godot servers now fail closed unless an operator-mounted JSON roster of control-plane signed envelopes and an HMAC-SHA256 key are present, and MatchNet verifies the canonical Go claim bytes, checks exact token membership plus match/server/protocol/slot/expiry claims before admitting a peer, rejects concurrent reuse of an active token, tracks server-owned reconnect generations across the 60-second reclaim window, fences expired reclaims and invalidates old peers; migration 0002 and the Go store adapter now persist/recover the complete player-scoped assignment projection with conflict-safe identical replay; AssignmentProviderFromStore wires that durable projection into the API injection point; SaveAssignments publishes a complete signed roster atomically instead of allowing partial player visibility; SaveVerifiedAssignmentRoster rechecks signed claims before deriving player rows server/api/service.go, store_adapters.go, service_test.go, assignment_state.gd, control_plane_client.gd, match_net.gd, server_boot.gd, server_config.gd, test_assignment_state.gd, test_control_plane_client.gd, test_match_net.gd, server/contracts/v1/openapi.json, server/migrations/0002_assignments.sql and server/store/assignment_sql.go cover participant/identity/expiry/shape/transport/endpoint boundaries, player-scoped schema keys, expiry-filtered reads, assignment upsert conflict handling, atomic batch validation, signed-claim binding, strict endpoint splitting, allowlisted claim rejection, canonical HMAC interoperability, forged-signature rejection, duplicate active-token rejection, reclaim generation/expiry behavior and 146-test Godot compatibility coverage; the three-process disconnect/reclaim smoke passes ship retention, replacement drive and old-peer invalidation; SDR relay-ticket installation, live allocated-token process integration and live Godot/PostgreSQL verification remain
8.42 [D:8.22,8.23,8.24,8.40] IN PROGRESS. RankedProfileState and ControlPlaneClient.fetch_ranked_profile() expose the backend-authoritative rating/RD/volatility/games/tier/provisional/season view; matchmaking UI displays provisional/tier status without client-side rating math test_control_plane_client.gd validates profile shape, numeric safety and provisional display; ranked profile fetch/display, committed revision after reconnect, abandon status and season countdown remain dependent on live auth/backend events and Godot runtime verification
8.43 [D:8.39,8.40,8.41] IN PROGRESS. Matchmaking client now distinguishes expired queue recovery, session expiry, missing records and retryable control-plane outages; a 401 clears the in-memory token, emits session_expired and disables retry until a new session is configured; terminal messages remain visible and active searches are not falsely failed on transient errors MatchmakingState and ControlPlaneClient tests cover explicit expiry and the existing terminal/retry-safe state paths; decline, version mismatch, regional outage retry UI, failed reconnect, duplicate-action recovery and live Godot verification remain

8F — Observability, verification, cost and rollout

# Task Acceptance
8.44 [D:8.3,8.4,8.28,8.31] IN PROGRESS. Go observability package encodes queue/proposal/match/server IDs and lifecycle stage in structured events while recursively redacting auth/relay tokens and credentials. It now actually emits: Service.Log is a nil-safe optional hook, wired into the two workload-authenticated server routes (register, result) at every outcome plus queue create/heartbeat/cancel and proposal accept/decline (state on success, rejected on a domain error, never the error text), and cmd/control-plane writes those events as JSON lines to stderr server/observability/ covers correlation fields, nested secret redaction and unnamed-event rejection; server/api/service.go/service_test.go cover the wiring plus a secret-canary test that drives the server routes with real-looking bearer-token/nonce values and asserts neither appears anywhere in what Log actually received (stronger than the unit test, which only proves a synthetic value under a denylisted key is stripped), and a lifecycle test asserting the exact event/id/stage sequence across a real create→heartbeat→cancel and an accept→stale-revision-reject. redact() is still key-name-based, not content-based — a field logged under an unlisted key would leak and neither test would catch it, only the discipline of never putting raw secret bytes into Fields; read-only routes (queue/proposal GET, assignment fetch), early availability/not-found rejections, and a real metrics/traces backend (this is stderr only) remain
8.45 [D:8.2,8.44] IN PROGRESS. Go observability package turns the documented RTT, allocation/connect latency, result-success, API-latency and tick/headroom thresholds into executable window checks server/observability/slo.go covers healthy/violating/empty windows; production metrics export, dashboards, alert routing, wait/MMR/proposal/flood/cost series and runbooks remain
8.46 [D:8.5,8.7,8.9,8.10,8.14,8.18,8.21,8.23,8.25] IN PROGRESS. Go unit/race coverage spans the current domain/store/supervisor policies, and fuzz targets now exercise queue input, result payload hashing and revision events server/domain/*_test.go, server/store/*_test.go, server/supervisor/*_test.go, server/migrations/*_test.go and server/domain/fuzz_test.go pass normal/race suites; go test -race ./... passes across API, domain, migrations, observability, store, supervisor and testkit; go vet ./... passes; each of the three declared domain fuzz targets passes a bounded 4-second run; PostgreSQL live migration execution now runs clean (§8.5), and four real-concurrency cases are covered against a live database with -race: §8.14's queue-heartbeat revision race, §8.18's two-matcher contested-ticket race, §8.30's cross-allocator-replica capacity race, and §8.21/§8.25's concurrent identical-result-submission race; the "lost Redis" fixture is covered live against a real server (§8.14: real TTL expiry, repair-after-FLUSHALL; fake Steam/allocator fixtures are §8.47's testkit, already done). Further transaction fixtures (e.g. concurrent proposal-recovery expiry races, live Redis failover mid-write under load) remain
8.47 [D:8.7,8.30] IN PROGRESS. Offline testkit provides deterministic fake Steam verification and fake allocation with forced failure injection server/testkit/ covers verified identity/replay, unknown identity, wrong App ID, expiry, no capacity, compatibility-key conflict, idempotent allocation replay and cloud-free forced allocation failure in TestOfflineFakesCoverVerificationAndAllocationFailureMatrix; API/Compose integration and live exhaustive matrix remain
8.48 [D:8.10,8.14,8.17,8.18,8.27,8.31,8.35,8.47] IN PROGRESS. Offline testkit exercises verified queue projection → ranked six-player proposal → ENet allocation → assignment-ready manifest → certified durable result receipt server/testkit/pipeline_test.go covers the cross-domain success path without Steam/cloud secrets; independent Compose fixture, process shutdown, result ack over HTTP and legacy fixture non-regression remain
8.49 [D:8.25,8.26,8.28,8.29,8.30,8.31,8.35,8.36] Disposable kind + Agones integration gate CI covers dynamic ports, both readiness stages, roster/no-show, races, multi-match node, result-pending reconciliation, drain and rollback
8.50 [D:8.25,8.37,8.43,8.49] Network/chaos suite: 100 ms RTT, jitter/loss, client/API/matcher restart, game-pod death, node drain, Redis failover and control-plane loss System recovers to a defined state; infrastructure-caused cases cannot penalise affected players
8.51 [D:8.17,8.18,8.30,8.31,8.45] Load test >=10,000 queued clients, >=100 proposals/s and forecast launch concurrency x2 API p95 <=250 ms, durable matcher fence holds, both readiness/allocation SLOs are met and replicas scale without duplicate claims
8.52 [D:8.32,8.34,8.45,8.51] Per-region cost model from measured density, warm capacity, bandwidth, DB/Redis and telemetry; add budgets and allocation quotas Cost per completed match and forecast monthly bands are recorded; a denial-of-wallet test triggers limits/alerts before budget breach
8.53 [D:7.8,8.13,8.38,8.45,8.46,8.48,8.49,8.50,8.51,8.52] Progressive release: development → internal → casual canary → casual → provisional ranked → ranked Each promotion requires SLO/security/cost gates, rollback rehearsal, EU+NA playtests and unchanged legacy gates; rollback criteria and owner are explicit

Implementation invariants for every task above:

  • Matchmade mode is opt-in; every new ServerConfig default preserves the existing community-server path.
  • compose.phase6-smoke.yml, make verify-phase6, and make verify-enet-integration are not repurposed or weakened.
  • Production uses ticketed Hosted Dedicated Server SDR; ENet remains the deterministic local/CI and direct-IP path.
  • One process serves one match. Warm processes/nodes absorb startup variance; capacity and cost are determined from 8.34 measurements, not old estimates.
  • Implementation evidence is appended under the completed task as in earlier phases; design changes first update docs/MATCHMAKING.md and dependencies.

8. What needs refactoring, not extending

# Location Why extension is insufficient
1 objects/ship.tscn, ship.gd:175-180, 189-208, 241-278 No node exists to carry a render-only offset — meshes hang directly off the RigidBody3D. Needs $Visual.
2 ship_camera.gd:115, 149, 150 Camera reads the body's transform, so it would jump the full correction error while the mesh smoothly lags.
3 match_mode.gd:36, 59-64, 76-82, 93-96, 107-109 The Timer + _process clock is frame-rate and time_scale coupled. Must become tick-derived. Five call sites.
4 match_mode.gd:162-171 get_tree().paused = true stops the client's own send loop and snapshot processing, and the return-to-lobby RPC lands in a tree that cannot act on it.
5 game_mode.gd:95-121, 171-194 Engine.time_scale is fundamentally incompatible with a shared tick clock — sequence numbers ride on Engine.get_physics_frames(), so a hit-stop at 0.06 starves the jitter buffer within a few frames. The effects must be reimplemented, not merely disabled.
6 game_mode.gd:85-92 _handle_goal_scored interleaves timing with presentation. On a headless server _play_goal_celebration returns synchronously, so the reset fires on the same frame as the goal — while clients are 1.6 s into a cinematic.
7 game_mode.gd:248-263 _jittered uses global RNG; _reset_body uses set_deferred. Both must become authoritative-broadcast plus a Jolt-correct teleport.
8 game_mode.gd:54-55, 284-285 Unconditional goal-signal connection (an interpolated ball entering a client's local Goal would score locally) and unconditional escape-respawn both write authoritative state on clients.
9 main_menu.gd (all handlers) Every mode launch is a synchronous change_scene_to_file. Connecting is async and can fail — a genuinely new UI state, not another button.
10 HUDController.gd:41-46 Hard-requires a ship; spectators have none.
11 player_ship_controller.gd Single reused ShipAction instance; buffering aliases every history entry.
12 ship_camera.gd:86 (whole rig) Runs in _physics_process, so on a 240 Hz display the FOV kick (:182) and PostFX parameters (:186-187) step at 60 Hz — neither is a transform, so global physics interpolation does not cover them — and the shake noise (:200-212) loses its high-frequency character. Must become _process + get_global_transform_interpolated() (§5.4a, task 0.16).
13 video_settings.gd:14-16, settings_menu.gd Persists AA, glow and brightness only — three values. The three genuinely expensive settings (SDFGI, SSIL, SSAO) and the five shadow-casting lights are unreachable, and neither vsync_mode nor max_fps is set anywhere. A player chasing 240 fps has exactly one lever: turn glow off. Needs a preset system, not another checkbox (§5.5, tasks 0.17/0.17b).
14 scenes/arena_base.tscn:18-50, 61-105 The Environment every arena inherits enables SDFGI + SSIL + SSAO + a 5-level glow pyramid simultaneously, with four shadow-casting OmniLight3Ds (24 cubemap faces/frame). Not tunable per-arena around a preset; the preset must gate the shared base (§5.5).
15 shaders/post_process.gdshader:4 hint_screen_texture forces a full-screen backbuffer copy every frame, not only during turbo — vignette_strength never reaches 0 (ship_camera.gd:187, 243). Either bake the static vignette into Environment.adjustment_* and hide PostProcess when chromatic_aberration is at rest, or drop the screen read for a plain gradient overlay and keep it only for the turbo chroma.
16 project.godot [display] stretch/mode="viewport" + 1920×1080 base + aspect="expand" fixes the 3D render at ~1080p and blits. A 4K player cannot render native; a 1080p player cannot render lower. Blocks any render-scaling setting until decided (task 0.17c).

On Engine.time_scale: replace hit-stop and goal slow-mo with the camera-based effects in single-player as well (task 0.12), so there is one code path and one game feel to maintain rather than a networked variant that drifts away from the single-player one. ShipCameraRig already has _shake_strength, shake_decay, max_shake_offset and a PostFX ShaderMaterial to build on.

What does not need surgery: the ShipController seam, the Arena/GameMode split, code-driven spawning, group-based discovery, and the dumb Goal sensor all extend cleanly. CLAUDE.md's claim about the three load-bearing seams is accurate — they hold. rl_ship_controller.gd is already the remote-input controller (a public action field that something else writes, pulled each tick), so no new class is needed for it.


9. Godot 4.7 + Jolt gotchas

  1. ENetMultiplayerPeer.server_relay defaults to true — clients can RPC each other through your server. Set it false.
  2. MultiplayerAPI.poll() runs on the idle frame, so an rpc() from _physics_process waits up to a full frame — and Engine.max_fps = 60 on the server is what creates that delay on the return leg. Take manual control (task 1.3). ~1633 ms of round-trip, for ~10 lines.
  3. Jolt sleeps bodies. A ship corrected to near-zero velocity can sleep and then ignore state.linear_velocity writes. can_sleep = false on Ship and Ball.
  4. Teleporting a rigid body: state.transform inside _integrate_forces is the only path with no frame of lag. set_deferred("global_transform", …) lands between frames and interacts badly with Jolt's sleep/wake ordering.
  5. reset_physics_interpolation() is not automatic for state.transform writes (it is when you set global_transform directly). Call it explicitly, on the body and on $Visual.
  6. physics_jitter_fix = 0.0 does not give you "a flat 60 Hz." You still get occasional 0-tick and 2-tick frames, because frame time is never exactly 16.667 ms. The real reason to set it to 0 is that you never want a tick's input delayed by the accumulator smoother. The send path must therefore transmit both ticks' actions on a 2-tick frame — redundancy-4 covers this, but only if you actually send both.
  7. _integrate_forces is not called on frozen bodies, so remote ships never pull get_action() — hence set_visual_action. Use FREEZE_MODE_KINEMATIC, not STATIC, or contact velocity transfer breaks.
  8. Never write linear_velocity to a frozen body — Godot/Jolt zeroes and holds it.
  9. Engine.max_physics_steps_per_frame defaults to 8. If a server tick overruns 16.7 ms the accumulator backs up and the next frame runs multiple ticks, spiking CPU further. Log overruns (task 1.6).
  10. ENet channel indices are offset by Godot's reserved system channels — verify the mapping empirically.
  11. ENet peer timeout defaults to ~5 s. Tune via ENetPacketPeer.set_timeout() for faster drop detection.
  12. Jolt is not bit-deterministic across platforms or across differing contact orderings. Never rely on it anywhere, including in "obviously safe" places like a client-side goal check.
  13. dedicated_server=true exports strip visual resources. Verify against a real stripped build (task 6.2).
  14. MTU: ENet fragments above ~1400 B. At 219 B/snapshot there is ~6× headroom; recheck if per-body cosmetic state is ever added.
  15. RPC NodePath caching: the first rpc() to a node sends the full path, later calls send a cached int. Routing hot paths through autoloads warms the cache once at connect and never invalidates it on scene change.
  16. Physics tick rate is 60 for v1 — and must never be a literal. Every policy in Game/bots/ is tick-coupled through ship.gd:450's _tick_scaled (defined at a 60 Hz reference) and ai_ship_controller.gd's reaction_ticks, so raising it toward Rocket League's 120 invalidates every trained model and halves server density. But it is the largest single latency term left (§5.4), so it will be revisited: derive everything from TICK_HZ (tasks 0.18, 1.1) so that day is a config change plus a retrain.
  17. Node3D.get_global_transform_interpolated() is the only correct way to track a physics-interpolated body from _process. global_transform returns the last physics tick's pose, so a per-frame camera reading it chases a 60 Hz staircase. Per the engine docs the method "creates an interpolation pump… the first time it is called" — call it once before any reset_physics_interpolation() on that node, or the first hard snap streaks (§4.5).
  18. Physics interpolation covers transforms only. camera.fov, shader parameters, light energy and anything else written from _physics_process steps at 60 Hz on a 240 Hz display. Either write them from _process or accept the stepping deliberately.
  19. display/window/vsync_mode defaults to enabled (FIFO) and max_fps to uncapped. Neither is set in project.godot. FIFO present latency is 1.53 refresh intervals depending on swapchain image count (2 vs 3) and whether the present queue is full — §5's tables use the optimistic 1.5, which assumes the renderer is not GPU-bound. The model does not hold below refresh, where a missed vblank under strict FIFO halves the effective rate and roughly doubles present latency. Prefer Adaptive as the default, not Mailbox (§5.4). (Swapchain image count per platform needs empirical verification.)
  20. Engine.max_fps is a throttle, not a frame pacer. It pads each frame with a post-frame sleep; it has no vblank phase lock. Caps that are not integer divisors of the refresh rate beat against scanout, and combining a cap with an active vsync paces worse than either alone (§5.4). Derive the offered caps from DisplayServer.screen_get_refresh_rate().
  21. DisplayServer.window_get_vsync_mode() echoes your request, not the driver's grant. There is no GDScript API for the negotiated VkPresentModeKHR, so a UI cannot honestly report what was applied. Show a live fps readout instead and let the player infer it.
  22. Engine.max_physics_steps_per_frame = 8 is a client problem too, not just a server one (gotcha 9). A client hitching to 20 fps runs 3 ticks per frame, and each of those frames also runs the per-frame camera rig and remote-visual sampling. Set it to 4 client-side (task 0.22). On a multi-tick frame the send path must transmit every tick's action (gotcha 6) — §4.3's _physics_process sampling does this naturally, but nothing else guarantees it.
  23. hint_screen_texture forces a full-screen backbuffer copy on every frame the node is drawn, regardless of what the shader then does with it. Branching inside the shader saves taps, not the copy. Hide the node when the effect is at rest.
  24. physics_jitter_fix matters less the higher the frame rate. Its purpose is smoothing when frame rate ≈ tick rate; at 240 fps against 60 Hz physics most frames run zero ticks and the accumulator is never near an edge. Gotcha 6's reasoning for setting it to 0.0 still holds, but do not expect a visible difference on a high-refresh machine — test that change at 60 fps.
  25. MultiplayerAPI.multiplayer_peer's default value is an OfflineMultiplayerPeer sentinel, not null. Resetting it with multiplayer_peer = null (rather than a fresh OfflineMultiplayerPeer.new()) leaves the API in a state distinct from its own default and is a known source of "the server never sees peer_connected, get_peers() stays empty" bugs (godotengine/godot#81540) — confirmed the hard way while building task 1.2's NetworkManager.shutdown(). Always reset to a real OfflineMultiplayerPeer.
  26. Don't tear down a peer the instant its own connect signal fires. connected_to_server (client-side) fires once the client's local view of the handshake completes, but the final ACK the server needs to consider its side complete may not have hit the wire yet — closing the peer or quitting the process in the same callback can drop it, and the other side then never sees peer_connected/connected_to_server at all, even though your own side looked successful. This isn't a corner case: it reproduced on every attempt until fixed, is easy to misdiagnose as a server-side bug (the server-side symptom — get_peers() staying empty — is identical to gotcha 25's), and cost significant debugging time before the actual cause (client-side premature teardown) was found. Give at least one frame — in practice tests/net_smoke.gd uses 0.3 s — between a fresh connect signal and calling shutdown()/quit(). Directly relevant to task 5.6's disconnect/reconnect controller swap and any CLI test client that connects, asserts, and exits quickly.
  27. change_scene_to_file() must be called on (or from a descendant of) the actual get_tree().current_scene, and never synchronously from _ready(). Both failure modes were hit building task 1.5's lobby.tscn/tests/lobby_smoke.gd: (a) a test harness that instantiated lobby.tscn as a plain child of a driver node — rather than loading it as the real current scene, the way main_menu.gd's Host/Join flow will — caused lobby.gd's own (entirely correct, standard-pattern) change_scene_to_file(ScenePaths.MAIN_MENU) disconnect handler to hang the process completely on a real disconnect, with near-zero CPU (blocked, not spinning) and no error output; the fix was to load the scene the way production actually will, not to change the production code. (b) calling change_scene_to_file() (or add_child() on get_tree().root) synchronously from inside _ready() throws "Parent node is busy … Consider using .call_deferred()", because the tree is still mid-traversal adding the very node whose _ready() is running; main_menu.gd's real button-press handlers won't hit this (they run outside any _ready()), but anything that needs to trigger a scene change during its own initialization must .call_deferred() it.
  28. ENetMultiplayerPeer's connection_failed signal is not bounded to anything a UI should make a player wait for. Verified empirically (task 1.7): against a genuinely refused loopback connection (nothing listening on the target port), connection_failed had still not fired 14 seconds in. Don't rely on it alone to end a "Connecting…" state — run your own app-level timeout (main_menu.gd's CONNECT_TIMEOUT_SECONDS = 6.0) that shuts the peer down and shows an error regardless of whether ENet ever gets around to reporting failure itself.
  29. A MultiplayerPeer's "am I a client" flag (however you track it — NetworkManager.is_client here) turns true the instant join()/create_client() is called, not once the connection actually completes. Anything gated on that flag alone (task 1.8's clock ping, in network_manager.gd's _process) will try to rpc_id() on a peer that's still CONNECTING — or has already failed — during a slow or refused connect attempt, and Godot logs "Trying to call an RPC via a multiplayer peer which is not connected" every single frame until it resolves. Gate on the peer's actual get_connection_status() == MultiplayerPeer.CONNECTION_CONNECTED, not just the higher-level intent flag.
  30. load() on a .gd file with a parse/compile error does not return null. Found via adversarial review of tests/test_runner.gd: it returns a non-null but uninstantiable GDScript resource, so if script == null silently fails to catch the failure — and the natural next line, script.new(), throws "Invalid call: Nonexistent function 'new'", severe enough to abort the entire calling function (not just that statement) without ever reaching whatever cleanup/exit code follows. In a loop over multiple files with no per-iteration error boundary, this reads as a hang: the loop that would have moved to the next file, and the code that would have called quit(), both never run. The real guard is Script.can_instantiate().
  31. An @rpc method named _input collides with Node's built-in _input(event: InputEvent) virtual. Found building task 2.1's MatchSim autoload: naming the client→server input RPC _input(bytes: PackedByteArray) produced a parse error ("function signature doesn't match the parent") — and because this was on an autoload, the error broke the entire autoload from loading, cascading into unrelated failures across every scene that touched MatchSim at all, none of which mentioned RPCs or _input in their own error output. Renamed to _recv_input. General lesson: on an autoload especially, treat any bare virtual-sounding method name (_input, _process, _ready, _unhandled_input, …) as reserved regardless of what you intend it to do — a signature mismatch there doesn't fail locally, it fails the whole autoload.
  32. Disabling automatic multiplayer polling (task 1.3) is global, not autoload-scoped — every scene that touches an RPC, not just NetworkManager-adjacent code, must call NetworkManager.poll() itself every frame it wants traffic to move. Building task 2.12.3, networked_match.gd's _physics_process/_process sent and listened for RPCs (MatchSim.request_match_config, send_input, snapshot RPCs) but never called poll() — nothing sent via rpc() in this scene ever reached the wire in either direction, silently, with no error in either process's log. Confirmed via debug prints: the client's request fired, but the host's handler print never appeared. The first (wrong) hypothesis was a startup race between the server's broadcast and the client's listener connecting — that fix (a request/response retry pattern, still worth keeping for the genuine late-join case) didn't resolve it alone. The real fix was adding NetworkManager.poll() at the top of both _physics_process and _process in the new scene. If a scene sends or receives RPCs and nothing arrives with no errors at all, check for a missing poll() before anything else.
  33. A request/response fallback for a one-shot broadcast can double-deliver, and the receiving handler must be idempotent. Once gotcha 32's fix made polling actually work, _on_match_config_received ran twice per client — once from the server's original one-shot _match_config.rpc() broadcast (queued the whole time, since it had been sent before polling was fixed) and again from the request/response retry — producing two arenas, two ship sets, two HUDs (_slots.size() == 2 instead of 1). Any handler reachable via both an original broadcast and a "resend on request" path needs its own guard (here: if not _slots.is_empty(): return at the top) rather than assuming "only sent once" from the RPC design alone.
  34. An Area3D's body_entered signal fires as part of physics tick N's own step, strictly before tick N's _physics_process callback — not "on the next frame." Found while fixing the goal-reset-ordering bug above: a boolean "handle this on the next _physics_process" flag set from inside a body_entered handler is a no-op, because that same tick's _physics_process hasn't run yet and sees the flag already true — it "defers" to the same tick it was set on, not the next one. If you actually need next-tick-or-later semantics, compare Engine.get_physics_frames() against the tick the flag was set on and require strictly-greater, not just "check a boolean at the top of _physics_process."
  35. A queued queue_teleport() (task 0.15) can take one tick longer to land than "the very next _integrate_forces" suggests, when the call originates from a signal handler mid-physics-step rather than from a _physics_process callback. Empirically confirmed by teleporting a body into a goal and logging the server's own per-tick broadcast: the goal was detected on tick N (per gotcha 34, during tick N's own step), but the reset position didn't appear in a broadcast until tick N+1's, one tick later than "queued during N, applied on N+1's _integrate_forces" alone would predict. Don't assume queued-teleport timing without checking a real tick-by-tick log for your specific call site — the exact tick it lands on depends on where in the physics step the queuing call happens, not just "next frame" intuition.
  36. NetworkManager.get_server_time_estimate_ms() (and anything derived from it) is not just imprecise before the first clock pong lands — it's actively wrong in a way that can persist far longer than the sync window implies. clock_offset_ms is 0.0 until the first pong, so a value derived from get_server_time_estimate_ms() during that window means "my own raw process uptime," not a server-synced estimate — and if that value feeds a rolling-window filter (e.g. a min-tracked bias, per the interpolator epoch-bias fix in Phase 2's adversarial review), the bad early sample can dominate the window for the filter's entire configured duration if a short test or a short match doesn't run long enough for real time to age it out. Always gate recording, not just consuming, anything derived from this estimate on rtt_ms >= 0.0.
  37. Anything that deliberately delays an RPC dispatch (task 2.8's net_sim.gd) must re-validate its target at fire time, not just at the moment it was scheduled. Found by actually running Phase 2's own gate (networked_match_smoke under --net-sim-latency=80 --net-sim-jitter=20), not the isolated ping/pong test alone: _broadcast_snapshot's existing get_peers() filter (gotcha from task 2.2's own fix) only proves the target was valid when the send was queued — a target that legitimately disconnects during the ~80100ms hold produces "Attempt to call RPC with unknown peer ID" anyway, and if the delay outlives this process's own shutdown(), multiplayer_peer has already been reset to a fresh OfflineMultiplayerPeer (§9 gotcha re: never resetting to raw null), so a stale rpc_id(1, …) now means "call yourself" and Godot rejects it. Any deliberate-delay layer needs to re-check both "do I still have a real peer at all" and "is this specific target still in get_peers()" inside the delayed callback itself, immediately before dispatching — not reuse whatever validation the caller did before the delay was added.
  38. A GDScript lambda captures an enclosing local variable BY VALUE at the moment the lambda is created, not by reference. Bit two separate Phase 3 test scripts the same way: var disconnected := false; some_signal.connect(func(): disconnected = true) compiles and runs with no error or warning, but the assignment inside the lambda mutates only that lambda's own captured copy — the enclosing function's disconnected stays false forever, even after the signal genuinely fires (confirmed firing via an extra debug print before the real cause was found). The underlying disconnect-detection code was correct the whole time; only the test's own assertion logic was broken. The fix is to capture a container instead of a value — var disconnected := [false] and disconnected[0] = true inside the lambda — since capturing an Array/Dictionary/Object captures a reference to the same instance, and mutating its contents from inside the lambda is visible outside it. Relevant anywhere a lambda is used to flip a flag or accumulate a result for a caller to read later (a connect(func(): ...) one-liner is the single most common place this bites).
  39. A fixed-size ring buffer fed by an unbounded-rate producer needs an explicit resync path, not just "wait for the next expected slot." InputJitterBuffer's 32-entry ring assumed the consumer (consume(), one call per server physics tick) would never fall more than RING_SIZE ticks behind the producer (ingest(), driven by real wall-clock packet arrival, unrelated to the consumer's own tick rate) — but a server-side stall, or even ordinary client/server clock drift with zero external trigger, breaks that assumption, and once broken, a design that only ever advances its "expected" pointer by exactly one per call can never catch up: newer arrivals silently overwrite the exact slot still being waited on, and the wait never ends. If a ring's producer and consumer rates aren't provably bounded relative to each other, the consumer needs a way to detect "the data I'm waiting for no longer exists in the ring at all" (track the newest value ever seen, independent of ring capacity) and jump directly to what's still available, rather than assuming "keep waiting" is always eventually correct.
  40. A client-owned adaptive control loop must react to the actual ground-truth signal it's regulating, not to its own memory of past decisions. InputLeadController's release logic was gated on lead > LEAD_MIN — a count of the controller's own past attacks — rather than on the real server-reported input_buffer_depth it exists to keep near target. Any elevated depth the controller didn't itself cause (an external stall, drift, a burst redelivery) was invisible to that gate and so never got drained, even while the "real" signal sat well above target the whole time. When a control loop's condition for acting can be satisfied or blocked by state the loop itself controls, rather than by the environment it's meant to respond to, it can silently stop responding to the environment.
  41. A "consecutive N over-budget windows" streak counter that hard-resets to 0 on any single clean window is trivially evaded by a duty-cycled attacker (burst hard, one clean window, repeat) — confirmed sustaining ~33x a stated packet budget indefinitely with zero disconnect warnings. A leaky-bucket accumulator (grows by each window's actual total, drains by exactly one window's worth of budget every window, disconnect once the accumulated excess crosses a threshold) is immune to the same evasion by construction, since it doesn't matter how the excess is distributed in time — only the sustained average matters.
  42. Two counters that don't share an epoch must never be compared directly, even when both are monotonically increasing integers that "look like" the same kind of thing. seq > Engine.get_physics_frames() + 20 compiled, ran, and looked like a sane bound — but Engine.get_physics_frames() counts from the SERVER PROCESS's own start while a client's _input_seq starts at 0 when ITS match scene loads, so the check either never fires (on a long-running server, no real protection despite its own comment's claim) or fires wrongly and silently drops an honest client's input forever, depending entirely on how much unrelated head-start or drift has accumulated between the two clocks. Bound a value against another value that shares its own actual epoch (here: the receiving buffer's own last_applied_seq), not against a same-typed number from a conceptually different clock.
  43. A regression test that doesn't independently exercise the specific mechanism it claims to gate will pass even when that mechanism is completely broken. Task 3.6's CI driver asserted snapshot throughput and a server-forced goal's score agreement — neither of which depends on client input ever reaching the server — and kept reporting PASS with a real, reproduced bug (§7's ring-overflow) actively zeroing both bots' input for the whole run. A CI gate's assertions should trace back to the specific claim in the task's own acceptance text, not just "the match ran and didn't crash."
  44. When adding a "is this connection still healthy" check to a test, sample it while the peer is still actively connected — not after its own end-of-run disconnect, which produces symptoms indistinguishable from the bug being checked for. Fixing gotcha 43 first sampled InputJitterBuffer.stalled and ship movement after the full test run (plus a buffer for score-file writes), which meant both readings came from ~4s after the bot had already legitimately shut down — a departed peer's input naturally starves and goes stalled=true too, and that's correct, expected behaviour, not the bug. Move the check to a moment still comfortably inside the peer's own active connection window.
  45. Two fixes landed in the same commit, each individually correct in isolation, can share a variable and silently cancel each other out — and a fix's own unit test can miss it by testing the mechanism in isolation from the thing that defeats it. Gotcha 39's resync fix and gotcha 42's guard rebound were reviewed, tested, and verified independently, each against its own scenario, both passing. Combined, the guard caps the exact variable (highest_ingested_seq) the resync's own trigger condition depends on, making it permanently unreachable — recreating the original critical bug at a lower failure threshold than before either fix existed. The resync's own new unit test called InputJitterBuffer.ingest() directly, which is correct in isolation but bypasses the guard entirely, so it could never have caught this regardless of how thorough it was on its own terms. When two fixes in the same round touch the same subsystem, explicitly re-test the combination end-to-end (here: a real SIGSTOP freeze against the actual production RPC call path, not a direct unit-level call into the class the fix lives in) — passing tests for each fix individually is not evidence the pair composes correctly.
  46. A guard that bounds an incoming value against the consumer's position, rather than against the producer's own epoch, re-introduces exactly the "consumer can never catch up past a stall" failure it's often added specifically to prevent. The seq-range guard bounded seq against last_applied_seq (advanced only by consume(), i.e. gated on however fast the physics tick loop is actually running) rather than highest_ingested_seq (advanced by ingest(), i.e. gated on however fast packets are actually arriving and being processed by poll()) — during a stall where ticks fall behind but polling keeps pace (the common case: a single-frame hitch, or Engine.max_physics_steps_per_frame capping tick catch-up while poll() itself isn't similarly capped), bounding against the lagging consumer rejects the very packets that would let the buffer refill and the resync condition ever trigger. Bound against whichever side of a producer/consumer pair is not the one already known to be falling behind.
  47. A trace that holds its inputs steady cannot falsify anything about which sequence a prediction is filed under — and "we hold thrust for 60 seconds" describes almost every prediction test people write. Phase 4's history was filed under the wrong sequence (the estimated server-consumption seq, input_lead ticks behind issuance, instead of the issuing seq), and the dedicated action-marker instrument built to catch exactly that reported a flawless marker=0/3784 across 60-second LAN, 80±20 ms and 5%-loss runs. It was not broken: while the commanded action is constant, "the intent from this tick" and "the action the server consumes for seq S" hold the same value, so a right and a wrong label are indistinguishable. Only an input edge separates them, and only for about input_lead ticks per edge. The bug then scales with input_lead — 9.3% mismatch at lead 1, 24% at lead 3 — meaning it was worst precisely on the impaired links the test matrix existed to cover, and invisible in all of them. When a test is meant to validate a label, an index, or a phase relationship rather than a magnitude, the trace has to change that quantity frequently; a steady-state trace validates the magnitude and silently asserts nothing about the label.
  48. A guard whose bound is derived from a value only the ACCEPTED path can advance is a latch, not a guard. The seq-range check has now been written three times — bounded against server uptime, then last_applied_seq, then highest_ingested_seq — and all three could permanently reject an honest client's input, because in every version the quantity being compared against could only move forward via a packet that got through. Once enough drift or loss accumulated, nothing could ever move it again. The property to check when writing a guard like this is not "is the bound correct?" but "if this guard rejects everything from now on, what advances the bound?" If the answer is "an accepted packet", it needs an independent escape path (here: resync after N consecutive rejections) regardless of how well-chosen the bound is.
  49. Advancing a consumer cursor past data that has not arrived is not a lossy shortcut — it is permanent, because the producer-side filter then rejects the very data being waited for. InputJitterBuffer.consume() advanced last_applied_seq on a starve, and ingest() discards seq <= last_applied_seq. One starve on a sequence the client had not sent yet therefore stranded the stream one ahead of arrivals forever — both sides advancing in lockstep, the gap never closing, every packet discarded on arrival. The client's own routine input_lead release was enough to trigger it, roughly every 6.5 s on a clean LAN. Only give up on an expected item once strictly newer data proves it lost; "it hasn't arrived yet" and "it will never arrive" are different states and must not share a code path.
  50. A metric that stops sampling during a failure will report that failure as healthy. The action-marker gate printed SMOKE PASS at 3.76 % on a run where the player's input was permanently dead — because reconciliation suppression stops _record_metrics being called, so the worse the outage, the fewer samples and the lower the computed mismatch rate. Every rate-shaped assertion needs a companion assertion on the denominator (here: a sample count scaled to run length), or an outage silently becomes an absence of evidence and then evidence of absence.
  51. An architectural blocker inherited from a previous session is a claim to verify, not a premise to build on. Phase 4 was handed over blocked on approval for a client-only shadow Jolt world — a large subsystem, and effectively the whole-world rollback §1's locked decisions rule out. The actual same-sequence defect turned out to be a one-line mislabel, falsifiable in about an hour with instrumentation that already existed; the shadow world remains genuinely necessary for the contact cohort but nothing else, which is a far smaller commitment than "Phase 4 is blocked on it." Reconstruct the failing invariant from the code and reproduce it against a control before accepting a scope estimate attached to it — especially when the recommendation arrives without the cheaper alternative recorded as tested.

10. Testing

Editor. Debug → Run Multiple Instances, 23 instances with per-instance args (-- --server, -- --connect 127.0.0.1:27015) and --position so windows don't stack.

CLI.

godot --headless --path Game res://scenes/server_boot.tscn -- --port 27015 --team-size 1 --auto-start
godot --path Game -- --connect 127.0.0.1:27015 --name Alice

CI smoke test (task 3.6). Headless server plus two headless --test-bot clients, driven by the existing AIShipController. Asserts:

  • snapshots received ≥ N * snapshot_hz * 0.9
  • own-ship prediction error p95 < 0.5 m, p99 < 2.0 m, hard-snap count < 3
  • final score identical on the server and both clients
  • no push_error emitted (scrape stderr)

Network conditions. net_sim.gd (task 2.8) is first-class: seeded so failures reproduce, works in CI, needs no display, and can be applied asymmetrically — which OS tools make painful. tc netem / Network Link Conditioner / clumsy for a pre-release realism pass only. A real remote host once per phase from Phase 4 onward is the only true test of the jitter buffer's adaptivity.

Unit tests (task 1.0). No test framework exists today, so keep it minimal — a scene that runs pure-function assertions and exits with a code. High-value targets, all zero-engine-state: codec quantise/dequantise round-trip and bounds; quaternion max error; snapshot pack→unpack identity; input packet framing; jitter-buffer policy against scripted arrival traces; ShipAction.copy() non-aliasing. These are exactly where a bug is invisible in play and catastrophic in aggregate.


11. Flagged, not solved

Slot reservation and takeover are keyed on display name alone — item C of §0, and the only open item here with a security character. _try_reclaim_slot matches a joining peer against a departed slot on slot.player_name == player_name and nothing else. There is no secret, no token, and no uniqueness constraint on names anywhere in MatchNet, so any peer that connects during the 30 s reservation window using a departed player's display name is handed their slot, their ship (mid-flight, at whatever pose it holds), and their team. Demonstrated with a real three-process run, not reasoned about. §6.3's late-joiner queue inherits the same weakness for the name it records, though the queue itself is ordered by arrival and cannot be jumped, so the reservation reclaim is the exploitable path.

Bounded, but not by much: the attacker must race a genuine disconnect, and they must know the name — which is displayed to everyone in the lobby. The right fix is the one §6.2 step 1 already specifies and Phase 7 already schedules: hello carries an auth_ticket, and the reservation is keyed to the resulting verified identity rather than to a string the client chooses. Building a bespoke token now would be inventing half of task 7.4 and then throwing it away, so this is deliberately left for that task — with the consequence stated plainly: this build must not be exposed to strangers before 7.4 lands, and it is a listed precondition of Phase 6's "connect from another machine over the internet" gate rather than a footnote to it.

Low-latency present and graphics presetsnow specified, see §5.4, §5.5 and tasks 0.17/0.17b. Left here as a pointer because they are the largest wins in the document per line of code changed, and they are video settings rather than netcode.

120 Hz simulation — deliberately deferred, not dismissed. §5.4 and §5.6 record what it would buy (≈21 ms of world response once L1 has taken the interpolation buffer out, plus ≈8 ms of own-ship feel — the difference between ≈127 ms and ≈107 ms), what it costs (a full bot retrain, half the server density, double the bandwidth), and the one rule that keeps the door open: TICK_HZ, never 60.

The latency gap to the reference has a plan but not yet a measurement. §5.2 lands at ≈174 ms as designed; §5.6 routes that to ≈127 ms (tasks 0.17d, 4.9) and ≈103 ms (tasks 4.10 plus 120 Hz simulation), against ~90110 ms for the reference class at the same RTT. Every figure in §5.6 is arithmetic on the budget, not a measurement — task 4.9's acceptance criterion exists to make it one. Beyond that the residual is RTT, which is a server-siting problem (§6) rather than a code one and is worth more than every remaining code lever combined.

Audio. TODO.md records that there is none. set_visual_action / set_visual_speed (task 0.14) is precisely where remote-ship engine audio will hang, and "ball feel" (task 4.6) is half auditory. Design those hooks with that in mind rather than retrofitting.

Split-screen. Tracked separately in TODO.md; unrelated to this effort, though the camera-outside-the-ship structure that enables it is the same structure this plan relies on.

A second, distinct source of the same "Unable to send packet on channel N, max channels: 0" stderr noise — item E of §0, in networked_match.gd's _broadcast_snapshot rather than match_net.gd's _remove_player. Only reproduced via the deliberately-adversarial client-abuse-malformed smoke role: _broadcast_snapshot's per-peer send races match_sim.gd's host-forced disconnect_peer() (the abuse-disconnect path) against the same tick's connected_peers.has(slot.peer_id) snapshot, the same general shape of race as the fixed site but on a different call path (a server-initiated forced disconnect, not a normal client-initiated one) and not currently known to be reachable from ordinary play. Left for a dedicated pass — not fixed under this round's time pressure, since the fixed site (gotcha 46's neighbor, the round-2 addendum above) was the one an adversarial review actually flagged as a "clean stderr" violation in the tests this project's own conventions rely on.