9 Commits

Author	SHA1	Message	Date
Josh Creek	cf73074e27	fix(multiplayer): resolve composition regression from second adversarial review ... A second adversarial review of the previous fix commit found two of its nine fixes silently defeated each other: the seq-range guard (fix for a MEDIUM epoch-mismatch finding) capped the exact variable the ring-overflow resync (fix for the original CRITICAL finding) depends on, making the resync unreachable in production and recreating permanent input death at a lower failure threshold, reachable via ordinary server tick loss alone. - CRITICAL: rebind the seq-range guard to InputJitterBuffer's own highest_ingested_seq (now public) instead of the consumer-side last_applied_seq, so it tracks the client's send epoch rather than a value that can lag arbitrarily far behind during a stall. - HIGH: InputLeadController's release logic still ANDed the old `lead > LEAD_MIN` gate onto the new depth-driven condition, so a backlog the controller never caused still couldn't drain. Split into two independent decisions: the seq-duplicate action follows real depth alone; lead's own bookkeeping separately never drops below its floor. - MEDIUM: widen the CI driver's movement/stalled sampling margin (run_seconds - 2.0, was - 0.5) and assert the peer is still in multiplayer.get_peers() at sample time, since the old margin let the check pass on residual starvation grace after a bot had already disconnected. - LOW: measure horizontal-only displacement in the human smoke test's movement check — the old 3D-distance bar was beatable by pure gravity settling with fully dead input. - LOW: fix a real "clean stderr" violation (match_net.gd broadcasting a departure notice to a peer whose ENet channels are already torn down, including a second peer disconnecting in the same poll batch) by deferring the notification to the next idle frame. - Wire the server's per-slot stalled bit into the client debug overlay for real — a prior commit message claimed this already reached the overlay when only the CI gate actually read it. Re-verified end-to-end against the real production RPC path (not just unit tests in isolation, which is how the composition bug got past the first round): a 2-bot CI match with a 1.5s host SIGSTOP freeze injected mid-run, well past the 0.6s threshold the review reproduced the bug at, now recovers cleanly on repeated runs with zero stderr noise.	2026-08-20 18:26:12 +01:00
Josh Creek	2325313ad2	fix(multiplayer): adversarial review fixes for Phase 3 ... An Opus subagent's adversarial review of Phase 3 found a critical, silent, permanent bug plus eight smaller real issues, all empirically verified with real two- and three-process runs: CRITICAL: InputJitterBuffer's 32-entry ring permanently bricked a player's input once the un-consumed backlog exceeded the ring's capacity - a fresh arrival would land in the exact slot consume() was still waiting on, and since both counters only ever advance, the gap never closed. Reproduced with a real SIGSTOP/SIGCONT host freeze: client movement dropped from ~26m to 0.00m at ~0.7s, worse under real loss (a lossy link lowered the fatal threshold to ~400ms), and reachable via ordinary clock drift with no external trigger at all. Fixed by tracking the highest seq ever ingested and having consume() jump directly to what the ring can still provide once the gap exceeds capacity, instead of starving through an unrecoverable span. Re-verified with a 3s freeze (well past the original threshold): full recovery. HIGH: InputLeadController's release logic was gated on its own past attacks (lead > LEAD_MIN) rather than the real server-reported depth, so a backlog it didn't itself cause was never drained. Fixed to gate on actual depth vs target. MEDIUM-HIGH: the rate limiter's "N consecutive over-budget seconds" streak hard-reset to 0 on any clean window, letting a duty-cycled flood (burst, one clean window, repeat) sustain ~33x budget indefinitely with zero warnings. Replaced with a leaky-bucket accumulator immune to the same evasion by construction. MEDIUM: the seq > server_tick + 20 guard compared two unrelated clock epochs (server process uptime vs. client's own from-zero seq numbering), so it never actually protected anything on a long-running server and could silently drop an honest client's input forever. Bound against the buffer's own last_applied_seq instead. MEDIUM: InputJitterBuffer.stalled was computed but never reached the wire - the one signal that would have made the ring-overflow bug visible anywhere. Now wired through _ship_to_net_body_state. MEDIUM: task 3.6's CI driver's assertions didn't depend on client input reaching the server at all, so it kept passing with the ring-overflow bug actively triggered. Added real ship-movement and non-stalled checks, sampled while bots are still connected (an initial attempt sampled after their own legitimate disconnect, which starves identically to the bug). LOW-MEDIUM: a lead change silently mislabelled _input_history's older entries, since the wire format has no per-entry seq field. Fixed by handling each delta case (ordinary/release/attack) on its own terms. LOW: bandwidth and snapshot-loss overlay metrics froze at their last value during a total outage instead of decaying - exactly when they matter most. Both now report honest post-outage values. LOW: a guard comment on NetworkManager._ping misdescribed the actual disconnect_peer() arguments in use. Corrected. New permanent regression tests: test_ring_overflow_resyncs_to_fresh_data _instead_of_starving_forever, test_release_drains_a_backlog_it_never_ caused_itself, and client-abuse-flood-dutycycle (reproduces the exact duty-cycle evasion). Full regression suite, including the net-sim-latency milestone gate, all abuse roles, and the CI driver, re-run clean after every fix.	2026-08-20 15:28:44 +01:00
Josh Creek	caa9f44ab6	feat(multiplayer): Phase 3 task 3.6 - --test-bot client mode + CI driver ... networked_match.gd's client can now swap its input sampler for a real AIShipController (--test-bot, optionally --test-bot-model=<path>, defaulting to bots/promoted/medium.json) instead of PlayerShipController. Unlike the human sampler, AIShipController needs real scene context (get_parent() as Ship, plus ball/teammate/opponent discovery via groups), so it's parented onto the client's own ship via Ship.set_controller() rather than left floating - and the field's static type widened from PlayerShipController to the shared ShipController base to allow either. Known, documented limitation: this client's ships are all FREEZE_MODE_KINEMATIC and driven purely by transform writes, so nothing ever writes linear_velocity/angular_velocity onto them - the bot's observations always see every ship as stationary. It still produces well-formed, bounded actions from that degraded input (the policy network's output layer is bounded regardless of input quality), which is sufficient for this task's actual job: generating realistic sustained network traffic for CI, not winning matches. New CI driver (tests/networked_match_ci.gd/.tscn): a headless server plus two headless --test-bot clients playing a real match. task 3.6's original acceptance text also named "p95/p99 prediction error" and "snap count" - both Phase 4 concepts that don't exist until client-side prediction and its hard-snap threshold are built, so asserting on them now would be fabricated. What's checked instead: snapshot throughput (500+ received over an 8s run, comfortably above a 60Hz-scaled floor), and genuine cross-peer score agreement - forced via a deterministic server-side goal (bot-vs-bot scoring isn't reliable enough within a short run to gate on), with each client independently writing its own final score to a peer-id- keyed file for the host to compare against the other bot's, not just trusting the server's own view. "Clean stderr" is left as the external invocation's job, same as every other smoke test in this project. Verified with real 3-process runs (host + two bots): both clients independently confirmed identical scores after a forced goal, both saw 500+ snapshots, and all three processes exited 0 with clean stderr on a representative run (one run separately hit the same known, already- documented single-benign-error disconnect-timing race task 3.4's own abuse tests hit - not a new issue). Full regression suite, including the net-sim-latency milestone gate and the abuse-detection tests, re-run clean.	2026-08-20 13:40:48 +01:00
Josh Creek	b290f49143	feat(multiplayer): Phase 3 task 3.4 - input validation, rate limiting, disconnect policy ... MatchSim._recv_input now validates before decoding (§3.1 steps 2-3): per-peer rolling-1s rate limiting (packet count AND byte budget, dropping over-budget packets and disconnecting after 3 consecutive over-budget seconds), and framing validation (redundancy count and payload size checked against NetCodec's own layout before unpack_input ever runs, disconnecting after 20 malformed packets). Framing has to be validated explicitly rather than relying on decode failure: StreamPeerBuffer silently zero-fills past EOF instead of erroring, a finding from Phase 2's adversarial review. networked_match.gd's _on_input_received now rejects any seq claiming to be more than 20 ticks ahead of the current server tick (§3.1 step 4) and counts (rather than silently ignoring) input from a peer with no slot, for observability. Verified with two new permanent regression tests (networked_match_smoke.gd --role=client-abuse-malformed / client-abuse-flood) that call MatchSim._recv_input directly with garbage bytes and a legitimate-but- too-frequent flood, respectively, bypassing the honest client encoder entirely - the same thing a hostile custom client sending raw ENet packets would look like. Both confirm real disconnection, not just that the server tolerates the abuse. Two bugs surfaced by getting these tests to actually pass cleanly: a GDScript lambda-capture-by-value mistake in the tests themselves (a plain `var disconnected := false` mutated inside a signal-handler lambda never became visible to the enclosing function - fixed by capturing a single-element Array instead, which is captured by reference); and a narrow real race where NetworkManager's own ping/pong reply could target a peer that a concurrent abuse-triggered disconnect had just removed from the same poll() batch, now guarded. (Passing disconnect_peer's `force` parameter as an attempted fix for a related one-off benign error was tried and reverted - it made Godot's own peer-list bookkeeping inconsistent, producing hundreds of errors instead of one; verified empirically rather than assumed.) Full regression suite, including the net-sim-latency milestone gate, re-run clean.	2026-08-20 13:27:03 +01:00
Josh Creek	5bbb319161	feat(multiplayer): Phase 3 task 3.3 - client-owned input_lead control loop ... New InputLeadController (scripts/input_lead_controller.gd, standalone and unit-tested like input_jitter_buffer.gd): fast attack (+3 immediately, debounced to once per 30 ticks) on any server-reported starve, slow release (-1 per 60 ticks, gated behind a one-time 2s clean-surplus bar) otherwise, clamped [1, 12]. Deliberately the only thing that adapts buffer depth - the server (InputJitterBuffer) stays a pure reporter, per §3.3's explicit warning that multiple control loops acting on one plant (buffer occupancy) oscillate and present as unattributable sticky controls. Wired into the client's per-tick input send: a lead change is realized as extra distance between the client's outgoing sequence numbers and what the server has consumed - an attack skips extra sequence numbers, a release duplicates the current one (sent again, unincremented). The server's ring buffer needs no special handling for either: a skipped seq is an ordinary drop, a duplicated one is a same-seq resend already discarded by the existing "already consumed" check. Verified with real two-process runs: on a clean LAN, one early attack (a momentary hiccup during connection setup) recovers via two releases within the test's own ~4s window, settling back near minimum. Under sustained 30% simulated loss, lead climbs to 7 via repeated attacks and never releases while genuine loss continues - confirming the debounce, attack, and release gates all fire on real conditions, not just in isolated unit tests. Full regression suite, including the net-sim-latency milestone gate, re-run clean.	2026-08-20 13:14:40 +01:00
Josh Creek	86a597f0f5	feat(multiplayer): Phase 3 tasks 3.1/3.2/3.5 - input redundancy + server jitter buffer ... Client now sends the last 4 ticks' actions per packet (newest-first, already-supported by net_codec's wire format from Phase 1) instead of a single action with no redundancy. Server gains a real per-slot ring buffer (new InputJitterBuffer class, scripts/input_jitter_buffer.gd) that consumes exactly one sequence number per physics tick: repeats the last action on a starve, zeroes only after a sustained 500ms stall, and reports real input_buffer_depth/last_input_seq/echo_client_send_ms in every snapshot instead of the hardcoded zeros Phase 2 shipped with. InputJitterBuffer is a standalone, scene-free RefCounted (same pattern as net_codec.gd/net_interpolator.gd) specifically so it's unit-testable against scripted arrival traces (tests/cases/test_input_jitter_buffer.gd): sequential consumption, redundancy surviving a 3-packet burst loss (3.1's own acceptance criterion), starvation repeat-then-zero timing, stale/ reordered packet handling, buffered-depth reporting, and ring-wraparound slot-tagging safety. One real bug found wiring this into a live match: the server's ring buffer started counting its own "expected sequence" from 0 the instant a player's slot was created - well before that player's first real packet could possibly have arrived (connection handshake, arena/ship spawn all take real time first). Since both sides only ever advance monotonically with no resync mechanism, that gap between the server's arbitrary local counter and the client's actual from-1 sequence numbers never closed, so the ship simply never received the client's input (0m movement in a two-process test). Fixed by seeding the buffer's expected-sequence counter from the client's own numbering on first real ingest, rather than assuming a shared from-zero baseline. Verified with real two-process runs: clean baseline movement restored, zero starvation observed under 25% random simulated input loss (well above what redundancy-4 needs to fully absorb), and correct starve-then- stall behaviour confirmed under 100% loss as a sanity check that the mechanism isn't a silent no-op. Full regression suite, including the net-sim-latency milestone gate, re-run clean.	2026-08-20 13:07:33 +01:00
Josh Creek	7b150ef72e	feat(multiplayer): task 2.8 net_sim.gd, close out Phase 2 ... New NetSim autoload: seeded, CLI-driven (--net-sim-latency/-jitter/-loss/-dup) latency/jitter/loss/duplicate decorator, a true no-op passthrough unless a flag is set. Wraps MatchSim.send_input/send_snapshot per the design doc's scope, plus NetworkManager's ping/pong so the already-tested RTT/clock measurement becomes the acceptance signal for "raises observed RTT" without waiting on Phase 3's per-peer snapshot echo. Two real bugs found while building and verifying this against Phase 2's own milestone gate (a real match under --net-sim-latency 80 --net-sim-jitter 20, not just LAN): a timestamp captured inside a delayed RPC closure silently ate that side's own added delay out of the round-trip measurement instead of adding to it; and a delayed send whose target disconnected (or whose own process had already shut down) during the hold threw RPC errors, since the existing get_peers() filtering only checked validity at schedule time. Fixed by capturing timestamps before handing off to NetSim, and by having NetSim re-validate the target at fire time. Phase 2's milestone gate now passes for real: a full 1v1 under simulated 80ms latency / 20ms jitter still shows clean server-authoritative movement and zero RPC errors. Full Phase 1 + Phase 2 regression suite re-verified clean with NetSim present but inactive.	2026-08-20 08:50:47 +01:00
Josh Creek	39a41c016c	feat(multiplayer): Phase 2 server-authoritative simulation, dumb client ... Implements tasks 2.1-2.7: NetworkedMatch spawns a deterministic slot layout from the lobby roster, the server drives each connected peer's ship via RLShipController fed by decoded client input and broadcasts 60Hz snapshots, and the client renders everything (including its own ship) from a per-body NetInterpolator with no local prediction yet. Dual-time remote entities split collider updates (present-time, for correct contacts) from $Visual updates (interp-delayed, for smoothness). Camera/HUD wiring and remote engine-flame VFX fell out of the existing Ship API for free once snapshots were flowing. Three real bugs found and fixed while getting a two-process test green: an RPC method named _input collided with Node's built-in _input virtual and broke the whole MatchSim autoload from loading; networked_match.gd never called NetworkManager.poll(), so nothing sent via RPC in this scene reached the wire despite Phase 1's manual polling being wired up everywhere else; and a match_config request/response fallback (added to close a startup race) could double-deliver once polling was fixed, requiring an idempotency guard. Verified with tests/networked_match_smoke: a real headless two-process host+client run shows the client rendering 31m of server-authoritative movement from a held forward-thrust input, with thrust_z=1.0 confirmed on the interpolated snapshot mid-drive and camera/HUD both wired. Full Phase 1 regression suite re-run clean alongside it. Task 2.8 (net_sim.gd latency/jitter/loss decorator) is not yet done; Phase 2's own gate needs it before it's fully met.	2026-08-20 08:42:13 +01:00
Josh Creek	4533da34e0	feat(multiplayer): Phase 1 transport, connection, and lobby ... Lands tasks 1.0-1.8 of multiplayer-todo.md: the pure-function test runner, net_codec (wire format quantizers/pack-unpack), NetworkManager (ENet transport, manual polling, min-RTT clock sync), MatchNet (handshake, protocol/tick-rate gating, roster with team+ready state), lobby.tscn (team columns, switch team, ready toggle), server_boot.tscn (headless dedicated server with structured logging and an overrun watchdog), and main_menu.gd's Host/Join-by-IP UI (connecting overlay, cancel, bounded failure path). Followed by an adversarial review (Opus subagent) that found and fixed two real bugs - an unvalidated player_name broadcast that let one client's oversized name head-of-line-block the reliable channel for everyone, and a server-side roster leak across a host/re-host cycle - plus three gaps in the test suite itself where a claim of "verified" wasn't actually backed by what the test checked. All five two-process smoke tests plus the pure-function suite are green with the strengthened assertions in place.	2026-08-20 08:18:59 +01:00