# Multiplayer — next work Short, current checklist for online multiplayer. Historical decisions, implementation evidence and task-level acceptance criteria stay in [`multiplayer-todo.md`](multiplayer-todo.md). Phase 8 architecture and locked product policy are in [`docs/MATCHMAKING.md`](docs/MATCHMAKING.md). ## Existing release blockers - [ ] **Phase 4 playtest:** play at roughly 100 ms RTT; confirm ship/ball interaction feels local and contact corrections read as bumps, not glitches. - [ ] **Phase 5 session:** finish a real 3v3 match with a mid-match disconnect and late joiner. - [ ] **Phase 6 external check:** play the exported Docker server from separate internet machines. Keep the check controlled until verified identity lands. ## Phase 7 — production Steam prerequisite - [ ] Obtain the pinned GodotSteam client/server builds and Steamworks SDK; pass `make verify-steam-templates` without weakening ENet verification. - [ ] Validate two real accounts through the explicit Steam transport and build Internet/LAN/favourites/history server-browser views. - [ ] Add single-use auth tickets, asynchronous server validation, verified Steam identity, identity-keyed reconnect and persistent bans. - [ ] Obtain the real App ID, publisher key, coordinator SDK/signing approval, certificates and Hosted Dedicated Server data-centre support from Valve. - [ ] Implement ticketed Hosted Dedicated Server SDR routing, ticket install, reconnect and expiry. Preserve direct ENet for local/CI/community servers. ## Phase 8 — architecture, contracts and durable data - [x] Lock the Go/PostgreSQL/Redis, Kubernetes/Agones, SDR, EU/NA and provider-portability ADR ([ADR-001](docs/ADR-001-matchmaking-platform.md)); measurable launch SLOs are defined in [MATCHMAKING-SLOs.md](docs/MATCHMAKING-SLOs.md). - [x] Publish versioned OpenAPI/WebSocket contracts, stable IDs, legal state transitions, revisions and idempotency semantics ([v1 contracts](server/contracts/v1/)). - [ ] **IN PROGRESS:** Add PostgreSQL queue ownership/active-participation fences, durable domain migrations/outbox and Redis indexes/TTLs; lost Redis writes must not split a proposal or corrupt durable state. Initial migration and serializable store boundaries are implemented, including durable queue create/heartbeat/cancel/recovery adapters; an opt-in pgx/Docker harness now executes the migrations and real queue create/idempotency/ownership/recovery path, an executable migration runner now serializes and records forward application, and a TTL-bound Redis candidate index now supports atomic rebuild, snapshot and removal with durable-source repair on partial/malformed cache state; proposal/result transactions and live Redis restart/failover gates remain. The matcher package now performs bounded candidate formation and delegates the final proposal claim to the durable transaction boundary; queue tickets now also retain server-derived probe RTT metadata for authoritative matcher reads; ranked metadata/provider wiring and live Redis repair remain. The authenticated probe API now records validated server-computed RTT values into the active player's durable queue ticket and fails closed when that write is unavailable; Steam/coordinator evidence acquisition and multi-region probe population remain. - [ ] **IN PROGRESS:** Durable allocator registry now records READY GameServer projections and atomically claims compatible capacity with replay/conflict fencing; `server/agones` now submits and validates namespaced `GameServerAllocation` responses, including dynamic address/port data; `server/allocator` now requires provider allocation reconciliation into the durable registry before returning an endpoint; allocator-facing roster publication now requires an allocated endpoint and verifies canonical join-authorisation signatures before exposing player rows; live Agones integration remains. - [ ] **IN PROGRESS:** Run the Go control plane against PostgreSQL/Redis with independently runnable API, matcher, allocator and maintenance roles. The `cmd/control-plane` API role now opens PostgreSQL, applies migrations, wires authenticated durable queue/proposal/assignment/session adapters, and shuts down gracefully; optional `--redis-addr` publishes queue mutations to a TTL-bound best-effort candidate projection without making Redis authoritative; a runnable casual `cmd/matcher` role now polls PostgreSQL and delegates proposal claims to the durable transaction; `cmd/maintenance` now runs bounded ranked-season rollover batches with signal-bound shutdown; provider-backed allocation, ranked provider wiring, Redis worker wiring and live service checks remain. - [ ] **IN PROGRESS:** Define assignment compatibility and opt-in `ServerConfig` flags whose defaults reproduce the community-server path. Allocation manifest validation now covers client build and future expiry; allocated servers now expose loopback process-ready/drain control, an Agones REST bridge for health/lifecycle calls, and fence new admissions while draining; signed admission remains. ## Phase 8 — identity and security - [ ] **IN PROGRESS:** Validate Steam Web API tickets only in the secure backend; issue revocable sessions and reconnect-safe match/identity/slot authorisations with server-owned connection-generation fencing. Pure Go ticket/session and reconnect policies exist, including canonical signed join-authorisation issuance/verification; production Steam/backend adapters and persistent lease fencing remain. - [ ] **IN PROGRESS:** Authenticate results with pod/GameServer-bound workload identity; make identical duplicates idempotent and conflicting results inert/alerting. Pure Go credential-claim validation, binding, hashing, reconciliation, and the atomic receipt/completion/outbox SQL boundary exist; projected-token/JWT adapters, trusted-cluster verification, rating-lock integration, and production alerting remain. The API now exposes the workload-authenticated server result route and delegates completion to the durable receipt/outbox adapter. A dependency-free projected JWT adapter now verifies the compact-token signature through an injected trust boundary and delegates exact claim/time binding to the domain policy. - [x] Complete the threat model for forgery, replay, queue/flood/bot abuse, workload/insider compromise, DDoS, supply chain and denial-of-wallet ([THREAT-MODEL.md](docs/THREAT-MODEL.md)). - [ ] **IN PROGRESS:** Enforce restricted workloads/RBAC/networks/private stores/backups/secrets; isolate SDR signing behind an audited non-exportable signer and add volumetric edge defense, WebSocket limits and overload shedding. A provider-neutral restricted Kubernetes baseline and structural policy tests now exist; live edge/data-plane controls remain. - [ ] **IN PROGRESS:** Pin, scan, SBOM and sign artifacts; verify signatures at admission and document the critical vulnerability SLA. Repository image references are now digest-pinned with a static secret-hygiene guard and a 24-hour critical-fix policy; registry execution and concrete release provenance remain. ## Phase 8 — queues, playlists and rating - [ ] **IN PROGRESS:** Add one PostgreSQL-owned queue ticket/player with 10 s heartbeat, 30 s expiry, Redis candidate cache and restart/failover repair. Authenticated queue creation now requires playlist, client build and protocol version and passes them to the server-owned candidate provider; PostgreSQL/Redis wiring remains. - [ ] **IN PROGRESS:** Validate opaque Steam ping locations and nonce-bound probes server-side; require <=100 ms, enforce discrepancy quarantine and the locked widening/ region/team tie-break rules. Authenticated probe transport now routes opaque location/nonce data through a server-owned evidence provider and refuses client RTT values; Steam/coordinator adapters remain. - [ ] **IN PROGRESS:** Form deterministic candidate sets and balanced teams from the server-owned queue projection. Queue-backed oldest-anchor formation and duplicate-player fencing now exist; durable matcher claims remain. - [ ] **IN PROGRESS:** Send 10 s proposals to every selected human: ranked six, relaxed casual two to six with disclosed bots; enforce exact cooldown and queue-precedence behavior. Playlist-aware proposal preparation now validates casual team humans and ranked identity/arena metadata before creating proposal state; durable queue precedence and allocation integration remain. - [ ] **IN PROGRESS:** Fence proposals/participants in a PostgreSQL serializable transaction; prove loss of an acknowledged Redis write cannot split players. The Go store adapter now performs proposal insertion, participant insertion, and every queue-ticket promotion in one rollback-safe SERIALIZABLE callback; live DB/ Redis failover testing remains. - [ ] Casual: target 3v3 humans, after 60 s allow >=2 humans (one/team) plus bots, kickoff-only human backfill and no backfill loss/decline penalty. - [ ] **IN PROGRESS:** Ranked: exactly six humans, solo-only, no bots/backfill, random-enabled non-elevated arenas only, 60 s reconnect grace and escalating abandons. - [ ] **IN PROGRESS:** Implement the documented exact Glicko-2 equations, fractional 3v3 weights, inactivity/update locking/golden vectors and ten provisional games. Backend-owned provisional status and validated ranked-tier derivation now exist; authenticated ranked-profile transport now exists; client display and persisted tier configuration remain. - [ ] **IN PROGRESS:** Add ranked-only exactly-once 12-week soft seasons; distinguish retryable result-delivery outages from match-integrity failures and rating exemptions. A durable per-player/per-season rollover marker and SERIALIZABLE rating update boundary now exist; live scheduler/DB execution remains. ## Phase 8 — Agones and regional server capacity - [ ] **IN PROGRESS:** Add portable EU/NA Agones Fleets with provider edge/network/secret and Valve-approved SDR POP/certificate/public-UDP overlays. A restricted provider-neutral Fleet base and distinct EU/NA Kustomize overlays now exist; live rendering and provider/Valve overlays remain. - [ ] Add the local-safe Agones adapter and separate process-ready (listen then Ready) from assignment-ready (Allocated manifest verified and registered). The Go supervisor now validates dynamic address/port data and gates Ready on an explicit probe; Godot adapter and emulator integration remain. - [ ] **IN PROGRESS:** Allocate from Ready by region/build/protocol/transport; use separately verified ENet and SDR dynamic/passthrough port mappings. The Go allocator now owns assignment publication with idempotent replay/conflict handling; Agones integration remains. - [ ] Deliver/verify the signed roster after allocation and expose client tickets only after backend `assignment_ready`. - [ ] **IN PROGRESS:** Keep >=2 Ready processes across >=2 on-demand nodes/failure domains per queue-enabled region; only Allocated count may fall to zero. A provider-neutral Agones FleetAutoscaler now encodes a two-process Ready buffer and six-process warm cap; regional node pools, pre-pull rollout and measured N+1 capacity remain. - [ ] **IN PROGRESS:** Spread on-demand capacity across zones with N+1 headroom; do not place live matches on interruptible nodes. The Fleet now requires the on-demand capacity label and uses a zone topology spread constraint; force-loss testing of the largest node and measured headroom remain. - [ ] Benchmark native x86_64 boot, p99 CPU/RSS/network and tick health; set requests/limits and node density from measurements plus 30% headroom. - [ ] Add 30 s no-show handling, Go PID-1 TERM/drain supervision, PDB/Fleet drain, signed result annotation/retry, RPO <=5 m and RTO <=30 m. The Go supervisor now owns bounded drain-before-kill orchestration, the drain boundary is authenticated and loopback-only, and the base PDB protects the two-Ready floor; lifecycle/PDB/Fleet integration remains. - [ ] Rehearse migration only after the second provider's EU/NA locations have Valve approval, POP/certs, public UDP/firewall and coordinator trust. ## Phase 8 — client and recovery - [ ] Build queue/proposal/allocation/connect/rating UI with explicit latency, capacity, expiry and recovery states. - [ ] Use one authenticated revisioned WebSocket plus REST resync; resume a valid ticket/assignment after restart rather than duplicating it. - [ ] After assignment-ready, install SDR ticket and send reconnect-safe join authorisation in `hello`; fence old connections and retain ENet behavior. - [ ] Display only backend-authoritative provisional rank/tier/delta, abandon status and season time; clients perform no rating calculation. ## Phase 8 — operations and release gates - [ ] Correlate queue→result with IDs and add dashboards/alerts for SLOs, security, failures and cost without logging credentials. - [ ] Add Go race/fuzz/property/migration/concurrency coverage plus fake Steam and fake allocation for offline deterministic CI. - [ ] Add an independent allocated-server Compose flow; do not mutate `compose.phase6-smoke.yml` or weaken either existing Make gate. - [ ] Add disposable `kind`/Agones integration, 100 ms network/chaos cases and proof that infrastructure failures cannot punish players. - [ ] Load-test >=10,000 queued clients, >=100 proposals/s and forecast launch concurrency x2 while holding API p95 <=250 ms and allocation correctness. - [ ] Record cost per completed match, budget/denial-of-wallet controls and deploy progressively: internal → casual canary → casual → provisional ranked → ranked, with EU/NA playtests and rollback gates. ## Known issues before public hosting - [ ] Replace display-name slot reclaim with verified Steam identity. - [ ] Investigate occasional transport input loss during a long server stall. - [x] Fix the remaining `_broadcast_snapshot` packet-send stderr race. ## Decide after the latency playtest - [ ] Decide whether client-only contact-cohort shadow physics is worthwhile. ## Explicitly deferred Parties/premades, tournaments, ranked spectators, non-Steam identity, additional global regions, global leaderboards, 120 Hz simulation, latency-gap measurement, audio hooks and split-screen are not in the launch path.