docs: finalize scalable matchmaking plan

multiplayer-next.md

@@ -1,93 +1,141 @@
 # Multiplayer — next work
 
-Short, current checklist for online multiplayer. Historical design decisions,
-implementation evidence, and completed work stay in
-[`multiplayer-todo.md`](multiplayer-todo.md).
+Short, current checklist for online multiplayer. Historical decisions,
+implementation evidence and task-level acceptance criteria stay in
+[`multiplayer-todo.md`](multiplayer-todo.md). Phase 8 architecture and locked
+product policy are in [`docs/MATCHMAKING.md`](docs/MATCHMAKING.md).
 
-## Release blockers
+## Existing release blockers
 
-- [ ] **Phase 4 playtest:** a human playtest at roughly 100 ms RTT. Confirm
-  that ship and ball interaction feel local and contact corrections feel like
-  bumps rather than glitches.
-- [ ] **Phase 5 session:** complete a real 3v3 match with a mid-match
-  disconnect and late joiner.
-- [ ] **Phase 6 external check:** run the exported Docker server and clients
-  from separate machines over the internet, then play a full match. Keep this
-  controlled-only until Steam identity is complete.
+- [ ] **Phase 4 playtest:** play at roughly 100 ms RTT; confirm ship/ball
+  interaction feels local and contact corrections read as bumps, not glitches.
+- [ ] **Phase 5 session:** finish a real 3v3 match with a mid-match disconnect
+  and late joiner.
+- [ ] **Phase 6 external check:** play the exported Docker server from separate
+  internet machines. Keep the check controlled until verified identity lands.
 
-## Phase 7 — Steam, identity, discovery
+## Phase 7 — production Steam prerequisite
 
-- [ ] Obtain the pinned GodotSteam client/server builds and Steamworks SDK
-  access described in [`STEAM.md`](STEAM.md).
-- [ ] Run `make verify-steam-templates` with the custom executables and fix
-  any custom-template failures.
-- [ ] Validate a two-account Steam SDR host/join using the existing explicit
-  `NetworkManager` Steam transport. ENet direct-IP must keep passing its smoke
-  test.
-- [ ] Build the Steam server browser: internet, LAN, favourites, and history.
-- [ ] Add Steam auth tickets, verified Steam identity in the roster, and a
-  persistent ban list. This fixes the slot-reclaim security issue below.
+- [ ] Obtain the pinned GodotSteam client/server builds and Steamworks SDK;
+  pass `make verify-steam-templates` without weakening ENet verification.
+- [ ] Validate two real accounts through the explicit Steam transport and
+  build Internet/LAN/favourites/history server-browser views.
+- [ ] Add single-use auth tickets, asynchronous server validation, verified
+  Steam identity, identity-keyed reconnect and persistent bans.
+- [ ] Obtain the real App ID, publisher key, coordinator SDK/signing approval,
+  certificates and Hosted Dedicated Server data-centre support from Valve.
+- [ ] Implement ticketed Hosted Dedicated Server SDR routing, ticket install,
+  reconnect and expiry. Preserve direct ENet for local/CI/community servers.
 
-## Phase 8 — casual and ranked matchmaking (1.0 launch blocker)
+## Phase 8 — architecture, contracts and durable data
 
-Design and reasoning: [`docs/MATCHMAKING.md`](docs/MATCHMAKING.md). This is a
-different server model from the community-server one that exists today —
-players queue, a matchmaker groups them, and a server is allocated per match.
-Phase 7's Steam auth tickets are a hard prerequisite: a rating attached to a
-spoofable identity is worse than no rating.
+- [ ] Lock the Go/PostgreSQL/Redis, Kubernetes/Agones, SDR, EU/NA and
+  provider-portability ADR; define measurable launch SLOs.
+- [ ] Publish versioned OpenAPI/WebSocket contracts, stable IDs, legal state
+  transitions, revisions and idempotency semantics.
+- [ ] Add PostgreSQL queue ownership/active-participation fences, durable
+  domain migrations/outbox and Redis indexes/TTLs; lost Redis writes must not
+  split a proposal or corrupt durable state.
+- [ ] Define assignment compatibility and opt-in `ServerConfig` flags whose
+  defaults reproduce the community-server path.
 
-- [ ] Decide the rating algorithm (Glicko-2 recommended over Elo for a small
-  launch population) and how a team result distributes across individuals.
-- [ ] Choose the backend language and hosting, and cost out allocated servers
-  per match at expected population.
-- [ ] Stand up the backend: Steam auth ticket validation via the Steamworks
-  Web API, queue, rating store, server allocator.
-- [ ] Add an assigned-roster server mode so only matched SteamIDs may claim a
-  slot, replacing the first-come model.
-- [ ] Add server-authoritative match result reporting to the backend over a
-  channel a client cannot forge.
-- [ ] Client queue UI: playlist select, estimated wait, accept/decline,
-  connect-on-assignment, post-match rating delta.
-- [ ] Casual and ranked playlist rulesets (backfill, bots, abandon penalties,
-  arena restriction — see the comparison table in the design doc).
+## Phase 8 — identity and security
 
-Server orchestration (same phase — the servers must autoscale and bill only
-for the duration of a match):
+- [ ] Validate Steam Web API tickets only in the secure backend; issue
+  revocable sessions and reconnect-safe match/identity/slot authorisations
+  with server-owned connection-generation fencing.
+- [ ] Authenticate results with pod/GameServer-bound workload identity; make
+  identical duplicates idempotent and conflicting results inert/alerting.
+- [ ] Complete the threat model for forgery, replay, queue/flood/bot abuse,
+  workload/insider compromise, DDoS, supply chain and denial-of-wallet.
+- [ ] Enforce restricted workloads/RBAC/networks/private stores/backups/secrets;
+  isolate SDR signing behind an audited non-exportable signer and add
+  volumetric edge defense, WebSocket limits and overload shedding.
+- [ ] Pin, scan, SBOM and sign artifacts; verify signatures at admission and
+  document the critical vulnerability SLA.
 
-- [ ] Choose the orchestrator (Agones on Kubernetes is the recommended
-  default; it models allocation, readiness and per-match lifetime natively).
-- [ ] Fix readiness detection. Godot's stdout is block-buffered off a TTY, so
-  `server_started` never appears in `docker logs` for a detached container —
-  a log-grep readiness probe hangs forever. Probe the UDP socket, or flush.
-- [ ] Support more than one match per host: a per-container port from a range,
-  or one address per match. `--port` defaults to 7777 and the Dockerfile
-  hardcodes `EXPOSE 7777/udp`.
-- [ ] Add a no-show timeout so an allocated server that never fills abandons
-  and exits instead of idling at cost.
-- [ ] Re-measure boot-to-listening on native x86_64 Linux. The repo's current
-  figure is ~870 ms, measured under emulation on arm64 — a pessimistic bound.
-- [ ] Re-measure the SERVER.md sizing estimate (6–10 processes/core,
-  150–250 MB RSS) under real concurrency before it sizes a bill.
-- [ ] Keep `make verify-phase6` and `make verify-enet-integration` green:
-  every allocation feature is opt-in via a `ServerConfig` flag defaulting to
-  today's behaviour, with a second Compose file for the allocated path rather
-  than mutating `compose.phase6-smoke.yml`.
+## Phase 8 — queues, playlists and rating
 
-## Known issues to resolve before public hosting
+- [ ] Add one PostgreSQL-owned queue ticket/player with 10 s heartbeat, 30 s
+  expiry, Redis candidate cache and restart/failover repair.
+- [ ] Validate opaque Steam ping locations and nonce-bound probes server-side;
+  require <=100 ms, enforce discrepancy quarantine and the locked widening/
+  region/team tie-break rules.
+- [ ] Send 10 s proposals to every selected human: ranked six, relaxed casual
+  two to six with disclosed bots; enforce exact cooldown and queue-precedence
+  behavior.
+- [ ] Fence proposals/participants in a PostgreSQL serializable transaction;
+  prove loss of an acknowledged Redis write cannot split players.
+- [ ] Casual: target 3v3 humans, after 60 s allow >=2 humans (one/team) plus
+  bots, kickoff-only human backfill and no backfill loss/decline penalty.
+- [ ] Ranked: exactly six humans, solo-only, no bots/backfill, random-enabled
+  non-elevated arenas only, 60 s reconnect grace and escalating abandons.
+- [ ] Implement the documented exact Glicko-2 equations, fractional 3v3
+  weights, inactivity/update locking/golden vectors and ten provisional games.
+- [ ] Add ranked-only exactly-once 12-week soft seasons; distinguish retryable
+  result-delivery outages from match-integrity failures and rating exemptions.
 
-- [ ] Slot reclaim is currently keyed by display name, so someone can take a
-  disconnected player's reserved slot. Do not expose public servers before
-  verified Steam identity lands.
-- [ ] Investigate occasional input loss during a long server stall; the
-  existing sequence resync recovers it, but transport delivery is variable.
+## Phase 8 — Agones and regional server capacity
+
+- [ ] Add portable EU/NA Agones Fleets with provider edge/network/secret and
+  Valve-approved SDR POP/certificate/public-UDP overlays.
+- [ ] Add the local-safe Agones adapter and separate process-ready (listen then
+  Ready) from assignment-ready (Allocated manifest verified and registered).
+- [ ] Allocate from Ready by region/build/protocol/transport; use separately
+  verified ENet and SDR dynamic/passthrough port mappings.
+- [ ] Deliver/verify the signed roster after allocation and expose client
+  tickets only after backend `assignment_ready`.
+- [ ] Keep >=2 Ready processes across >=2 on-demand nodes/failure domains per
+  queue-enabled region; only Allocated count may fall to zero.
+- [ ] Spread on-demand capacity across zones with N+1 headroom; do not place
+  live matches on interruptible nodes.
+- [ ] Benchmark native x86_64 boot, p99 CPU/RSS/network and tick health; set
+  requests/limits and node density from measurements plus 30% headroom.
+- [ ] Add 30 s no-show handling, Go PID-1 TERM/drain supervision, PDB/Fleet
+  drain, signed result annotation/retry, RPO <=5 m and RTO <=30 m.
+- [ ] Rehearse migration only after the second provider's EU/NA locations have
+  Valve approval, POP/certs, public UDP/firewall and coordinator trust.
+
+## Phase 8 — client and recovery
+
+- [ ] Build queue/proposal/allocation/connect/rating UI with explicit latency,
+  capacity, expiry and recovery states.
+- [ ] Use one authenticated revisioned WebSocket plus REST resync; resume a
+  valid ticket/assignment after restart rather than duplicating it.
+- [ ] After assignment-ready, install SDR ticket and send reconnect-safe join
+  authorisation in `hello`; fence old connections and retain ENet behavior.
+- [ ] Display only backend-authoritative provisional rank/tier/delta, abandon
+  status and season time; clients perform no rating calculation.
+
+## Phase 8 — operations and release gates
+
+- [ ] Correlate queue→result with IDs and add dashboards/alerts for SLOs,
+  security, failures and cost without logging credentials.
+- [ ] Add Go race/fuzz/property/migration/concurrency coverage plus fake Steam
+  and fake allocation for offline deterministic CI.
+- [ ] Add an independent allocated-server Compose flow; do not mutate
+  `compose.phase6-smoke.yml` or weaken either existing Make gate.
+- [ ] Add disposable `kind`/Agones integration, 100 ms network/chaos cases and
+  proof that infrastructure failures cannot punish players.
+- [ ] Load-test >=10,000 queued clients, >=100 proposals/s and forecast launch
+  concurrency x2 while holding API p95 <=250 ms and allocation correctness.
+- [ ] Record cost per completed match, budget/denial-of-wallet controls and
+  deploy progressively: internal → casual canary → casual → provisional
+  ranked → ranked, with EU/NA playtests and rollback gates.
+
+## Known issues before public hosting
+
+- [ ] Replace display-name slot reclaim with verified Steam identity.
+- [ ] Investigate occasional transport input loss during a long server stall.
 - [ ] Fix the remaining `_broadcast_snapshot` packet-send stderr race.
 
 ## Decide after the latency playtest
 
-- [ ] Decide whether client-only, contact-cohort shadow physics is worthwhile
-  for the remaining prediction weakness.
+- [ ] Decide whether client-only contact-cohort shadow physics is worthwhile.
 
 ## Explicitly deferred
 
-120 Hz simulation, latency-gap measurement, audio hooks, and split-screen are
-not part of the current multiplayer release path.
+Parties/premades, tournaments, ranked spectators, non-Steam identity,
+additional global regions, global leaderboards, 120 Hz simulation,
+latency-gap measurement, audio hooks and split-screen are not in the launch
+path.