CosmicClash/multiplayer-next.md

# Online multiplayer — task breakdown

The tracking document for the online multiplayer effort's outstanding work:
what's left, why, and the task breakdown. `TODO.md` points here for anything
multiplayer-related. The architecture decisions, wire format, input
handling, prediction, latency budget, and match lifecycle spec this work
assumes now live in **[`MULTIPLAYER_SPEC.md`](MULTIPLAYER_SPEC.md)** as its
own sections 1–6 — read those before picking up work in Phase 2 or later.

**How to use this doc:** start at §0 for what's outstanding right now. Pick up
a single numbered task, do it, verify it against its stated acceptance
criterion, and stop. §9 is a running gotchas list — check it before debugging
something that looks like a Godot/Jolt engine quirk, and add to it when you
find a new one.

**This revision keeps only outstanding work.** Phases 0–6 are fully
implemented and verified locally; their task-by-task implementation evidence
has been trimmed from this document and lives in git history
(`git log -- multiplayer-next.md`) rather than here. Phase 7 (Steam) and
Phase 8 (matchmaking) are in progress — the tables below list only what
remains on each task, not what's already built. **Phase 8 is a 1.0 launch
blocker**, adds a component outside the Godot project (a Go backend
service), and has a critical open blocker: see §0.

---

## 0. Outstanding work — the short list

The one place to look before planning. Everything here is also written up
where it belongs; this is the index, not the detail.

Anything below that needs a *person* rather than an agent is also a GitHub
issue, labelled
[`needs:human`](https://github.com/jcreek/CosmicClash/issues?q=is%3Aissue+is%3Aopen+label%3A%22needs%3Ahuman%22)
plus a `P0-blocker`…`P3-low` priority, and linked inline below. The numbered
tasks in §7 are agent-actionable and deliberately have no issues — this
document is their tracker.

**Phase 8 (matchmaking, ranked, per-match server autoscaling) is a 1.0 launch
blocker and is in progress.** It is larger than anything below and adds a
backend service outside the Godot project. Tasks are in §7; the design is in
[`docs/MATCHMAKING.md`](docs/MATCHMAKING.md).

**The former root blocker** ([#14](https://github.com/jcreek/CosmicClash/issues/14)) **is closed.** Nothing in production
used to publish a player's signed match assignment:
`store.SaveVerifiedAssignmentRoster` and `allocator.Service.PublishRoster`
were fully built and tested in isolation, but no real code path called them —
only tests did, by seeding the table directly. Since
`AdvanceServerRegistration`'s SQL requires an `assignments` row per
participant before a match can reach `ASSIGNMENT_READY`, a real deployment
could not advance any match past `PROCESS_READY`.

`allocator.Worker.RunOnce` now builds one signed join authorisation per
durable participant and publishes the roster after binding the allocation, and
`cmd/allocator` refuses to start without key material rather than stranding
every match silently. The signing-key design that was pending a decision is
settled: HMAC-SHA256 over the canonical claim bytes, with a **key ID** in
those bytes so allocated servers can hold the set of currently-valid keys and
rotation does not invalidate authorisations already issued for in-flight
matches. See `docs/MATCHMAKING.md` §2 for the rotation procedure.

Two further blockers of the same shape were found and closed alongside it:
regional RTT probing had no nonce-issuing endpoint (so no client-created
ticket could ever be selected — the matcher requires non-empty RTT evidence),
and the Kubernetes base deployed a control-plane image nothing built while
building a matcher image nothing deployed. What remains for a live deployment
is external: a Steamworks App ID and publisher key ([#15](https://github.com/jcreek/CosmicClash/issues/15)),
custom GodotSteam builds ([#16](https://github.com/jcreek/CosmicClash/issues/16)),
and a real cluster ([#17](https://github.com/jcreek/CosmicClash/issues/17)) and the images
to run there ([#31](https://github.com/jcreek/CosmicClash/issues/31)).

**Rows were audited against the code on 2026-09-05.** Nine understated what
was already built — 8.6, 8.8, 8.13, 8.16, 8.19, 8.30, 8.42, 8.43, 8.52 — on
top of 7.4, 8.7, 8.20, 8.22 and 8.39 corrected while working on them. The
drift ran one way: rows kept listing work that had since landed, which makes
the backlog look larger than it is and invites rebuilding what exists. Twice
during this branch a task was picked up only to find one of its named parts
already complete (8.20's allocation wiring, 8.22's client UI). **When picking
up a row, verify its claim against the code before planning against it** —
and correct the row if it is stale, since an unverified row is a rumour, not
a backlog item.

Every corrected claim is backed by an executable test rather than by having
located an implementation, because locating one proves it exists, not that it
works:

| Claim | Proof |
|---|---|
| 8.6 allocated `ServerConfig` fields | `test_server_config.gd::test_allocated_mode_is_opt_in_and_requires_compatibility_manifest` |
| 8.6 signed-authorisation admission | `test_match_net.gd` join-authorisation cases, incl. the key-rotation set |
| 8.6 endpoint wiring | `test_assignment_state.gd` — endpoint preserved, unsafe endpoint rejected |
| 8.8 cross-replica revocation | `TestPostgreSQLSessionRevocationIsImmediateOnAnotherReplica` |
| 8.19 lineup reached through formation | `TestPrepareProposalBuildsCasualLineupBeforeCreatingProposal` |
| 8.19 all four penalty kinds durable | existing integration tests, plus `TestPostgreSQLInitialConnectNoShowWritesADurablePenalty` |
| 8.30 signed roster metadata | `TestRealAllocatorWorkerPublishesSignedAssignmentRoster` |
| 8.42 season countdown | `test_control_plane_client.gd` — `"Season ends in 2d"` and the clamped case |
| 8.16/8.43 matcher deployed | `test_kubernetes_policies.py::test_every_required_workload_role_is_deployed` |

Two claims had no proof and needed one written: `INITIAL_CONNECT_NO_SHOW`
penalties and cross-replica revocation. Both new tests were mutation-checked —
disabling the behaviour makes them fail — so they assert something real. 8.13
and 8.52 are cross-references and assert nothing.

### Blocking sign-off — the work exists, the verification does not

| # | What | Why it is not done | Detail |
|---|---|---|---|
| A | ([#18](https://github.com/jcreek/CosmicClash/issues/18)) **Phase 4 human playtest at ~100 ms RTT.** Does the ship feel local? Does the ball? Do contact corrections read as bumps or as glitches? | Needs hands on a controller. Every numeric gate is green; feel is the milestone's actual subject and no percentile can answer it. | §5.7 |
| B | ([#19](https://github.com/jcreek/CosmicClash/issues/19)) **Phase 5 3v3 gate**: a full start-to-finish match with 6 players, a mid-match disconnect, and a late joiner. | Needs a real multi-client session. Every scenario is verified at 1v1 plus a two-bot CI match; nothing has run at 3v3. | §6 |

These two are independent and can be done in either order, but B is the
cheaper of the two to arrange and would also exercise A's conditions
incidentally.

### Known defects

| # | What | Severity | Detail |
|---|---|---|---|
| C | **Slot reservation and takeover are keyed on display name alone, for direct/unauthenticated servers only.** For allocated (signed-roster) matches this is resolved — reconnect reclaim and late-join promotion carry the verified `PlayerID` across peer-id changes. Direct/community servers with no Steam identity still resolve reclaim by display name; a peer connecting with a departed player's name inside the 30 s window claims their slot. | Real, demonstrated, bounded to the unauthenticated direct-server path. | §11, Phase 7 task 7.4 |
| D | **Input is still lost at the transport layer during a long server stall**, variably — 7 of 8 runs measured 0.00 % of the sequence stream missing, the eighth 23.54 %. | Low. Distinct from the rate-limiter cause, which is fixed. The seq-guard resync visibly recovers it. | §9 gotchas 39, 48, 49 |

The residual half of C (direct/community servers) is fixed for free by task
**7.4** (Steam auth tickets in `hello`) once Phase 7 lands; it has not been
given a bespoke solution for that reason.

### Open architectural question

| # | What | Detail |
|---|---|---|
| F | ([#23](https://github.com/jcreek/CosmicClash/issues/23)) **A contact-cohort-only shadow world.** The remaining known prediction weakness is the contact cohort. Whether it is worth a client-side shadow Jolt world scoped to contacts alone is undecided — and deliberately so until A supplies the felt evidence. | §5.7 |

### Unstarted phases

- **Phase 6 external gate** ([#20](https://github.com/jcreek/CosmicClash/issues/20))**:** run the exported Docker server and clients from separate machines over the internet, then play a full match. This is a controlled test only until **C** is fully closed (i.e. Phase 7 lands).
- **Phase 7 — Steam transport, browser, identity and production SDR** (8 tasks, in progress; its two human prerequisites are [#15](https://github.com/jcreek/CosmicClash/issues/15) App ID and [#16](https://github.com/jcreek/CosmicClash/issues/16) export templates): the optional bootstrap and `NetTransport` foundation now exist, but custom Steam client/server export templates have not yet been supplied. Browser, verified tickets, bans, production credentials and ticketed Hosted Dedicated Server SDR await a project-owned Steamworks App ID and Valve coordination. Carries the fix for the direct-server half of **C** and is the hard prerequisite for Phase 8's production Steam identity.

Phase 6's external gate has no dependency on Phase 7 for a controlled test,
but Phase 7 is next in priority because Steam identity is required before
public exposure.

### Deferred by choice, not forgotten

120 Hz simulation, the latency-gap *measurement* (§5.7's acceptance
criterion), authored audio, split-screen — all in §11 with what each would
buy and cost. The procedural audio hooks are implemented; authored assets
and production mixing remain open in `TODO.md`.

---

*Sections 1–6 (architecture, wire format, input handling, prediction,
latency budget, match lifecycle) live in
[`MULTIPLAYER_SPEC.md`](MULTIPLAYER_SPEC.md). A bare `§N` below refers to
that document for `N` 1–6, and to this one for `N` 7+.*

---

## 7. Phase and task breakdown

`[P]` parallelisable within its phase · `[D:x.y]` hard dependency

### Phases 0–6 — complete

Every task in Phases 0–6 is implemented and verified locally: non-networked
refactors, transport/connection/lobby, server-authoritative simulation with
a dumb client, input pipeline hardening, prediction and reconciliation for
ship and ball, match lifecycle, and dedicated-server productionisation
(Docker export, rotation/drain, CI). The two remaining gates on this work
are human verification, not code — see §0 gates A and B. Task-by-task
acceptance evidence for Phases 0–6 has been trimmed from this document;
`git log -- multiplayer-next.md` has the full history if a past task's
reasoning is needed.

### Phase 7 — Steam transport, browser, identity

**In progress.** GodotSteam requires custom engine builds and export
templates — **including for the headless server**; budget for it. The
`NetTransport` boundary (ENet + feature-gated `steam_transport.gd`) is
already extracted so this phase adds a second implementation rather than
retrofitting one.

| # | Task | Remaining |
|---|---|---|
| 7.1 `[D:1.2]` | GodotSteam integration and custom export templates, client *and* headless server | Awaiting the custom binaries/SDK access |
| 7.2 `[D:7.1]` | `NetTransport` Steam implementation (`SteamMultiplayerPeer`, SDR) | Server advertising waits for `ISteamGameServer` work |
| 7.3 `[D:7.2]` `[P]` | Server-browser UI and `ISteamMatchmakingServers` adapter | Unimplemented until real Steam SDK/API access is available; ENet direct-IP remains the supported browser-free path meanwhile |
| 7.4 `[D:7.2]` `[P]` | Auth tickets in `hello` → `BeginAuthSession`, Steam identity in the roster, persistent ban list | Durable ban storage landed: `identities.banned_until`/`ban_reason` are enforced on both session issuance and every authenticated request, and `ApplyIdentityBan` revokes an identity's sessions in the same transaction. Real GodotSteam auth integration and server-side VAC state remain (VAC state is read at login by the Web API adapter, but is not yet re-checked mid-session). **Fixes known defect C** for direct/community servers once landed |
| 7.5 `[D:7.2]` `[P]` | `SteamBootstrap` gating (stock builds keep ENet, explicit Steam selection fails closed) | Custom Steam client/server export smoke remains blocked on pinned GodotSteam binaries |
| 7.6 `[D:7.4]` | Backend `AuthCoordinator`, session persistence, `ControlPlaneClient.login_steam()`, real `ISteamUserAuth/AuthenticateUserTicket` adapter (`server/steam`), client web-API ticket acquisition, sign-in before matchmaking | Needs a real App ID and publisher key ([#15](https://github.com/jcreek/CosmicClash/issues/15)) and a custom GodotSteam build ([#16](https://github.com/jcreek/CosmicClash/issues/16)) to exercise live; sign-in is config-gated and returns 503 until both are set |
| 7.7 `[D:7.1]` `[P]` | Obtain the production App ID, publisher key, SDR coordinator SDK/signing approval, certificates and hosted-data-centre support from Valve | Not started |
| 7.8 `[D:7.6,7.7]` | Ticketed Hosted Dedicated Server SDR: routing registration, coordinator-issued player→server relay tickets, client ticket installation, reconnect and expiry | Not started; depends on 7.6 and 7.7 |

### Phase 8 — Matchmaking, ranked ladder, per-match server autoscaling

**1.0 launch blocker.** Full design and reasoning:
[`docs/MATCHMAKING.md`](docs/MATCHMAKING.md). This is the first phase to add
a component outside the Godot project — a Go backend service — and that is
the largest architectural departure in the project's history; read the
design doc before picking up any task below. The local control-plane,
durable-store, allocated-server, and verification paths are substantially
implemented; every row below is *intended* to list only what's still open,
not what's built — but see §0's audit note: rows drift toward understating
what has landed, so verify a row's claim against the code before planning
against it. **Task 8.31, formerly the critical path, is done — see §0.** What now
gates a live deployment is external: an App ID ([#15](https://github.com/jcreek/CosmicClash/issues/15)),
GodotSteam builds ([#16](https://github.com/jcreek/CosmicClash/issues/16)), and a
cluster ([#17](https://github.com/jcreek/CosmicClash/issues/17)).

**Hard dependency on 7.6 and 7.8.** The local allocated path binds slot
reclaim to a control-plane-signed player identity and locks its team/slot
pair, but production Steam ticket verification is still required before a
rating can be trusted. Production allocation also depends on the ticketed
Hosted Dedicated Server SDR route; ENet remains the local/CI/community
transport, not a silent production fallback.

This inverts the server model from Phases 1–7's **community server** (runs
forever, waits for `--min-players`, plays a match, rotates arena, repeats).
Matchmaking makes the *player* durable instead — queue, get grouped by
rating, and a server is **allocated for that one match** and destroyed
after. Both models ship; they are different playlists, not a replacement.

Tasks 8.1–8.4 (versioned contracts, state transitions, an ADR locking the
Go/PostgreSQL/Redis/Agones stack, and launch SLOs) and 8.11 (threat model)
are done; everything below is what's left on the tasks still open.

#### 8A — Architecture, contracts and data

| # | Task | Remaining |
|---|---|---|
| 8.5 `[D:8.4]` | PostgreSQL migrations 0001–0017 (idempotency, queue fencing, identities, ratings, matches, results, audits, outbox, allocator registry, proposal plans, leases, quotas, outbox dead-letter, retention indexes, allocation endpoints, probe challenges) | Verified against a live PostgreSQL; migrations now run to 0017. The local Docker storage exhaustion is a recurring symptom, not a one-off — see §9 gotcha on the integration scripts leaking anonymous volumes |
| 8.6 `[D:8.3,8.4]` | Allocated-mode `ServerConfig` fields | Allocated-mode fields are all present in `ServerConfig` (`allocated-mode`, `match-id`, `server-id`, `playlist`, `client-build`, `assignment-expiry-unix`, `server-image-digest`, `transport`, `region`, the join-authorisation file/key pair, `readiness-port`, `drain-token-env`). Signed-authorisation admission is implemented in `MatchNet` and was hardened with key-set rotation; dynamic endpoint wiring exists via `AssignmentState` and `connect_to_assignment()`. Only live runtime verification against a real cluster remains ([#17](https://github.com/jcreek/CosmicClash/issues/17)) |

#### 8B — Authentication and secure control plane

| # | Task | Remaining |
|---|---|---|
| 8.7 `[D:7.6,8.3]` | Ticket policy binding expected App ID/identity | Adapter, bans and secret store landed: `server/steam` calls `ISteamUserAuth/AuthenticateUserTicket`, rejects family-shared and banned accounts, and separates a Valve outage (503) from a bad ticket (401); the publisher key is mounted into the control-plane Deployment alone from the `cosmic-clash-steam` Secret, asserted by a manifest test. Only verification against real Valve remains, which needs the App ID and key ([#15](https://github.com/jcreek/CosmicClash/issues/15)) |
| 8.8 `[D:8.7]` | Session policy (opaque tokens, digests, revocation) | Distributed revocation coordination is done by construction: sessions are durable and `PostgresSessions.Authenticate` reads the row on every authenticated request, so a revocation takes effect immediately on every replica without any cross-replica protocol, and `ApplyIdentityBan` revokes an identity's sessions in the same transaction as the ban. Live Steam/session integration remains ([#15](https://github.com/jcreek/CosmicClash/issues/15)) |
| 8.9 `[D:8.4,8.7]` | Join policy, durable reconnect leases | Live PostgreSQL/Godot process-restart and outage recovery verification remains |
| 8.10 `[D:8.5,8.31]` | Workload credential policy (signed tokens, not Kubernetes JWTs), delivery channel, conflict alerting | Never run against a real Agones cluster; alert validated only statically, not against live Prometheus/Alertmanager traffic |
| 8.12 `[D:8.11]` | Kubernetes hardening baseline, rate/quota limiting, degraded-mode gate | Private-store provisioning, distributed/global quotas, edge DDoS/WAF/origin shielding, encrypted backups, live policy/load tests remain. The workload namespace currently enforces `privileged` because Agones' Dynamic port policy injects a `hostPort` that `baseline`/`restricted` forbid; splitting game servers into their own namespace so `cosmic-clash` can enforce `restricted` again is tracked by [#33](https://github.com/jcreek/CosmicClash/issues/33) |
| 8.13 `[D:8.12]` | Digest-pinned images, supply-chain policy checker | Registry SBOM/scan/sign/admission execution and a concrete production overlay remain — the build-and-pin half is tracked by [#31](https://github.com/jcreek/CosmicClash/issues/31) |

#### 8C — Queueing, matchmaking, playlists and rating

| # | Task | Remaining |
|---|---|---|
| 8.14 `[D:8.4,8.5,8.8]` | Queue policy (ownership, heartbeat/expiry, candidate projection) | Live Redis failover-under-load and worker integration remain |
| 8.15 `[D:7.8,8.3]` | Probe validation (RTT, nonce/freshness/region, quarantine), `POST /v1/probes/{region}/challenge`, durable single-use nonces, client probe collection before queueing, candidate-index refresh after probe | Steam coordinator ping-location source remains (a placeholder blob is sent without a Steam runtime); multi-region endpoint deployment remains |
| 8.16 `[D:8.14,8.15]` | Candidate/team formation, matcher worker | The matcher is a real long-running poll loop and now has casual and ranked Deployments in `deploy/k8s/base`; what remains is live soak against a cluster rather than the integration itself ([#17](https://github.com/jcreek/CosmicClash/issues/17)) |
| 8.17 `[D:8.14,8.16]` | Proposal policy (response window, cooldowns, offender/innocent split) | Live PostgreSQL execution and allocation integration remain |
| 8.18 `[D:8.5,8.14,8.17]` | Store layer (serializable retries, claim SQL, atomic promotion) | Allocation runtime integration remains |
| 8.19 `[D:8.18]` | Casual lineup (2–6 humans, bot backfill) | Candidate selection landed (`domain.SelectCasualBackfillCandidate`: oldest ordinary casual ticket meeting build/region/tolerance, ties by ticket ID, deterministic across replicas). Casual lineup formation was already built and wired, and all four penalty kinds are written durably. What remains is the backfill proposal itself, the matcher pass that finds vacated kickoff slots, the client offer UI, and **late roster delivery** — a backfilled player's authorisation is issued after their server started, and the supervisor fetches the roster once before launching the game child with no reload path. That delivery design is now decided (supervisor re-fetches and signals a reload; see `docs/MATCHMAKING.md` § Casual) and the remaining work is tracked in [#32](https://github.com/jcreek/CosmicClash/issues/32). End-to-end verification needs a live cluster ([#17](https://github.com/jcreek/CosmicClash/issues/17)) |
| 8.20 `[D:8.18]` | Ranked admission (six unique verified humans) | Done. Allocation wiring was already complete end to end (allocator sets the `cosmic-clash.io/arena-path` annotation → `supervisor.withAllocatedCompatibility` maps it to `--arena-path` → `server_boot.gd` → `ServerMatchLoop.allocated_arena_path`), with coverage at each hop. `ArenaRegistry` integration is now a cross-language guard rather than a shared list: `server/domain/ranked.go` must keep its own ranked-eligible subset (the choice is server-authoritative and made before any Godot process exists), so `arena_registry_sync_test.go` parses `arena_registry.gd` and fails if the two disagree in either direction, if rotation order diverges, or if a ranked path has no scene behind it. Verified against four drift scenarios including promoting an elevated variant, which the registry's own comment anticipates. Live ranked admission against a real cluster remains ([#17](https://github.com/jcreek/CosmicClash/issues/17)) |
| 8.21 `[D:8.5,8.20]` | Rating core (Glicko-2, weights, transactional updates) | Live maintenance/DB execution remains |
| 8.22 `[D:8.21]` | Ranked profile (provisional games, tiers) | Persisted tier policy done: bands live in `tier_bands`, seeded with the exact compiled launch policy so storage changed without behaviour changing, loaded at startup with a malformed policy failing startup rather than silently mis-tiering, and an empty table falling back to the compiled default so an operator can truncate back to known-good. Retuning is now a rolling restart rather than a rebuilt image. `PROVISIONAL` is rejected as a durable band, being derived from game count rather than rating. Client UI was already built (`RankedProfileState.display_text()` renders tier, provisional status, ranked games and the season countdown). Reconnect transport is tracked by 8.42 and depends on live auth/backend events |
| 8.23 `[D:8.21]` | Ranked season policy (compression, rollover) | Live maintenance/DB execution remains |
| 8.24 `[D:8.9,8.20,8.21]` | Ranked connection policy, reconnect lease, abandon ladder | Live PostgreSQL execution now verified (`make verify-phase6` and every integration script run clean). Process-restart and outage execution remain |
| 8.25 `[D:8.10,8.24]` | Result policy (workload-bound, idempotent, transactional) | Production credentials, Agones annotation persistence/reconciliation, integrity-evidence adapters remain |

#### 8D — Agones, allocation and regional scaling

| # | Task | Remaining |
|---|---|---|
| 8.26 `[D:8.1,8.6,8.12]` | Provider-neutral Fleet, EU/NA overlays, RBAC | Operator secret/image replacement, second-provider fixtures, edge/DNS, SDR POP/cert/public-UDP overlays remain |
| 8.27 `[D:8.26]` | Supervisor package (Agones discovery, Ready transition) | Metadata watch, real Agones annotation/shutdown confirmation, emulator integration remain |
| 8.28 `[D:8.6,8.27]` | Process-ready/Agones-Ready separation, control-plane registration | Remaining gates are live Agones annotation/shutdown behavior and production cluster readiness — see task 8.49 |
| 8.29 `[D:8.26,8.27]` | Dynamic port/SDR env propagation | Real Agones dynamic/passthrough mapping, POP/cert/firewall/NAT, multi-match fixture remain |
| 8.30 `[D:8.18,8.26,8.28,8.29]` | Allocation leasing, compatibility validation | Signed roster metadata landed with 8.31 — the allocator publishes one signed join authorisation per participant plus a manifest committing to a digest over the whole roster, and the supervisor materialises it before starting the game child. Full unknown-outcome cluster recovery remains ([#17](https://github.com/jcreek/CosmicClash/issues/17)) |
| **8.31** `[D:8.9,8.30]` | Signed assignment/roster persistence, player recovery | **Done — this was the root blocker of the allocation-to-connect pipeline.** `allocator.Worker.RunOnce` now builds one signed join authorisation per durable participant and calls `PublishRoster` after binding; `cmd/allocator` takes `--join-authorisations-key-file`/`--join-authorisations-key-id` and refuses to start without them. The signing design is settled: HMAC-SHA256 over the canonical claim bytes with a key ID inside them, so servers hold a key *set* and rotation does not invalidate in-flight matches. The provider endpoint is now persisted on the allocation so a worker crashing between allocating and publishing can retry. Verified by an integration test that drives the real worker through the supervisor's own roster read path without seeding `assignments`. Live Agones verification remains ([#17](https://github.com/jcreek/CosmicClash/issues/17)) |
| 8.32 `[D:8.2,8.26,8.30]` | FleetAutoscaler baseline, Ready buffer | Regional on-demand node pools/failure domains, pre-pull rollout, warm-allocation p95/p99, N+1 certification remain |
| 8.33 `[D:8.26,8.32]` | Fleet scheduling, zone spread | Regional node pools, forced node-loss testing, measured N+1 headroom remain |
| 8.34 `[D:8.28,8.29]` | Native x86_64 benchmark of boot-to-process-ready/assignment-ready, p99 CPU/RSS/network, node cap with 30% headroom | Not started |
| 8.35 `[D:8.17,8.19,8.20,8.30,8.31]` | Admission lease, disconnect/reconnect generations, no-show/bot policy | Live PostgreSQL execution, allocated process termination evidence, real Agones multi-client verification remain |
| 8.36 `[D:8.10,8.25,8.28,8.30]` | Authenticated drain, PodDisruptionBudget | Live 300 s/285 s lifecycle, PDB/Fleet drain, infrastructure-abort classification remain |
| 8.37 `[D:8.5,8.10,8.25,8.26,8.31]` | Horizontally scaled primary control plane + warm standby, EU/NA fleets, RPO ≤5 m/RTO ≤30 m | Not started |
| 8.38 `[D:7.7,8.26,8.36,8.37]` | Provider migration (restore, coordinator trust, switch allocations, drain old) | Not started; needs Valve approval for both providers' EU/NA POPs/certs and public UDP |

#### 8E — Client experience and recovery

| # | Task | Remaining |
|---|---|---|
| 8.39 `[D:8.3,8.14,8.17]` | `MatchmakingState`/`ControlPlaneClient`, queue/proposal UI, targeted revisioned events | Cross-replica fan-out landed: committed outbox events are published through PostgreSQL LISTEN/NOTIFY so the replica owning a subscriber's WebSocket delivers it, rather than whichever replica happened to drain the row. Verified against real PostgreSQL with two listeners. Live multi-replica verification under load remains |
| 8.40 `[D:8.3,8.14]` | Revisioned event stream, REST resync, outbox dispatcher | Allocator and Redis fan-out live verification remain |
| 8.41 `[D:7.8,8.9,8.31,8.40]` | Player-scoped assignment API, `connect_to_assignment()` wiring, join-authorisation verification in `MatchNet` | SDR relay-ticket installation and live Agones cluster integration remain |
| 8.42 `[D:8.22,8.23,8.24,8.40]` | `RankedProfileState`, backend-authoritative rating/tier display | Season countdown is implemented (`RankedProfileState.display_text()` renders the remaining days alongside tier, provisional status and ranked games). Committed revision after reconnect and abandon status remain dependent on live auth/backend events and Godot runtime verification |
| 8.43 `[D:8.39,8.40,8.41]` | Error/expiry UX, generic mutation retry, version-mismatch and failed-reconnect messaging | Long-running worker soak (§8.16) remains; the worker itself is deployed |

#### 8F — Observability, verification, cost and rollout

| # | Task | Remaining |
|---|---|---|
| 8.44 `[D:8.3,8.4,8.28,8.31]` | Structured logging, redaction | **Local complete; production gate open** — production metrics/traces backend and dashboard/alert routing remain |
| 8.45 `[D:8.2,8.44]` | SLO window checks, API latency histogram | **Local complete; production gate open** — production scrape configuration, alert routing, wait/MMR/proposal/flood/cost series, runbooks remain |
| 8.46 `[D:8.5,8.7,8.9,8.10,8.14,8.18,8.21,8.23,8.25]` | Go unit/race/fuzz coverage, local verification gate | Live matcher-worker-under-load-during-failover integration remains |
| 8.47 `[D:8.7,8.30]` | Offline testkit (fake Steam, fake allocation) | Live exhaustive matrix and production Steam remain |
| 8.48 `[D:8.10,8.14,8.17,8.18,8.27,8.31,8.35,8.47]` | Allocated Compose end-to-end (queue → proposal → allocation → assignment → result) | **Local complete; production gate open** — real Agones/kind and production evidence remain open |
| 8.49 `[D:8.25,8.26,8.28,8.29,8.30,8.31,8.35,8.36]` | Disposable kind+Agones cluster runner | CI/live evidence for production control-plane registration, roster/no-show, both readiness stages, races, multi-match node, result-pending reconciliation, drain, rollback remains open. Blocked locally on kind/Helm availability |
| 8.50 `[D:8.25,8.37,8.43,8.49]` | Chaos recovery (stale allocation, no-penalty requeue) | **Local complete; production gate open** — 100 ms RTT/jitter/loss, matcher/client restart, game-pod death, node drain, Redis failover, control-plane loss, live chaos evidence remain |
| 8.51 `[D:8.17,8.18,8.30,8.31,8.45]` | 10,000-client API load gate | **Local complete; production gate open** — PostgreSQL saturation, durable matcher fencing under load, forecast launch concurrency ×2, replica scaling remain live infrastructure gates |
| 8.52 `[D:8.32,8.34,8.45,8.51]` | Per-replica + shared regional allocator quota | Real image digest/secrets ([#31](https://github.com/jcreek/CosmicClash/issues/31)), measured regional cost model, threshold tuning, denial-of-wallet rehearsal remain |
| 8.53 `[D:7.8,8.13,8.38,8.45,8.46,8.48,8.49,8.50,8.51,8.52]` | Fail-closed release-gate promotion validator | Actual reports, production rollback rehearsal, regional playtests, live promotion remain open |

Implementation invariants for every task above:

- Matchmade mode is opt-in; every new `ServerConfig` default preserves the
  existing community-server path.
- `compose.phase6-smoke.yml`, `make verify-phase6`, and
  `make verify-enet-integration` are not repurposed or weakened.
- Production uses ticketed Hosted Dedicated Server SDR; ENet remains the
  deterministic local/CI and direct-IP path.
- One process serves one match. Warm processes/nodes absorb startup variance;
  capacity and cost are determined from 8.34 measurements, not old estimates.
- Design changes first update `docs/MATCHMAKING.md` and dependencies.

---

## 8. What needs refactoring, not extending

Historical note: this table described Phase 0's non-networked refactors,
all of which are now implemented (see §7's Phase 0 summary). Kept for the
underlying reasoning where it's still relevant to Phase 7/8 work touching
the same files.

**On `Engine.time_scale`:** replaced with camera-based effects in
single-player as well, so there is one code path and one game feel to
maintain rather than a networked variant that drifts away from the
single-player one.

**What does not need surgery:** the `ShipController` seam, the Arena/GameMode split, code-driven spawning, group-based discovery, and the dumb `Goal` sensor all extend cleanly. `CLAUDE.md`'s claim about the three load-bearing seams is accurate — they hold. `rl_ship_controller.gd` is *already* the remote-input controller (a public `action` field that something else writes, pulled each tick), so no new class was needed for it.

---

## 9. Godot 4.7 + Jolt gotchas

1. **`ENetMultiplayerPeer.server_relay` defaults to `true`** — clients can RPC each other through your server. Set it `false`.
2. **`MultiplayerAPI.poll()` runs on the idle frame**, so an `rpc()` from `_physics_process` waits up to a full frame — and `Engine.max_fps = 60` on the server is what creates that delay on the return leg. Take manual control. **~16–33 ms of round-trip, for ~10 lines.**
3. **Jolt sleeps bodies.** A ship corrected to near-zero velocity can sleep and then ignore `state.linear_velocity` writes. `can_sleep = false` on Ship and Ball.
4. **Teleporting a rigid body**: `state.transform` inside `_integrate_forces` is the only path with no frame of lag. `set_deferred("global_transform", …)` lands between frames and interacts badly with Jolt's sleep/wake ordering.
5. **`reset_physics_interpolation()` is not automatic for `state.transform` writes** (it is when you set `global_transform` directly). Call it explicitly, on the body **and** on `$Visual`.
6. **`physics_jitter_fix = 0.0` does not give you "a flat 60 Hz."** You still get occasional 0-tick and 2-tick frames, because frame time is never exactly 16.667 ms. The real reason to set it to 0 is that you never want a tick's input *delayed* by the accumulator smoother. **The send path must therefore transmit both ticks' actions on a 2-tick frame** — redundancy-4 covers this, but only if you actually send both.
7. **`_integrate_forces` is not called on frozen bodies**, so remote ships never pull `get_action()` — hence `set_visual_action`. Use `FREEZE_MODE_KINEMATIC`, **not `STATIC`**, or contact velocity transfer breaks.
8. **Never write `linear_velocity` to a frozen body** — Godot/Jolt zeroes and holds it.
9. **`Engine.max_physics_steps_per_frame` defaults to 8.** If a server tick overruns 16.7 ms the accumulator backs up and the next frame runs multiple ticks, spiking CPU further. Log overruns.
10. **ENet channel indices** are offset by Godot's reserved system channels — verify the mapping empirically.
11. **ENet peer timeout** defaults to ~5 s. Tune via `ENetPacketPeer.set_timeout()` for faster drop detection.
12. **Jolt is not bit-deterministic** across platforms or across differing contact orderings. Never rely on it anywhere, including in "obviously safe" places like a client-side goal check.
13. **`dedicated_server=true` exports strip visual resources.** Verify against a real stripped build.
14. **MTU**: ENet fragments above ~1400 B. At 219 B/snapshot there is ~6× headroom; recheck if per-body cosmetic state is ever added.
15. **RPC NodePath caching**: the first `rpc()` to a node sends the full path, later calls send a cached int. Routing hot paths through autoloads warms the cache once at connect and never invalidates it on scene change.
16. **Physics tick rate is 60 for v1 — and must never be a literal.** Every policy in `Game/bots/` is tick-coupled through `ship.gd:450`'s `_tick_scaled` (defined at a 60 Hz reference) and `ai_ship_controller.gd`'s `reaction_ticks`, so raising it toward Rocket League's 120 invalidates every trained model and halves server density. But it is the largest single latency term left (§5.4), so it *will* be revisited: everything derives from `TICK_HZ`, so that day is a config change plus a retrain.
17. **`Node3D.get_global_transform_interpolated()` is the only correct way to track a physics-interpolated body from `_process`.** `global_transform` returns the last physics tick's pose, so a per-frame camera reading it chases a 60 Hz staircase. Per the engine docs the method "creates an interpolation pump… the first time it is called" — **call it once before any `reset_physics_interpolation()` on that node**, or the first hard snap streaks (§4.5).
18. **Physics interpolation covers transforms only.** `camera.fov`, shader parameters, light energy and anything else written from `_physics_process` steps at 60 Hz on a 240 Hz display. Either write them from `_process` or accept the stepping deliberately.
19. **`display/window/vsync_mode` defaults to enabled (FIFO) and `max_fps` to uncapped.** Neither is set in `project.godot`. FIFO present latency is **1.5–3 refresh intervals** depending on swapchain image count (2 vs 3) and whether the present queue is full — §5's tables use the optimistic 1.5, which assumes the renderer is *not* GPU-bound. **The model does not hold below refresh**, where a missed vblank under strict FIFO halves the effective rate and roughly doubles present latency. Prefer **Adaptive** as the default, not Mailbox (§5.4). *(Swapchain image count per platform needs empirical verification.)*
20. **`Engine.max_fps` is a throttle, not a frame pacer.** It pads each frame with a post-frame sleep; it has no vblank phase lock. Caps that are not integer divisors of the refresh rate beat against scanout, and combining a cap with an active vsync paces *worse* than either alone (§5.4). Derive the offered caps from `DisplayServer.screen_get_refresh_rate()`.
21. **`DisplayServer.window_get_vsync_mode()` echoes your request, not the driver's grant.** There is no GDScript API for the negotiated `VkPresentModeKHR`, so a UI cannot honestly report what was applied. Show a live fps readout instead and let the player infer it.
22. **`Engine.max_physics_steps_per_frame = 8` is a client problem too**, not just a server one (gotcha 9). A client hitching to 20 fps runs 3 ticks per frame, and each of those frames also runs the per-frame camera rig and remote-visual sampling. Set it to 4 client-side. On a multi-tick frame the send path must transmit **every** tick's action (gotcha 6) — §4.3's `_physics_process` sampling does this naturally, but nothing else guarantees it.
23. **`hint_screen_texture` forces a full-screen backbuffer copy on every frame the node is drawn**, regardless of what the shader then does with it. Branching inside the shader saves taps, not the copy. Hide the node when the effect is at rest.
24. **`physics_jitter_fix` matters less the higher the frame rate.** Its purpose is smoothing when frame rate ≈ tick rate; at 240 fps against 60 Hz physics most frames run zero ticks and the accumulator is never near an edge. Gotcha 6's reasoning for setting it to `0.0` still holds, but do not expect a visible difference on a high-refresh machine — test that change at 60 fps.
25. **`MultiplayerAPI.multiplayer_peer`'s default value is an `OfflineMultiplayerPeer` sentinel, not `null`.** Resetting it with `multiplayer_peer = null` (rather than a fresh `OfflineMultiplayerPeer.new()`) leaves the API in a state distinct from its own default and is a known source of "the server never sees `peer_connected`, `get_peers()` stays empty" bugs (godotengine/godot#81540). Always reset to a real `OfflineMultiplayerPeer`.
26. **Don't tear down a peer the instant its own connect signal fires.** `connected_to_server` (client-side) fires once the client's *local* view of the handshake completes, but the final ACK the server needs to consider *its* side complete may not have hit the wire yet — closing the peer or quitting the process in the same callback can drop it, and the other side then never sees `peer_connected`/`connected_to_server` at all, even though your own side looked successful. This reproduced on **every** attempt until fixed and is easy to misdiagnose as a server-side bug (the server-side symptom — `get_peers()` staying empty — is identical to gotcha 25's). Give at least one frame (in practice `tests/net_smoke.gd` uses 0.3 s) between a fresh connect signal and calling `shutdown()`/`quit()`.
27. **`change_scene_to_file()` must be called on (or from a descendant of) the actual `get_tree().current_scene`, and never synchronously from `_ready()`.** (a) instantiating a scene as a plain child of a driver node, rather than loading it as the real current scene, breaks its own disconnect-handling `change_scene_to_file()` calls with a silent hang. (b) calling `change_scene_to_file()` (or `add_child()` on `get_tree().root`) synchronously from inside `_ready()` throws "Parent node is busy … Consider using `.call_deferred()`", because the tree is still mid-traversal adding the very node whose `_ready()` is running.
28. **`ENetMultiplayerPeer`'s `connection_failed` signal is not bounded to anything a UI should make a player wait for.** Verified empirically: against a genuinely refused loopback connection, `connection_failed` had still not fired 14 seconds in. Don't rely on it alone to end a "Connecting…" state — run your own app-level timeout.
29. **A `MultiplayerPeer`'s "am I a client" flag turns true the instant `join()`/`create_client()` is called, not once the connection actually completes.** Anything gated on that flag alone will try to `rpc_id()` on a peer that's still `CONNECTING` — or has already failed. Gate on the peer's actual `get_connection_status() == MultiplayerPeer.CONNECTION_CONNECTED`, not just the higher-level intent flag.
30. **`load()` on a `.gd` file with a parse/compile error does not return `null`.** It returns a non-null but uninstantiable `GDScript` resource, so `if script == null` silently fails to catch the failure. The real guard is `Script.can_instantiate()`.
31. **An `@rpc` method named `_input` collides with `Node`'s built-in `_input(event: InputEvent)` virtual.** On an autoload especially, treat any bare virtual-sounding method name (`_input`, `_process`, `_ready`, `_unhandled_input`, …) as reserved regardless of what you intend it to do — a signature mismatch there doesn't fail locally, it fails the whole autoload.
32. **Disabling automatic multiplayer polling is global, not autoload-scoped — every scene that touches an RPC, not just `NetworkManager`-adjacent code, must call `NetworkManager.poll()` itself every frame it wants traffic to move.** If a scene sends or receives RPCs and nothing arrives with no errors at all, check for a missing `poll()` before anything else.
33. **A request/response fallback for a one-shot broadcast can double-deliver, and the receiving handler must be idempotent.** Any handler reachable via both an original broadcast and a "resend on request" path needs its own guard rather than assuming "only sent once" from the RPC design alone.
34. **An `Area3D`'s `body_entered` signal fires as part of physics tick N's own step, strictly *before* tick N's `_physics_process` callback — not "on the next frame."** If you actually need next-tick-or-later semantics, compare `Engine.get_physics_frames()` against the tick the flag was set on and require strictly-greater, not just "check a boolean at the top of `_physics_process`."
35. **A queued `queue_teleport()` can take one tick longer to land than "the very next `_integrate_forces`" suggests, when the call originates from a signal handler mid-physics-step rather than from a `_physics_process` callback.** Don't assume queued-teleport timing without checking a real tick-by-tick log for your specific call site.
36. **`NetworkManager.get_server_time_estimate_ms()` (and anything derived from it) is not just imprecise before the first clock pong lands — it's actively wrong in a way that can persist far longer than the sync window implies.** Always gate recording, not just consuming, anything derived from this estimate on `rtt_ms >= 0.0`.
37. **Anything that deliberately delays an RPC dispatch must re-validate its target at *fire* time, not just at the moment it was scheduled.** Any deliberate-delay layer needs to re-check both "do I still have a real peer at all" and "is this specific target still in `get_peers()`" inside the delayed callback itself, immediately before dispatching.
38. **A GDScript lambda captures an enclosing local variable BY VALUE at the moment the lambda is created, not by reference.** Capture a container instead of a value — `var disconnected := [false]` and `disconnected[0] = true` inside the lambda — since capturing an `Array`/`Dictionary`/`Object` captures a reference to the same instance.
39. **A fixed-size ring buffer fed by an unbounded-rate producer needs an explicit resync path, not just "wait for the next expected slot."** If a ring's producer and consumer rates aren't provably bounded relative to each other, the consumer needs a way to detect "the data I'm waiting for no longer exists in the ring at all" and jump directly to what's still available, rather than assuming "keep waiting" is always eventually correct.
40. **A client-owned adaptive control loop must react to the actual ground-truth signal it's regulating, not to its own memory of past decisions.** When a control loop's condition for acting can be satisfied or blocked by state the loop itself controls, rather than by the environment it's meant to respond to, it can silently stop responding to the environment.
41. **A "consecutive N over-budget windows" streak counter that hard-resets to 0 on any single clean window is trivially evaded by a duty-cycled attacker.** A leaky-bucket accumulator is immune to the same evasion by construction, since it doesn't matter how the excess is distributed in time — only the sustained average matters.
42. **Two counters that don't share an epoch must never be compared directly, even when both are monotonically increasing integers that "look like" the same kind of thing.** Bound a value against another value that shares its own actual epoch, not against a same-typed number from a conceptually different clock.
43. **A regression test that doesn't independently exercise the specific mechanism it claims to gate will pass even when that mechanism is completely broken.** A CI gate's assertions should trace back to the specific claim in the task's own acceptance text, not just "the match ran and didn't crash."
44. **When adding a "is this connection still healthy" check to a test, sample it while the peer is still actively connected — not after its own end-of-run disconnect, which produces symptoms indistinguishable from the bug being checked for.**
45. **Two fixes landed in the same commit, each individually correct in isolation, can share a variable and silently cancel each other out — and a fix's own unit test can miss it by testing the mechanism in isolation from the thing that defeats it.** When two fixes in the same round touch the same subsystem, explicitly re-test the combination end-to-end — passing tests for each fix individually is not evidence the pair composes correctly.
46. **A guard that bounds an incoming value against the *consumer's* position, rather than against the *producer's* own epoch, re-introduces exactly the "consumer can never catch up past a stall" failure it's often added specifically to prevent.** Bound against whichever side of a producer/consumer pair is not the one already known to be falling behind.
47. **A trace that holds its inputs steady cannot falsify anything about *which sequence* a prediction is filed under — and "we hold thrust for 60 seconds" describes almost every prediction test people write.** When a test is meant to validate a label, an index, or a phase relationship rather than a magnitude, the trace has to change that quantity frequently; a steady-state trace validates the magnitude and silently asserts nothing about the label.
48. **A guard whose bound is derived from a value only the ACCEPTED path can advance is a latch, not a guard.** The property to check when writing a guard like this is not "is the bound correct?" but "**if this guard rejects everything from now on, what advances the bound?**" If the answer is "an accepted packet", it needs an independent escape path regardless of how well-chosen the bound is.
49. **Advancing a consumer cursor past data that has not arrived is not a lossy shortcut — it is permanent, because the producer-side filter then rejects the very data being waited for.** **Only give up on an expected item once strictly newer data proves it lost**; "it hasn't arrived yet" and "it will never arrive" are different states and must not share a code path.
50. **A metric that stops sampling during a failure will report that failure as healthy.** Every rate-shaped assertion needs a companion assertion on the **denominator**, or an outage silently becomes an absence of evidence and then evidence of absence.
51. **An architectural blocker inherited from a previous session is a claim to verify, not a premise to build on.** Reconstruct the failing invariant from the code and reproduce it against a control before accepting a scope estimate attached to it — especially when the recommendation arrives without the cheaper alternative recorded as tested.
52. **`docker run --rm` reclaims the container, not its anonymous volumes.** Every run of `scripts/run_*_integration.sh` leaves a throwaway PostgreSQL/Redis data volume behind. They accumulate invisibly — 64 of them, ~4 GB, after one working session — until the Docker VM disk fills and the next container silently fails to start, surfacing only as the script's own `PostgreSQL did not become ready` timeout rather than as a disk error. This is the actual cause behind the "Docker storage exhausted locally" notes elsewhere in this document. `docker system df` shows it (`Local Volumes … 100% reclaimable`); `docker volume prune` clears it. Worth checking first whenever an integration script starts timing out on a machine where it previously worked. **Fixed** by adding `-v` to each script's cleanup trap: `--rm` does reclaim anonymous volumes on a normal exit, but these scripts force-remove the container from a trap instead, and `docker rm -f` without `-v` keeps the volume. Verified as one leaked volume per run before, zero after.

---

## 10. Testing

**Editor.** Debug → Run Multiple Instances, 2–3 instances with per-instance args (`-- --server`, `-- --connect 127.0.0.1:27015`) and `--position` so windows don't stack.

**CLI.**
```bash
godot --headless --path Game res://scenes/server_boot.tscn -- --port 27015 --team-size 1 --auto-start
godot --path Game -- --connect 127.0.0.1:27015 --name Alice
```

**CI smoke test.** Headless server plus two headless `--test-bot` clients, driven by the existing `AIShipController`. Asserts:
- snapshots received ≥ `N * snapshot_hz * 0.9`
- own-ship prediction error p95 < 0.5 m, p99 < 2.0 m, hard-snap count < 3
- final score identical on the server and both clients
- no `push_error` emitted (scrape stderr)

**Network conditions.** `net_sim.gd` is first-class: seeded so failures reproduce, works in CI, needs no display, and can be applied *asymmetrically* — which OS tools make painful. `tc netem` / Network Link Conditioner / `clumsy` for a pre-release realism pass only. A real remote host once per phase from Phase 4 onward is the only true test of the jitter buffer's adaptivity.

**Unit tests.** `godot --headless --path Game res://tests/test_runner.tscn`. High-value targets, all zero-engine-state: codec quantise/dequantise round-trip and bounds; quaternion max error; snapshot pack→unpack identity; input packet framing; jitter-buffer policy against scripted arrival traces; `ShipAction.copy()` non-aliasing. These are exactly where a bug is invisible in play and catastrophic in aggregate.

---

## 11. Flagged, not solved

**120 Hz simulation** — deliberately deferred, not dismissed. §5.4 and §5.6 record what it would buy (≈21 ms of world response once L1 has taken the interpolation buffer out, plus ≈8 ms of own-ship feel — the difference between ≈127 ms and ≈107 ms), what it costs (a full bot retrain, half the server density, double the bandwidth), and the one rule that keeps the door open: `TICK_HZ`, never `60`.

**The latency gap to the reference has a plan but not yet an implementation.** §5.2 lands at ≈174 ms as designed; §5.6 routes that to ≈127 ms and ≈103 ms (tasks L1–L4 plus 120 Hz simulation), against ~90–110 ms for the reference class at the same RTT. Every figure in §5.6 is arithmetic on the budget, not a measurement or a shipped change — L1–L4 remain unimplemented. Beyond that the residual is RTT, which is a server-siting problem (§6, Phase 8) rather than a code one and is worth more than every remaining code lever combined.

**Audio.** The runtime has dependency-free procedural placeholder hooks for UI, countdown, engine/thrust/turbo, impacts, wall contacts, goals, and camera/gameplay events. `TODO.md` tracks authored engine/turbo/impact/wall/goal/crowd/music assets and production mixing/QA as still open.

**Split-screen.** Tracked separately in `TODO.md`; unrelated to this effort, though the camera-outside-the-ship structure that enables it is the same structure this plan relies on.

**Graphics: baked GI (task 0.26) and low/mid-tier hardware profiling.** See §5.5 and §5.7 — real but smaller wins than originally assumed on reference-class desktop hardware; unmeasured on low-end/integrated GPUs.

**Not locally certifiable from this workspace, and open prerequisites rather than done:** Valve/GodotSteam credentials and hosted SDR (Phase 7 tasks 7.1–7.8), live Agones/kind lifecycle (tasks 8.30–8.38, 8.49), public-network chaos/load/cost/release gates (tasks 8.50–8.53), and real-hardware graphics profiling on low/mid-tier GPUs. `make verify-kind-agones` is the committed runner for 8.49; it has not yet completed a full run against a real cluster from this workspace (blocked on local Docker/kind/Helm resource availability, not a code gap). `TODO.md`'s AI-training and presentation tasks remain separate from multiplayer.