jcreek/CosmicClash
1
0
Fork 0
mirror of https://github.com/jcreek/CosmicClash.git synced 2026-09-10 16:04:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(multiplayer): fail closed without durable leases

Browse Source
This commit is contained in:
Josh Creek
2026-09-03 21:04:07 +01:00
parent 947fefc95c
commit a4de140424
3 changed files with 10 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Game/scripts/server_boot.gd
+7 -1
View File
@@ -116,7 +116,13 @@ func _ready() -> void:
else:
_connection_leases.queue_free()
_connection_leases = null
ServerLog.warn("connection_lease_backend_unavailable", {"reason": "invalid_or_missing_configuration"})
# Allocated matches must never fall back to an in-memory connection
# generation. Doing so would admit a player without the durable fence
# that prevents a second process (or a stale peer) from owning the same
# ranked slot. Direct/community servers do not enter this branch.
printerr("cosmic-clash-server: refusing allocated startup without connection-lease configuration")
get_tree().quit(1)
return
NetworkManager.client_connected.connect(_on_client_connected)
NetworkManager.client_disconnected.connect(_on_client_disconnected)
Game/tests/cases/test_server_config.gd
+2
View File
@@ -178,6 +178,8 @@ func test_allocated_start_floor_is_the_verified_roster_size() -> void:
func test_connection_reporting_requires_safe_workload_configuration() -> void:
var boot = preload("res://scripts/server_boot.gd")
assert_true(boot.valid_connection_report_configuration("http://control-plane:8080", "signed-token", "match-1234567890", "server-123456789", "player-123456789"), "allocated workload configuration is accepted")
assert_true(not boot.valid_connection_report_configuration("", "signed-token", "match-1234567890", "server-123456789", "player-123456789"), "allocated startup fails closed without a control-plane lease URL")
assert_true(not boot.valid_connection_report_configuration("http://control-plane:8080", "", "match-1234567890", "server-123456789", "player-123456789"), "allocated startup fails closed without a workload lease credential")
assert_true(not boot.valid_connection_report_configuration("http://control-plane:8080?token=leak", "signed-token", "match-1234567890", "server-123456789", "player-123456789"), "query-bearing control-plane URL is rejected")
assert_true(not boot.valid_connection_report_configuration("http://control-plane:8080", "token\nforged", "match-1234567890", "server-123456789", "player-123456789"), "header injection token is rejected")
assert_true(not boot.valid_connection_report_configuration("http://control-plane:8080", "signed-token", "short", "server-123456789", "player-123456789"), "non-opaque match identity is rejected")
multiplayer-next.md
+1 -1
View File
@@ -1192,7 +1192,7 @@ production fallback.
|---|---|---|
| 8.7 `[D:7.6,8.3]` | **IN PROGRESS.** Pure Go ticket policy binds the expected App ID and verified identity, rejects expiry/replay/wrong app/malformed tickets, and consumes each ticket once. `cmd/control-plane` now wires `SessionIssuer: store.PostgresSessions{DB: db}` (same discovery/fix pattern as §8.10's `ResultSubmitter`: the adapter already correctly implemented `Issue`, just wasn't wired, so `/v1/session/steam` 503'd even before considering whether `SteamLogin` — the real, still-correctly-unwired Steam blocker — was available) | `server/domain/auth.go` covers single-use and binding invariants; real `AuthenticateUserTicket` backend adapter, bans, publisher secret store and Steam verification remain. A separate test-only `server/cmd/testkit-api` binary (never referenced by any Dockerfile/K8s manifest) substitutes a fake Steam login accepting any non-empty ticket, enabling §8.40's real Go+Postgres+Godot integration test |
| 8.8 `[D:8.7]` | **IN PROGRESS.** Pure Go session policy issues opaque short-lived tokens, stores only digests, authenticates by verified player identity and supports revocation; the production control-plane uses bounded atomic account+IP request limits | `server/domain/auth.go` covers wrong-token/expiry/revocation behavior; `server/store/session_sql.go` provides durable digest/revocation persistence and `server/api/rate_limit.go` plus `cmd/control-plane` provide per-replica request limiting; distributed revocation coordination and live Steam/session integration remain |
| 8.9 `[D:8.4,8.7]` | **IN PROGRESS.** Pure Go join policy binds SteamID/player/match/server/team/slot/protocol/expiry, rejects duplicate roster slots, permits same-identity reclaim through backend loss and fences prior server-owned generations. A workload-authenticated durable lease API atomically claims generations and records exact-generation disconnects against the allocation/match/server/participant roster. Allocated Godot admission now awaits one bounded durable claim before publishing the roster entry; definitive conflicts fail closed, while a known nonzero same-process generation may reconnect during an outage and queues its connect/disconnect sequence for ordered reconciliation. A fresh process never guesses generation one offline and can adopt a later backend generation only from a durably disconnected lease | `server/domain/reconnect.go`, `server/store/server_connection_sql.go`, migration 0011, `/servers/{serverId}/{connect|disconnect}`, `connection_lease_client.gd`, and adversarial tests cover active duplicate claims, stale disconnect fencing, exact 60-second reclaim, process recovery, wrong binding, initial assignment expiry, malformed/skipped responses, ordered outage rules, retry-safe active receipts, and migration backfill. Admission rechecks drain, token expiry, and peer presence after the awaited claim and releases a claim that became unusable. Live PostgreSQL/Godot process-restart and outage recovery verification remains |
| 8.9 `[D:8.4,8.7]` | **IN PROGRESS.** Pure Go join policy binds SteamID/player/match/server/team/slot/protocol/expiry, rejects duplicate roster slots, permits same-identity reclaim through backend loss and fences prior server-owned generations. A workload-authenticated durable lease API atomically claims generations and records exact-generation disconnects against the allocation/match/server/participant roster. Allocated Godot startup now fails closed without valid control-plane lease configuration, and admission awaits one bounded durable claim before publishing the roster entry; definitive conflicts fail closed, while a known nonzero same-process generation may reconnect during an outage and queues its connect/disconnect sequence for ordered reconciliation. A fresh process never guesses generation one offline and can adopt a later backend generation only from a durably disconnected lease | `server/domain/reconnect.go`, `server/store/server_connection_sql.go`, migration 0011, `/servers/{serverId}/{connect|disconnect}`, `connection_lease_client.gd`, and adversarial tests cover missing workload configuration, active duplicate claims, stale disconnect fencing, exact 60-second reclaim, process recovery, wrong binding, initial assignment expiry, malformed/skipped responses, ordered outage rules, retry-safe active receipts, and migration backfill. Admission rechecks drain, token expiry, and peer presence after the awaited claim and releases a claim that became unusable. Live PostgreSQL/Godot process-restart and outage recovery verification remains |
| 8.10 `[D:8.5,8.31]` | **IN PROGRESS.** Dependency-free workload credential policy validates an adapter-verified signature, issuer/audience/time bounds, namespace/service account, pod UID, GameServer UID, allocation ID, match ID and server ID before result submission; `server/workload` now parses projected compact JWTs, rejects unsigned/malformed/ambiguous tokens, verifies signing input through an injected trust callback, and delegates exact binding to the domain policy; the API exposes the workload-authenticated server result route. **`WorkloadVerify` is now wired for real** -- the blocker this row previously named (`cmd/control-plane/main.go` never wiring it, so `/register` and `/result` 503'd on every real request) is closed by taking a different, equally-valid approach instead of the Kubernetes-JWT one: `serverMutation` only ever compares `WorkloadBinding.ServerID`/`.MatchID` (and `AdvanceServerRegistration` only additionally needs `.AllocationID`) -- nothing downstream requires Namespace/ServiceAcct/PodUID/GameServerUID, so a Kubernetes-issued token was never actually required, just the assumption that it was. `server/workload/signed_token.go` mints and verifies a short-lived HMAC-signed token with a secret only the control plane holds (the same trust model `domain.SessionStore` already uses for player sessions), needing no cluster JWKS/TokenReview to validate -- signature and expiry are fully self-contained. The token binds only `allocation_id` (deliberately, not `match_id`/`server_id` too: it must be requestable in the same Agones request that asks for a server, before Agones has picked one) -- `store.AllocationBindingByAllocationID` resolves `match_id`/`server_id` durably from the `allocations` table at verify time, so a token can never claim a pairing that wasn't actually, durably allocated. `api.WorkloadVerifierFromSignedToken` combines both and is wired into `cmd/control-plane` (`--workload-secret`/`COSMIC_CLASH_WORKLOAD_SECRET`; a startup warning fires and the route stays 503 if it's left unset) and `cmd/testkit-api` (fixed test secret). **The delivery channel is now wired too**: `agones.Client` gains `WorkloadSecret`/`WorkloadTokenTTL` -- when set, `Allocate` mints a token for the allocation and requests it as a third `cosmic-clash.io/workload-token` annotation alongside match-id/allocation-id, wired from `cmd/allocator`'s own `--workload-secret`/`COSMIC_CLASH_WORKLOAD_SECRET` (must match the control plane's); `supervisor.Supervisor.workloadToken()` resolves the bearer credential with an explicit `--workload-token-path` (kept for a possible future Kubernetes-JWT path) always winning, otherwise falling back to that same annotation -- the same fallback pattern `matchID()` already used for `cosmic-clash.io/match-id`. `WorkloadTokenPath` is accordingly no longer required at construction time | `server/domain/workload.go`, `server/workload/jwt.go`, `server/api/service.go` and adversarial tests reject every binding mutation, missing/unverified signature, `none`/malformed JWT, ambiguous audience, server/match mismatch and time boundary; `server/testkit/pipeline_test.go` carries allocation identity through the offline result path; `cmd/control-plane` now wires `ResultSubmitter: store.PostgresResults{DB: db}` (a ready-made adapter that had been referenced from nowhere at all, not even a test); `server/workload/signed_token_test.go` covers round-trip, tampered payload, wrong secret, malformed input and the exact expiry boundary; `server/api/workload_verifier_integration_test.go` (opt-in, real PostgreSQL) covers acceptance against a real allocation row, rejection of an unrecorded allocation, that two distinct real allocations each resolve to their own and only their own match/server pairing, and the previously-503 `Service.WorkloadVerify` field itself now succeeding -- all verified clean with `-race` across multiple runs; `cmd/control-plane/main_test.go`'s `TestServerRoutesRequireWorkloadVerifyToBeWired` now documents and pins the misconfigured-secret case specifically, not the "always unwired" case; `server/agones/allocation_test.go`'s `TestAllocateRequestsAWorkloadTokenAnnotationWhenConfigured` and `server/supervisor/supervisor_test.go`'s new annotation-fallback/fails-closed pair cover the delivery channel end to end (mint -> annotation -> supervisor read -> Authorization header) short of a live cluster. **What's still missing**: this has never run against a real Agones cluster (only HTTP-level fakes), so the exact `object_meta` JSON casing Agones actually returns remains unverified from this sandbox (see `GameServer.ObjectMeta`'s existing caveat); live duplicate/conflict alerting also remains |
| 8.11 `[D:8.1]` | **DONE.** Write the threat model: forged clients/replay/queues/results, floods/bots, pod/insider compromise, gameplay and API DDoS, SDR signing-key theft, dependencies and denial-of-wallet | [`docs/THREAT-MODEL.md`](docs/THREAT-MODEL.md) records prevention, detection/response, owner and residual risk for every threat; it separates offline CA/online signer, client/game-server/PostgreSQL/Redis trust boundaries and recovery behavior |
| 8.12 `[D:8.11]` | **IN PROGRESS.** Provider-neutral Kubernetes baseline enforces restricted namespace admission, non-root/read-only/no-capability workloads, separate service accounts, default-deny networking and explicit edge/data/DNS/Agones/metrics flows; application manifests consume externally populated Secret objects; the Go API and allocator now have bounded per-replica account+IP rate/quota boundaries and metrics endpoints, plus an atomic operator-controlled degraded-mode gate that sheds new login/queue/proposal work while preserving live-match paths | `deploy/k8s/base/` includes hardened control-plane, allocator, and game-server resources; the control-plane now has health probes, a disruption budget, explicit zero-unavailable/one-surge rolling updates with graceful termination, failure-domain spreading/anti-affinity, and Secret-backed `COSMIC_CLASH_POSTGRES_DSN`/`COSMIC_CLASH_WORKLOAD_SECRET` runtime wiring; the authenticated WebSocket now requires RFC 6455 version 13, enforces a bounded 64 KiB frame size, two-minute idle deadline, 120-message/minute inbound budget, and bounded per-player fan-out; `server/security/test_kubernetes_policies.py`, `server/api/rate_limit.go`, `server/api/admission.go`, allocator metrics tests and adversarial checks cover static hardening, secret-reference invariants, fixed-window limits, bounded labels/key memory, atomic account+IP charging, route classification, concurrent toggling and degraded responses; production `cmd/control-plane` now wires the bounded limiter by default with `--rate-limit`, `--rate-limit-window`, and `--rate-limit-max-keys`, and the base Deployment declares those defaults explicitly; start with control-plane `--degraded`, enable with SIGUSR1, disable with SIGUSR2; private-store provisioning, distributed/global quotas, edge DDoS/WAF/origin shielding, encrypted backups and live policy/load tests remain |