diff --git a/multiplayer-next.md b/multiplayer-next.md index 7561bcac..2198a402 100644 --- a/multiplayer-next.md +++ b/multiplayer-next.md @@ -93,7 +93,9 @@ product policy are in [`docs/MATCHMAKING.md`](docs/MATCHMAKING.md). inert/alerting. Pure Go credential-claim validation, binding, hashing, reconciliation, and the atomic receipt/completion/outbox SQL boundary exist; projected-token/JWT adapters, trusted-cluster verification, rating-lock - integration, and production alerting remain. A dependency-free projected JWT + integration, and production alerting remain. The API now exposes the + workload-authenticated server result route and delegates completion to the + durable receipt/outbox adapter. A dependency-free projected JWT adapter now verifies the compact-token signature through an injected trust boundary and delegates exact claim/time binding to the domain policy. - [x] Complete the threat model for forgery, replay, queue/flood/bot abuse, diff --git a/multiplayer-todo.md b/multiplayer-todo.md index dda795b8..ff16ca0f 100644 --- a/multiplayer-todo.md +++ b/multiplayer-todo.md @@ -1183,7 +1183,7 @@ the local/CI/community transport, not a silent production fallback. | 8.7 `[D:7.6,8.3]` | **IN PROGRESS.** Pure Go ticket policy binds the expected App ID and verified identity, rejects expiry/replay/wrong app/malformed tickets, and consumes each ticket once | `server/domain/auth.go` covers single-use and binding invariants; real `AuthenticateUserTicket` backend adapter, bans, publisher secret store and Steam verification remain | | 8.8 `[D:8.7]` | **IN PROGRESS.** Pure Go session policy issues opaque short-lived tokens, stores only digests, authenticates by verified player identity and supports revocation | `server/domain/auth.go` covers wrong-token/expiry/revocation behavior; distributed revocation, account/IP limits, request limits and production session persistence remain | | 8.9 `[D:8.4,8.7]` | **IN PROGRESS.** Pure Go join policy binds SteamID/player/match/server/team/slot/protocol/expiry, rejects duplicate roster slots, permits same-identity reclaim through backend loss and fences prior server-owned generations | `server/domain/reconnect.go` covers SteamID/server/slot binding, expiry, repeated reclaim, grace boundary and old-generation fencing; signed token issuance/verification, persistent leases, Godot `hello` transport and production integration remain | -| 8.10 `[D:8.5,8.31]` | **IN PROGRESS.** Dependency-free workload credential policy validates an adapter-verified signature, issuer/audience/time bounds, namespace/service account, pod UID, GameServer UID, allocation ID, match ID and server ID before result submission; `server/workload` now parses projected compact JWTs, rejects unsigned/malformed/ambiguous tokens, verifies signing input through an injected trust callback, and delegates exact binding to the domain policy | `server/domain/workload.go`, `server/workload/jwt.go` and adversarial tests reject every binding mutation, missing/unverified signature, `none`/malformed JWT, ambiguous audience and time boundary; `server/testkit/pipeline_test.go` carries allocation identity through the offline result path; trusted-cluster key verification and live duplicate/conflict alerting remain | +| 8.10 `[D:8.5,8.31]` | **IN PROGRESS.** Dependency-free workload credential policy validates an adapter-verified signature, issuer/audience/time bounds, namespace/service account, pod UID, GameServer UID, allocation ID, match ID and server ID before result submission; `server/workload` now parses projected compact JWTs, rejects unsigned/malformed/ambiguous tokens, verifies signing input through an injected trust callback, and delegates exact binding to the domain policy; the API exposes the workload-authenticated server result route | `server/domain/workload.go`, `server/workload/jwt.go`, `server/api/service.go` and adversarial tests reject every binding mutation, missing/unverified signature, `none`/malformed JWT, ambiguous audience, server/match mismatch and time boundary; `server/testkit/pipeline_test.go` carries allocation identity through the offline result path; trusted-cluster key verification, live duplicate/conflict alerting and production result wiring remain | | 8.11 `[D:8.1]` | **DONE.** Write the threat model: forged clients/replay/queues/results, floods/bots, pod/insider compromise, gameplay and API DDoS, SDR signing-key theft, dependencies and denial-of-wallet | [`docs/THREAT-MODEL.md`](docs/THREAT-MODEL.md) records prevention, detection/response, owner and residual risk for every threat; it separates offline CA/online signer, client/game-server/PostgreSQL/Redis trust boundaries and recovery behavior | | 8.12 `[D:8.11]` | **IN PROGRESS.** Provider-neutral Kubernetes baseline enforces restricted namespace admission, non-root/read-only/no-capability workloads, separate service accounts, allocator-only RBAC, default-deny networking and explicit edge/data/DNS/Agones flows; application manifests consume externally populated Secret objects; the Go API now has an optional bounded per-replica rate-limit/429 boundary | `deploy/k8s/base/` plus `server/security/test_kubernetes_policies.py`, `server/api/rate_limit.go` and adversarial tests cover static hardening, secret-reference invariants, fixed-window limits and bounded key memory; private-store provisioning, distributed/global quotas, edge DDoS/WAF/origin shielding, WebSocket limits, overload shedding, encrypted backups and live policy/load tests remain | | 8.13 `[D:8.12]` | **IN PROGRESS.** Docker/Kubernetes references are digest-pinned, a dependency-free checker rejects mutable tags/plaintext credentials and concrete release overlays can reject template digests; release documentation defines SBOM, dependency/image scanning, signing, admission verification and a 24-hour critical-fix SLA | `scripts/verify_supply_chain.py`, `server/security/test_supply_chain.py`, `docs/SUPPLY-CHAIN.md` and `.github/workflows/supply-chain.yml` cover repository policy and provenance requirements; registry SBOM/scan/sign/admission execution and a concrete production overlay remain | @@ -1203,7 +1203,7 @@ the local/CI/community transport, not a silent production fallback. | 8.22 `[D:8.21]` | **IN PROGRESS.** Pure Go ranked profile exposes the first ten games as provisional, derives tiers only through validated backend-owned rating bands, and keeps casual ratings outside the API; authenticated HTTP now returns the authoritative ranked view | `server/domain/rating.go`, `tier_test.go` and `server/api/service.go` cover provisional override, exact band boundaries, malformed policy rejection, session authentication and ranked-only response fields; persisted tier policy, client UI and reconnect transport remain | | 8.23 `[D:8.21]` | **IN PROGRESS.** Pure Go ranked-only season policy compresses 25% toward 1500, clamps RD to 200–350, preserves volatility/history, is idempotent by season ID, and defines exact 12-week windows/due detection; migration and Go store now persist a per-player/per-season marker and rating update atomically; `cmd/maintenance` runs bounded due-season batches with signal-bound shutdown | `server/domain/rating.go`, `season_test.go`, `server/migrations/0001_initial.sql` and `server/store/maintenance_sql.go` cover compression, floor/cap, duplicate replay, window boundary, completed-season idempotence, bounded enumeration, row locking and conflict-safe rollover markers; opt-in PostgreSQL execution now covers the durable rating update, marker creation and duplicate replay without a second compression; live maintenance/DB execution remains | | 8.24 `[D:8.9,8.20,8.21]` | **IN PROGRESS.** Pure Go ranked connection policy binds match/server/player/team/slot/protocol, supports 60 s reclaim with server-owned generations, fences old connections, and applies the rolling 7-day 5 m/15 m/1 h/24 h abandon ladder; canonical signed authorisation issuance/verification now gates admission | `server/domain/reconnect.go`, `join_auth.go` and adversarial fixtures cover repeated backend-independent reclaim, all signed claim binding, tampering, failed verification, old-generation fencing, grace boundary and deterministic cooldown audit ordering; persistent lease fencing, join transport and full match/result integration remain | -| 8.25 `[D:8.10,8.24]` | **IN PROGRESS.** Pure Go result policy binds match/server/workload identity, hashes canonical payloads, makes identical retries idempotent, leaves conflicts inert, separates integrity eligibility, classifies roster/simulation/result/fairness evidence, validates annotation signatures/digests, and exposes 5 m alert/30 m review delivery thresholds; Go store SQL now executes the receipt → match lock → completion → receipt acknowledgment → outbox boundary atomically, and exposes bounded ordered outbox reads plus publish acknowledgements for replayable fan-out; `OutboxDispatcher` now delivers in order and acknowledges only after successful fan-out | `server/domain/result.go`, `workload.go`, `server/store/result_sql.go` and `outbox.go` plus adversarial fixtures cover credential binding, duplicate/conflict, annotation forgery, delivery-outage-versus-integrity classification, commit ordering, idempotent SQL reconciliation, unpublished-event replay/ack boundaries and delivery-before-ack failure ordering; opt-in PostgreSQL execution now covers result-pending completion, durable receipt/outbox publication, ack removal, identical replay and conflicting replay rejection; production credential verification, Agones annotation persistence/reconciliation, rating-lock integration and integrity evidence adapters remain | +| 8.25 `[D:8.10,8.24]` | **IN PROGRESS.** Pure Go result policy binds match/server/workload identity, hashes canonical payloads, makes identical retries idempotent, leaves conflicts inert, separates integrity eligibility, classifies roster/simulation/result/fairness evidence, validates annotation signatures/digests, and exposes 5 m alert/30 m review delivery thresholds; Go store SQL now executes the receipt → match lock → completion → receipt acknowledgment → outbox boundary atomically, and exposes bounded ordered outbox reads plus publish acknowledgements for replayable fan-out; `OutboxDispatcher` now delivers in order and acknowledges only after successful fan-out; the API now validates workload-bound server result submissions before invoking this durable boundary | `server/domain/result.go`, `workload.go`, `server/workload/jwt.go`, `server/api/service.go`, `server/store/result_sql.go` and `outbox.go` plus adversarial fixtures cover credential binding, duplicate/conflict, annotation forgery, server/match mismatch, delivery-outage-versus-integrity classification, commit ordering, idempotent SQL reconciliation, unpublished-event replay/ack boundaries and delivery-before-ack failure ordering; opt-in PostgreSQL execution now covers result-pending completion, durable receipt/outbox publication, ack removal, identical replay and conflicting replay rejection; production credential verification, Agones annotation persistence/reconciliation, rating-lock integration and integrity evidence adapters remain | #### 8D — Agones, allocation and regional scaling diff --git a/server/api/service.go b/server/api/service.go index 1a8a6fd9..d5a037e8 100644 --- a/server/api/service.go +++ b/server/api/service.go @@ -29,6 +29,10 @@ type ProbeProvider func(playerID, region string, opaqueLocation, nonce []byte, r type ProbeRecorder interface { RecordProbe(context.Context, string, string, time.Duration, time.Time) error } +type WorkloadVerifier func(string, time.Time) (domain.WorkloadBinding, error) +type ResultSubmitter interface { + SubmitResult(context.Context, string, domain.MatchResult, domain.WorkloadBinding, []byte, time.Time) error +} type QueueBackend interface { Create(context.Context, string, string, string, domain.QueueSpec, time.Time) (domain.QueueTicket, error) @@ -90,6 +94,8 @@ type Service struct { CandidateIndex CandidateIndex Probe ProbeProvider ProbeRecorder ProbeRecorder + WorkloadVerify WorkloadVerifier + ResultSubmitter ResultSubmitter Assignment AssignmentProvider Now func() time.Time Proposals map[string]*domain.Proposal @@ -113,6 +119,7 @@ func (s *Service) Handler() http.Handler { mux.HandleFunc("/v1/profile/ranked", s.rankedProfile) mux.HandleFunc("/v1/probes/", s.probe) mux.HandleFunc("/v1/events", s.controlPlaneEvent) + mux.HandleFunc("/v1/servers/", s.serverMutation) // The public contract is served below /api/v1. Keep the original /v1 // routes for the Godot client while exposing the documented names. mux.HandleFunc("/api/v1/session/steam", s.steamSession) @@ -122,6 +129,7 @@ func (s *Service) Handler() http.Handler { mux.HandleFunc("/api/v1/proposals/", s.contractProposalMutation) mux.HandleFunc("/api/v1/assignments/", s.contractAssignment) mux.HandleFunc("/api/v1/events", s.controlPlaneEvent) + mux.HandleFunc("/api/v1/servers/", s.contractServerMutation) if s.RateLimiter == nil { return mux } @@ -351,6 +359,82 @@ func (s *Service) contractAssignment(w http.ResponseWriter, r *http.Request) { s.assignment(w, clone) } +func (s *Service) contractServerMutation(w http.ResponseWriter, r *http.Request) { + path := strings.TrimPrefix(r.URL.Path, "/api/v1/servers/") + if path == "" || strings.Contains(path, "/") { + writeError(w, http.StatusNotFound, "not_found") + return + } + clone := r.Clone(r.Context()) + clone.URL.Path = "/v1/servers/" + path + s.serverMutation(w, clone) +} + +type resultRequest struct { + MatchID string `json:"match_id"` + ResultNonce string `json:"result_nonce"` + Score struct { + Team0 int `json:"team_0"` + Team1 int `json:"team_1"` + } `json:"score"` + IntegrityState domain.IntegrityState `json:"integrity_state"` +} + +func (s *Service) serverMutation(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + writeError(w, http.StatusMethodNotAllowed, "method_not_allowed") + return + } + parts := strings.Split(strings.TrimPrefix(r.URL.Path, "/v1/servers/"), "/") + if len(parts) != 2 || parts[0] == "" || parts[1] != "result" { + writeError(w, http.StatusNotFound, "not_found") + return + } + if s.WorkloadVerify == nil || s.ResultSubmitter == nil { + writeError(w, http.StatusServiceUnavailable, "result_unavailable") + return + } + key := r.Header.Get("Idempotency-Key") + if len(key) < 16 || len(key) > 128 { + writeError(w, http.StatusBadRequest, "invalid_idempotency_key") + return + } + partsAuth := strings.Fields(r.Header.Get("Authorization")) + if len(partsAuth) != 2 || partsAuth[0] != "Bearer" || partsAuth[1] == "" { + writeError(w, http.StatusUnauthorized, "unauthorized") + return + } + now := s.now() + binding, err := s.WorkloadVerify(partsAuth[1], now) + if err != nil || binding.ServerID != parts[0] { + writeError(w, http.StatusUnauthorized, "unauthorized") + return + } + var input resultRequest + if !decodeBody(w, r, &input) { + return + } + if input.MatchID == "" || binding.MatchID != input.MatchID || len(input.ResultNonce) < 16 || len(input.ResultNonce) > 128 || input.Score.Team0 < 0 || input.Score.Team1 < 0 || (input.IntegrityState != domain.IntegrityCertified && input.IntegrityState != domain.IntegritySuppressed && input.IntegrityState != domain.IntegrityReview) { + writeError(w, http.StatusUnprocessableEntity, "invalid_request") + return + } + result := domain.MatchResult{MatchID: input.MatchID, ServerID: parts[0], ResultNonce: input.ResultNonce, Team0Score: input.Score.Team0, Team1Score: input.Score.Team1, IntegrityState: input.IntegrityState} + payload, err := json.Marshal(input) + if err != nil { + writeError(w, http.StatusBadRequest, "invalid_request") + return + } + if err := s.ResultSubmitter.SubmitResult(r.Context(), key, result, binding, payload, now); err != nil { + if errors.Is(err, domain.ErrResultConflict) || strings.Contains(err.Error(), "conflict") { + writeError(w, http.StatusConflict, "conflict") + } else { + writeError(w, http.StatusUnprocessableEntity, "invalid_request") + } + return + } + w.WriteHeader(http.StatusAccepted) +} + func (s *Service) queueMutation(w http.ResponseWriter, r *http.Request) { if r.Method != http.MethodPost && r.Method != http.MethodGet && r.Method != http.MethodDelete { writeError(w, http.StatusMethodNotAllowed, "method_not_allowed") diff --git a/server/api/service_test.go b/server/api/service_test.go index cf4e5c97..7f5a7894 100644 --- a/server/api/service_test.go +++ b/server/api/service_test.go @@ -34,6 +34,19 @@ type probeRecorderSpy struct { } } +type resultSubmitterSpy struct { + calls int + err error + key string + result domain.MatchResult +} + +func (r *resultSubmitterSpy) SubmitResult(_ context.Context, key string, result domain.MatchResult, _ domain.WorkloadBinding, _ []byte, _ time.Time) error { + r.calls++ + r.key, r.result = key, result + return r.err +} + func (p *probeRecorderSpy) RecordProbe(_ context.Context, player, region string, rtt time.Duration, _ time.Time) error { p.calls++ p.last.player, p.last.region, p.last.rtt = player, region, rtt @@ -916,6 +929,40 @@ func TestRankedProfileAPIReturnsBackendTierAndHidesCasualData(t *testing.T) { } } +func TestServerResultAPIRequiresBoundWorkloadAndDelegatesDurableSubmission(t *testing.T) { + now := time.Unix(1000, 0).UTC() + binding := domain.WorkloadBinding{MatchID: "match-1", ServerID: "server-1"} + submitter := &resultSubmitterSpy{} + service := &Service{Now: func() time.Time { return now }, WorkloadVerify: func(token string, at time.Time) (domain.WorkloadBinding, error) { + if token != "workload-token" || !at.Equal(now) { + t.Fatalf("verifier input=%q %v", token, at) + } + return binding, nil + }, ResultSubmitter: submitter} + server := httptest.NewServer(service.Handler()) + defer server.Close() + body := `{"match_id":"match-1","result_nonce":"nonce-1234567890","score":{"team_0":3,"team_1":2},"integrity_state":"CERTIFIED"}` + req, _ := http.NewRequest(http.MethodPost, server.URL+"/v1/servers/server-1/result", strings.NewReader(body)) + req.Header.Set("Authorization", "Bearer workload-token") + req.Header.Set("Idempotency-Key", "result-key-123456") + response, err := http.DefaultClient.Do(req) + if err != nil || response.StatusCode != http.StatusAccepted { + t.Fatalf("status=%v err=%v", response.StatusCode, err) + } + response.Body.Close() + if submitter.calls != 1 || submitter.key != "result-key-123456" || submitter.result.Team0Score != 3 { + t.Fatalf("submission=%+v calls=%d", submitter, submitter.calls) + } + req, _ = http.NewRequest(http.MethodPost, server.URL+"/v1/servers/server-2/result", strings.NewReader(body)) + req.Header.Set("Authorization", "Bearer workload-token") + req.Header.Set("Idempotency-Key", "result-key-123456") + response, err = http.DefaultClient.Do(req) + if err != nil || response.StatusCode != http.StatusUnauthorized { + t.Fatalf("wrong server status=%v err=%v", response.StatusCode, err) + } + response.Body.Close() +} + func TestProbeAPIUsesServerEvidenceAndRejectsClientRTTField(t *testing.T) { now := time.Unix(1000, 0).UTC() sessions := domain.NewSessionStore() diff --git a/server/domain/result.go b/server/domain/result.go index 498f744b..b8c9c689 100644 --- a/server/domain/result.go +++ b/server/domain/result.go @@ -79,6 +79,10 @@ type ResultReceipt struct { CommittedAt time.Time } +// ResultDigest exposes the canonical payload digest to transport adapters; +// callers still need the domain validation and workload binding policy. +func ResultDigest(result MatchResult) [32]byte { return resultDigest(result) } + type ResultStore struct { expected WorkloadBinding receipts map[string]ResultReceipt diff --git a/server/store/result_sql.go b/server/store/result_sql.go index 242362c0..499970d4 100644 --- a/server/store/result_sql.go +++ b/server/store/result_sql.go @@ -51,6 +51,16 @@ WHERE player_id = ANY($1) ORDER BY player_id FOR UPDATE` +type PostgresResults struct{ DB *sql.DB } + +func (r PostgresResults) SubmitResult(ctx context.Context, resultID string, result domain.MatchResult, binding domain.WorkloadBinding, payload []byte, now time.Time) error { + if r.DB == nil || resultID == "" || binding.ServerID == "" || binding.MatchID != result.MatchID || binding.ServerID != result.ServerID || len(payload) == 0 || now.IsZero() { + return fmt.Errorf("invalid result submission") + } + receipt := domain.ResultReceipt{ResultID: resultID, MatchID: result.MatchID, ResultNonce: result.ResultNonce, PayloadDigest: domain.ResultDigest(result), IntegrityState: result.IntegrityState, ReceivedAt: now} + return CompleteResult(ctx, r.DB, receipt, binding.ServerID, resultID, payload, now) +} + // CompleteResult is the durable receipt/reconciliation boundary. The caller // must have already authenticated the workload and computed the receipt // digest. Duplicate identical receipts continue the same completion path;