Files
CosmicClash/server/allocator/roster.go
T
Josh Creek 5765532409 fix(allocator): publish signed assignment rosters before servers start
The root blocker (issue #14). The worker bound the provider allocation
and stopped. Service.PublishRoster and store.SaveVerifiedAssignmentRoster
both existed, fully tested, with zero non-test callers, and the
production allocator configured neither a roster store nor a signing
key. Nothing ever wrote the assignments table.

The allocated supervisor fetches a non-empty roster before it launches
the game child, so every real allocation failed at that fetch: no match
could reach ASSIGNMENT_READY or accept a player. Existing tests seeded
assignments directly, which is exactly why the missing hand-off went
unnoticed.

The worker now builds one join authorisation per durable participant,
signs each with the active key, and publishes them. Participants are
read through the same query SaveVerifiedAssignmentRoster re-validates
against, so the allocator cannot construct a roster the persistence
boundary would reject. The manifest commits to a digest over the whole
roster, so a server cannot be handed a truncated roster whose surviving
entries are each individually valid.

Persist the provider endpoint on the allocation: it arrived on the
provider response and was never stored, so a worker crashing between
allocating and publishing had no endpoint to recover and would have
stranded the match permanently. Republishing is idempotent, so that
crash now simply retries.

cmd/allocator refuses to start without key material rather than running
an allocator that binds allocations and silently strands every match.
The k8s allocator Deployment mounts the same key set the Fleet does, and
both now take the JSON key map so a rotation can publish several.

New integration test drives the real worker through to the supervisor's
own roster read path without seeding the assignments table. Verified it
fails with "assignments = 0, want 2" when the publish step is removed.
2026-09-05 10:42:31 +01:00

100 lines
3.8 KiB
Go

package allocator
import (
"context"
"fmt"
"strconv"
"time"
"github.com/cosmic-clash/cosmic-clash/server/domain"
)
// JoinAuthorisationLifetime bounds how long an issued authorisation may be
// replayed. It must outlive the initial-connect window (a player still loading
// must be able to join) without leaving a usable credential lying around after
// the match it belongs to is over.
const JoinAuthorisationLifetime = 30 * time.Minute
// AssignmentRosterSource reads the authoritative participants of an allocated
// match. It is deliberately the same query the persistence boundary
// re-validates against, so the allocator cannot construct a roster that
// disagrees with the durable match_participants rows.
type AssignmentRosterSource interface {
LoadAssignmentParticipants(context.Context, domain.Allocation) ([]domain.AssignmentParticipant, error)
}
// JoinSigningKeys is the allocator's key material. ActiveKeyID names the key
// new authorisations are signed with; Keys holds every currently-valid key so
// verification (including the re-check at the persistence boundary) still
// accepts authorisations issued before a rotation.
type JoinSigningKeys struct {
ActiveKeyID string
Keys map[string][]byte
}
func (k JoinSigningKeys) validate() error {
if k.ActiveKeyID == "" || len(k.Keys) == 0 {
return fmt.Errorf("join signing keys are not configured")
}
if len(k.Keys[k.ActiveKeyID]) == 0 {
return fmt.Errorf("active join signing key %q is not present in the key set", k.ActiveKeyID)
}
return nil
}
// BuildSignedRoster turns the durable participants into one signed join
// authorisation each, plus the manifest that commits to the whole set.
//
// Signing each entry proves each individual claim; the manifest's roster
// digest additionally commits to the set, so a server cannot be handed a
// truncated roster whose surviving entries are each individually valid.
func BuildSignedRoster(allocation domain.Allocation, participants []domain.AssignmentParticipant, keys JoinSigningKeys, now time.Time) (domain.Assignment, []domain.SignedJoinAuthorisation, error) {
if err := keys.validate(); err != nil {
return domain.Assignment{}, nil, err
}
if allocation.State != domain.ServerAllocated || allocation.Endpoint == "" || len(participants) == 0 || now.IsZero() {
return domain.Assignment{}, nil, domain.ErrManifestRejected
}
active := keys.Keys[keys.ActiveKeyID]
roster := make([]domain.SignedJoinAuthorisation, 0, len(participants))
for _, participant := range participants {
signed, err := domain.SignJoinAuthorisationHMAC(domain.JoinAuthorisation{
MatchID: allocation.MatchID,
ServerID: allocation.ServerID,
PlayerID: participant.PlayerID,
SteamID: participant.SteamID,
Slot: participant.Slot,
Team: participant.Team,
Protocol: strconv.Itoa(allocation.Protocol),
// Generation 1 is the first connection lease. Reconnects fence by
// advancing the durable generation, not by reissuing this token.
Generation: 1,
ExpiresAt: now.Add(JoinAuthorisationLifetime).UTC(),
KeyID: keys.ActiveKeyID,
}, active)
if err != nil {
return domain.Assignment{}, nil, fmt.Errorf("sign join authorisation for %s: %w", participant.PlayerID, err)
}
roster = append(roster, signed)
}
rosterDigest, err := domain.AssignmentRosterDigest(roster)
if err != nil {
return domain.Assignment{}, nil, err
}
assignment := domain.Assignment{
Allocation: allocation,
Endpoint: allocation.Endpoint,
Manifest: domain.AllocationManifest{
AllocationID: allocation.AllocationID,
MatchID: allocation.MatchID,
ServerID: allocation.ServerID,
Region: allocation.Region,
Build: allocation.Build,
Protocol: allocation.Protocol,
Transport: allocation.Transport,
RosterDigest: rosterDigest,
},
}
return assignment, roster, nil
}