docs: define matchmaking launch SLOs

This commit is contained in:
Josh Creek
2026-08-31 20:07:41 +01:00
parent 62ee3f2777
commit af8592082e
4 changed files with 45 additions and 2 deletions
+39
View File
@@ -0,0 +1,39 @@
# Matchmaking launch SLOs
These are the measurable release gates for the Phase 8 control plane. All
latency measurements use server-side monotonic timestamps and are labelled by
region, playlist, build, transport and warm/cold capacity. A request or match
is counted only after the corresponding terminal event is durably recorded.
| SLO | Metric and denominator | Window / target | Owner | Alert threshold |
| --- | --- | --- | --- | --- |
| Placement eligibility | `max(predicted_rtt_ms)` across all accepted players in a candidate | Every candidate; `<=100 ms` | Matcher | Any eligible candidate over 100 ms pages immediately and is rejected |
| Regional observed RTT | p95 of server-observed handshake/game RTT for connected assigned players | Rolling 1 h, per region; `<=80 ms` | Game-server + networking | 15 min above 80 ms, or any region p95 above 100 ms for 5 min |
| Acceptance → assignment-ready | `assignment_ready - unanimous_accept` for accepted proposals with warm capacity | Rolling 1 h; p95 `<=5 s`, p99 `<=10 s` | Allocator | p95 >5 s for 10 min or p99 >10 s for 5 min |
| Assignment → successful connection | `connected - assignment_published` for assignments not cancelled by policy | Rolling 1 h; p95 `<=5 s` | Game-server lifecycle | p95 >5 s for 10 min or connection success <99% for 5 min |
| Allocation + durable result | completed matches with both successful allocation and durable result / matches requiring allocation | Rolling 24 h; `>=99.9%` | Allocator + maintenance | <99.95% warning; <99.9% pages and blocks release |
| Server tick health | Physics ticks completed without backlog / expected physics ticks; resource headroom is measured independently | Every live match; zero backlog and `>=30%` CPU/RSS headroom | Game-server | Any sustained backlog, or headroom <30% for 5 min |
| Control-plane API | p95 request latency for non-streaming authenticated API requests, excluding client cancellation | Rolling 5 min, by route; `<=250 ms` | API | p95 >250 ms for 5 min or 5xx >1% |
## Measurement rules
- Do not combine EU and NA into one percentile; a healthy region must not hide
an unhealthy one.
- Exclude explicitly rejected requests from success denominators, but count
accepted work that later expires, fails allocation, or loses result delivery.
- Preserve queue, proposal, match, server and request IDs on every metric and
trace. Never attach Steam auth tickets, SDR relay tickets, publisher keys or
other credentials to labels, logs or traces.
- Warm-capacity SLOs are evaluated only when the region has the declared Ready
floor. Cold-start and capacity-exhaustion outcomes are separate dashboards,
not silently removed from availability accounting.
- Alert thresholds page the owning role; the release gate is the stricter
target in the table, not the warning threshold.
## Release evidence
A release candidate must provide one complete 24-hour report, route-level API
histograms, regional RTT histograms, allocation/connection cohort counts,
tick-health samples, and an incident review for every SLO breach. Load and
chaos tests must retain the same event IDs so the report can distinguish
retryable control-plane delay, player no-show, and match-integrity failure.
+3
View File
@@ -418,6 +418,9 @@ Minimum controls:
## 8. SLOs, observability, and release gates
The operational definitions, owners, windows and alert thresholds for these
targets are maintained in [`MATCHMAKING-SLOs.md`](MATCHMAKING-SLOs.md).
Launch SLOs:
| Measure | Target |
+2 -1
View File
@@ -31,7 +31,8 @@ product policy are in [`docs/MATCHMAKING.md`](docs/MATCHMAKING.md).
- [x] Lock the Go/PostgreSQL/Redis, Kubernetes/Agones, SDR, EU/NA and
provider-portability ADR ([ADR-001](docs/ADR-001-matchmaking-platform.md));
measurable launch SLOs remain task 8.2.
measurable launch SLOs are defined in
[MATCHMAKING-SLOs.md](docs/MATCHMAKING-SLOs.md).
- [ ] Publish versioned OpenAPI/WebSocket contracts, stable IDs, legal state
transitions, revisions and idempotency semantics.
- [ ] Add PostgreSQL queue ownership/active-participation fences, durable
+1 -1
View File
@@ -1169,7 +1169,7 @@ the local/CI/community transport, not a silent production fallback.
| # | Task | Acceptance |
|---|---|---|
| 8.1 | Add an ADR locking **Go + PostgreSQL + Redis**, provider-portable Kubernetes, Agones, ticketed Hosted Dedicated Server SDR, EU/NA fleets and independently runnable API, matcher, allocator and maintenance roles; keep `README.md`/`docs/TECH_STACK.md` consistent | The ADR names boundaries/rejected alternatives; current docs name the locked stack and replaceable provider; no application code calls a provider allocation API |
| 8.2 `[D:8.1]` | Encode the launch SLOs from `docs/MATCHMAKING.md`: RTT, allocation/connect latency, 99.9% allocation/result success, API latency and tick health | Each SLO has a metric, numerator/denominator, percentile window, owner and alert threshold before implementation is judged against it |
| 8.2 `[D:8.1]` | **DONE.** Encode the launch SLOs from `docs/MATCHMAKING.md`: RTT, allocation/connect latency, 99.9% allocation/result success, API latency and tick health | [`docs/MATCHMAKING-SLOs.md`](docs/MATCHMAKING-SLOs.md) defines each metric, denominator, percentile/window, owner, alert threshold and release evidence |
| 8.3 `[D:8.1]` | Publish versioned OpenAPI + WebSocket contracts for Steam login/session, profile/rating, queue create/heartbeat/cancel/resume, proposal accept/decline, assignment/status, server registration/roster/result/shutdown | Generated contract tests cover every request, response, event and external error; clients can REST-resync after a missed WebSocket revision |
| 8.4 `[D:8.3]` | Define opaque `player_id`, `queue_ticket_id`, `proposal_id`, `match_id`, `server_id`, `season_id`, legal queue/match state transitions, revisions and idempotency keys | Duplicate/out-of-order commands converge; invalid transitions are rejected without partial state |
| 8.5 `[D:8.4]` | Add PostgreSQL migrations for durable queue ownership, active-participation fencing, identities, sessions/revocations, seasons, ratings/events, matches/participants, penalties, results, audits and outbox; document Redis caches/TTLs | A blank DB migrates up; lost Redis writes cannot resurrect revocation, split a proposal or corrupt durable state; rollback/forward compatibility is tested |