mirror of
https://github.com/jcreek/CosmicClash.git
synced 2026-09-10 16:04:04 +00:00
Josh Creek
5453e19761
Each ten-second queue heartbeat mints a fresh idempotency key and permanently inserts a row. Published outbox rows and expired/revoked sessions were never purged either -- the maintenance role performed lifecycle reconciliation only. At 10,000 queued players heartbeats alone add roughly 60,000 durable rows per minute, so table and index growth, vacuum pressure, backup size and recovery time were all unbounded on a service intended to scale horizontally. Add retention windows chosen to exceed every retry and recovery horizon that could still consult the row -- deleting an idempotency key early would turn a client replay into a second real mutation, so this is a correctness bound, not just a housekeeping one. Dead-lettered outbox rows are kept longest, being the record of events never delivered. Deletes run in bounded SKIP LOCKED batches so a purge never blocks live traffic, never holds a long transaction, and concurrent maintenance replicas do not contend. Indexes back each predicate so a pass cannot degrade into a sequential scan of the table it is bounding. The maintenance role reports rows purged, the backlog past its window (deletion lag), and any dead-lettered events. Also make the migration-rollback test derive its step counts instead of hardcoding them: adding a migration silently shifted the fixed counts so the failure surfaced as an unrelated "0006 rollback did not drop matches.allocation_id".
25 lines
1016 B
SQL
25 lines
1016 B
SQL
-- Retention support. Three tables grow without bound today:
|
|
--
|
|
-- idempotency_keys -- the client heartbeats every 10s and mints a fresh key
|
|
-- each time, so at 10,000 queued players this alone adds roughly 60,000
|
|
-- rows per minute, forever.
|
|
-- outbox -- published rows are never purged.
|
|
-- sessions -- expired and revoked rows are never purged.
|
|
--
|
|
-- The maintenance role performed lifecycle reconciliation only, so storage,
|
|
-- index size, vacuum pressure, backup size and recovery time all grew without
|
|
-- limit on a service meant to scale horizontally.
|
|
--
|
|
-- These indexes exist to make the deletion predicates cheap; without them each
|
|
-- purge pass would sequentially scan the very tables it is trying to bound.
|
|
|
|
CREATE INDEX IF NOT EXISTS idempotency_keys_created_at
|
|
ON idempotency_keys (created_at);
|
|
|
|
CREATE INDEX IF NOT EXISTS outbox_published_at
|
|
ON outbox (published_at)
|
|
WHERE published_at IS NOT NULL;
|
|
|
|
CREATE INDEX IF NOT EXISTS sessions_expires_at
|
|
ON sessions (expires_at);
|