Files
CosmicClash/server/migrations/0014_outbox_dead_letter.sql
T
Josh Creek 1dd05c75f1 fix(server): repair the initial-connect outbox envelope and unblock dispatch
ApplyInitialConnectPlan wrote a payload of {match_id,state,action},
omitting event, revision, resource_id, occurred_at and player_ids --
every field deliverStateOutboxEvent requires. Delivery rejected the row,
dispatch returned on the first error so it was never acknowledged, and
because reads are ordered oldest-first it was retried ahead of every
later state_changed event on every 100ms poll. One initial-connect
transition therefore blocked lifecycle delivery for all matches, not
just its own.

Two independent fixes, since either alone leaves the system fragile:

Build envelopes through one validating helper (MarshalOutboxEnvelope)
and convert all five writers to it. A writer that omits a required
field now fails its own transaction instead of committing a row that
can only ever poison the queue. The helper takes revision as int64 so
the -1 "nothing matched" sentinel some CTEs return surfaces as an error
rather than wrapping to a huge uint64.

Make dispatch resilient regardless: a delivery failure is now counted
against that row and the batch continues, with the row dead-lettered
after MaxOutboxDeliveryAttempts so a poison event degrades to one lost
notification instead of a stalled queue. Ordering within an aggregate
is still honoured -- later events of a failed match are deferred, so no
client observes that match's newer state before its older state. An ack
failure still stops the batch, being a database rather than a payload
problem.

Initial-connect events now address every participant, not just the
connected ones: a no-show needs to learn their ticket was failed and a
penalty applied.
2026-09-05 10:17:16 +01:00

16 lines
684 B
SQL

ALTER TABLE outbox
ADD COLUMN delivery_attempts INTEGER NOT NULL DEFAULT 0,
ADD COLUMN last_delivery_error TEXT,
ADD COLUMN dead_lettered_at TIMESTAMPTZ;
-- The unpublished dispatchers read oldest-first and previously stopped on the
-- first delivery error, so one permanently malformed payload blocked every
-- later event of that type forever. Dead-lettered rows leave the working set
-- via this partial index so a poison row degrades to one lost event instead of
-- a stalled queue.
DROP INDEX IF EXISTS outbox_unpublished_order;
CREATE INDEX outbox_unpublished_order
ON outbox (created_at, event_id)
WHERE published_at IS NULL AND dead_lettered_at IS NULL;