copilot_messages had no column preserving message order: created_at (set
from each message's timestamp) ties at millisecond granularity in 58% of
chats, and some chats have out-of-order timestamps within their array. The
only other tiebreaker, id, is a random UUID — so ORDER BY created_at, id
renders same-timestamp user/assistant pairs swapped. This blocks the R+1
read cutover.
Add an integer seq = the message's 0-based index within the chat's JSONB
array (ground-truth order), backfilled inline in migration 0219 (no script
for self-hosters or us). Reads will use ORDER BY seq NULLS LAST, created_at,
id at cutover; reads still come from JSONB after this PR.
Design:
- seq is a tiebreaker, not the sole sort key (concurrent-append/NULL safety).
- Nullable now; defer NOT NULL so rolling-deploy old pods don't fail inserts.
- replace (update-messages snapshot) overwrites seq = array index
(re-densifies after a mid-conversation delete); append preserves existing
seq via COALESCE and assigns base+idx from a single MAX(seq) read (never
MAX+i in SQL — multi-row batches would collide). The non-atomic
read-then-insert window is documented and bounded by the read tiebreak +
snapshot re-densify.
- Dedupe message ids before insert (87 prod chats carry dup ids; a repeated
id in one INSERT...ON CONFLICT would otherwise throw).
- Backfill picks first-occurrence per (chat,id), gap-free via ROW_NUMBER;
validated on staging data (0-based, contiguous, 0 bad ranges).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
