feat(copilot): add seq ordinal to copilot_messages for order-preserving reads (#4791)

copilot_messages had no column preserving message order: created_at (set
from each message's timestamp) ties at millisecond granularity in 58% of
chats, and some chats have out-of-order timestamps within their array. The
only other tiebreaker, id, is a random UUID — so ORDER BY created_at, id
renders same-timestamp user/assistant pairs swapped. This blocks the R+1
read cutover.

Add an integer seq = the message's 0-based index within the chat's JSONB
array (ground-truth order), backfilled inline in migration 0219 (no script
for self-hosters or us). Reads will use ORDER BY seq NULLS LAST, created_at,
id at cutover; reads still come from JSONB after this PR.

Design:
- seq is a tiebreaker, not the sole sort key (concurrent-append/NULL safety).
- Nullable now; defer NOT NULL so rolling-deploy old pods don't fail inserts.
- replace (update-messages snapshot) overwrites seq = array index
  (re-densifies after a mid-conversation delete); append preserves existing
  seq via COALESCE and assigns base+idx from a single MAX(seq) read (never
  MAX+i in SQL — multi-row batches would collide). The non-atomic
  read-then-insert window is documented and bounded by the read tiebreak +
  snapshot re-densify.
- Dedupe message ids before insert (87 prod chats carry dup ids; a repeated
  id in one INSERT...ON CONFLICT would otherwise throw).
- Backfill picks first-occurrence per (chat,id), gap-free via ROW_NUMBER;
  validated on staging data (0-based, contiguous, 0 bad ranges).

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Waleed
2026-05-29 11:50:44 -07:00
committed by GitHub
co-authored by Claude Opus 4.8
parent dc6073eda4
commit fd77bb4069
7 changed files with 17650 additions and 23 deletions
@@ -749,6 +749,7 @@ export const schemaMock = {
model: 'model',
tokensIn: 'tokensIn',
tokensOut: 'tokensOut',
seq: 'seq',
deletedAt: 'deletedAt',
createdAt: 'createdAt',
updatedAt: 'updatedAt',