fix(db): serialize concurrent migrations with a Postgres advisory lock (#4939)

* fix(db): serialize concurrent migrations with a Postgres advisory lock

Deployments start N app replicas at once, each with a migration sidecar.
drizzle migrate() has no cross-process lock, so all N read
__drizzle_migrations, all see the same migration pending, and all apply it
concurrently — one wins, the losers run the same DDL against already-mutated
state and exit 1 (e.g. DROP TABLE "form" -> table does not exist /
TaskFailedToStart). Wrap migrate() in a session-level pg_advisory_lock so
runners serialize: the winner migrates, the losers block, then re-read and
find nothing pending. Session locks auto-release on disconnect, so a crashed
runner never wedges the lock.

* fix(db): guard pg_advisory_unlock so it cannot mask a successful migration

If the explicit unlock throws (e.g. connection drops in the window after
migrate() commits), the exception bubbled to the outer catch and exited 1 —
falsely reporting a failed migration to the deploy orchestrator. The session
lock auto-releases on disconnect anyway, so swallow and log instead.

* refactor(db): move unlock-guard rationale to TSDoc helper

packages/db/scripts/migrate.ts

@@ -38,12 +38,34 @@ if (!url) {
 
 const client = postgres(url, { max: 1, connect_timeout: 10 })
 
+/**
+ * Cross-process migration lock key (a stable, app-wide 64-bit constant).
+ *
+ * drizzle's `migrate()` has no built-in lock, so when a deployment starts N app
+ * replicas at once — each with a migration sidecar — all N read
+ * `__drizzle_migrations`, all see the same migration pending, and all try to apply
+ * it concurrently. One wins; the losers run the same DDL against already-mutated
+ * state and die (e.g. `DROP TABLE "form"` → `table "form" does not exist`,
+ * exit 1 / TaskFailedToStart).
+ *
+ * A session-level `pg_advisory_lock` serializes runners: the first to acquire it
+ * migrates while the rest block, then each loser acquires the lock, re-reads
+ * `__drizzle_migrations`, finds nothing pending, and exits cleanly. Session locks
+ * auto-release if the connection drops, so a crashed runner never wedges the lock.
+ */
+const MIGRATION_LOCK_KEY = 4_961_002_270n
+
 try {
   // statement_timeout=0: index builds (esp. CONCURRENTLY on large tables) can run
   // far longer than the app default; a migration must never be killed mid-build.
   await client`SET statement_timeout = 0`
-  await migrate(drizzle(client), { migrationsFolder: './migrations' })
-  console.log('Migrations applied successfully.')
+  await client`SELECT pg_advisory_lock(${MIGRATION_LOCK_KEY})`
+  try {
+    await migrate(drizzle(client), { migrationsFolder: './migrations' })
+    console.log('Migrations applied successfully.')
+  } finally {
+    await releaseMigrationLock()
+  }
 } catch (error) {
   console.error('ERROR: Migration failed.')
   printMigrationError(error)
@@ -52,6 +74,24 @@ try {
   await client.end()
 }
 
+/**
+ * Release the advisory lock without ever failing the process. The session-level
+ * lock auto-releases when the connection closes, so a thrown unlock — e.g. the
+ * connection dropped right after `migrate()` committed — must be swallowed.
+ * Letting it reach the outer `catch` would exit 1 and falsely report a
+ * successful migration as failed to the deploy orchestrator.
+ */
+async function releaseMigrationLock(): Promise<void> {
+  try {
+    await client`SELECT pg_advisory_unlock(${MIGRATION_LOCK_KEY})`
+  } catch (unlockError) {
+    console.error(
+      'WARN: pg_advisory_unlock failed; the session lock will auto-release on disconnect.',
+      unlockError
+    )
+  }
+}
+
 /**
  * Print every diagnostic field a Postgres driver puts on a thrown error. The default
  * `error.message` loses the constraint name, affected table/column, PG code, and hint —