improvement(logs): object storage backed tracespans (#4787)

* improvement(logs): obj storage backed tracespans

* fix storage write context

* fix tests

* address comments

* address comments

* chore(db): remove migration 0219 to regenerate after staging merge

Drops the 0219_robust_shard SQL, its snapshot, and the journal entry so the
trace-spans/cost schema migration can be regenerated on top of the latest
staging migration chain (avoids a number collision with staging's migrations).

Co-authored-by: Cursor <cursoragent@cursor.com>

* improvement(billing): accurate per-member usage via shared ledger helper

Per-member/per-user usage in the org-member routes now adds the usage_log
ledger to the currentPeriodCost baseline (which is no longer incremented),
via a shared getOrgMemberLedgerByUser helper to avoid repeating the
subscription→period→ledger lookup across the admin and member-facing routes.

Co-authored-by: Cursor <cursoragent@cursor.com>

* regen migrations

* update migration

* address comments

* more code cleanup

* incorrect type cast

---------

Co-authored-by: Cursor <cursoragent@cursor.com>

packages/db/scripts/migrate.ts

@@ -2,6 +2,33 @@ import { drizzle } from 'drizzle-orm/postgres-js'
 import { migrate } from 'drizzle-orm/postgres-js/migrator'
 import postgres from 'postgres'
 
+/**
+ * Concurrent-index convention (avoid write-blocking index builds on large tables)
+ * --------------------------------------------------------------------------------
+ * drizzle-kit emits plain `CREATE INDEX`, which takes a SHARE lock and blocks all
+ * writes for the build duration — on a big, write-hot table (e.g.
+ * workflow_execution_logs, usage_log) that stalls every in-flight workflow
+ * completion for minutes. drizzle wraps each migration in a transaction, and
+ * `CREATE INDEX CONCURRENTLY` cannot run inside a transaction block.
+ *
+ * So, after generating a migration that adds an index on a large/hot table, edit
+ * the generated SQL to end drizzle's transaction first, then build concurrently
+ * and idempotently:
+ *
+ *   COMMIT;--> statement-breakpoint
+ *   CREATE INDEX CONCURRENTLY IF NOT EXISTS "idx_name" ON "table" (...);
+ *
+ * Notes:
+ *  - Put the `COMMIT` breakpoint AFTER all transactional DDL (ALTER TABLE/TYPE)
+ *    in the file and only the concurrent CREATE INDEX statements below it.
+ *  - Use `IF NOT EXISTS` (and make sibling DDL idempotent, e.g.
+ *    `ADD COLUMN IF NOT EXISTS`, `ADD VALUE IF NOT EXISTS`) so a re-run after a
+ *    failed CONCURRENTLY build is safe — fresh DBs and re-applies both work.
+ *  - CONCURRENTLY only takes a SHARE UPDATE EXCLUSIVE lock (allows reads/writes).
+ *  - Always validate on staging before prod; a failed CONCURRENTLY build can
+ *    leave an INVALID index that must be dropped and rebuilt.
+ */
+
 const url = process.env.DATABASE_URL
 if (!url) {
   console.error('ERROR: Missing DATABASE_URL environment variable.')
@@ -12,6 +39,8 @@ if (!url) {
 const client = postgres(url, { max: 1, connect_timeout: 10 })
 
 try {
+  // statement_timeout=0: index builds (esp. CONCURRENTLY on large tables) can run
+  // far longer than the app default; a migration must never be killed mid-build.
   await client`SET statement_timeout = 0`
   await migrate(drizzle(client), { migrationsFolder: './migrations' })
   console.log('Migrations applied successfully.')