From 3518b999af3539742f1bff55cc0dbf23599cfdc6 Mon Sep 17 00:00:00 2001 From: Theodore Li Date: Wed, 3 Jun 2026 18:26:06 -0700 Subject: [PATCH] feat(tables): background import for large CSVs with live progress (#4861) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(tables): background import for large CSVs with live progress * fix(tables): address review — import heartbeat, overlap guard, column/empty validation * fix(tables): guard sync import overlap, scope fileKey to workspace, delete-on-replace after download * fix(tables): stream large CSV imports from storage instead of buffering the whole file * test(tables): fix async-import route tests for workspace-scoped fileKey + name uniquification * fix(tables): append imports start after existing rows; reconcile missed import failures in the tray * fix(tables): delete the uploaded CSV from storage after the import finishes * fix(tables): validate replace before deleting rows; ignore stale replayed import events by importId * fix(tables): bind import worker to its importId (no stale-worker clobber/overlap) and destroy storage stream on failure * feat(tables): byte-based import progress, cancel support, and a start toast that opens the import view * fix(tables): don't emit ready after cancel; honor cancel during the upload phase * improvement(tables): use a stop (square) icon for canceling an active import * fix(tables): make markTableImporting an atomic claim to close the concurrent-import TOCTOU race * improvement(tables): preview CSV import from a slice, drop client row-count warning The import dialog parsed the entire file in the browser to show an exact row count and a row-limit warning. That holds the whole file in memory, blocks the main thread, and hits V8's ~512MB string ceiling — so the dialog capped the effective import size well below what the streaming importer handles. Parse only the first 512KB (headers + sample for the mapping); drop the exact count and the "would exceed the row limit by N" gate. The DB row-count trigger already enforces max_rows server-side, so an over-limit import fails fast during the run with a clear message instead of being blocked by an expensive parse. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(tables): gate import ownership every batch and stop canceled imports reappearing - Worker checked run ownership only at the progress cadence (~every 5k rows), so a canceled/superseded import could insert several more batches (incl. the final partial batch) before stopping. Move the updateImportProgress ownership gate to the top of every flush — a run that lost the table stops within one batch. - A list/dialog import canceled mid-upload left the server row `importing` until the in-flight server cancel landed; hydration re-seeded it from useTablesList, so the dismissed import flickered back. Flag the real table id canceled on the mid-upload cancel path, skip re-seeding flagged tables in hydration, and clear the flag once the server import is terminal. Co-Authored-By: Claude Opus 4.8 (1M context) * refactor(tables): drive import tray by polling derived from server, not SSE Import progress no longer holds an SSE connection per importing table. The tray now derives its importing rows live from the table list (React Query), polled only while an import is in flight; the table detail page keeps its own cell-state SSE for grid refresh. - store holds only client-only state now: optimistic uploads, which terminal completions to surface this session, canceled ids, menu open — no copied importStatus/rowsProcessed. - useWorkspaceImports is the single source: polls via a data-predicate refetchInterval, derives rows, and fires completion toasts on the importing -> terminal transition. - kickoff handlers use startUpload/setUploadPercent/endUpload; the invalidated list refetch surfaces the server row and polling takes over. - removes use-hydrate-import-tray + use-import-progress-tracker (folded in). - trims over-verbose comments across the import paths. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(tables): ignore superseded-run import events in the detail SSE cache applyImport applied every replayed import payload to the detail cache. The SSE buffer can replay a prior import's terminal event for the same table, stomping a newer in-flight import's UI. Lock to the active run's importId (and ignore a replayed terminal before the id is known), matching the guard the header tracker used to have. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(tables): close sync-import TOCTOU by claiming the atomic import gate The sync import route checked importStatus from a checkAccess snapshot, then parsed/validated/wrote seconds later without taking the atomic claim. A concurrent async kickoff (markTableImporting) could slip into that window and both writers would run together — for replace mode, two delete+insert passes leave the table indeterminate. Claim the same atomic gate (markTableImporting) right before the write and release it in the finally (before the response returns, so a client refetch never sees the transient status). A row-level FOR UPDATE was avoided on purpose: it would invert lock order against the position advisory lock / row-count trigger and risk a deadlock — markTableImporting is the established gate. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(multipart): keep abort wired after resolve so a mid-upload disconnect tears down the stream readMultipart resolves on the file-part header and hands the caller an un-drained stream, but settle() ran cleanup() and detached the abort listener on that path too. A client disconnect mid-upload then destroyed nothing — busboy never saw EOF, the file stream stalled, and the route's `for await` held a request slot until maxDuration (300s). Re-arm an abort handler scoped to the file stream on resolve, detached when the stream closes. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) --- .../cron/cleanup-stale-executions/route.ts | 36 +- .../[tableId]/import-async/route.test.ts | 144 + .../api/table/[tableId]/import-async/route.ts | 92 + .../[tableId]/import/cancel/route.test.ts | 110 + .../table/[tableId]/import/cancel/route.ts | 54 + .../api/table/[tableId]/import/route.test.ts | 68 +- .../app/api/table/[tableId]/import/route.ts | 142 +- apps/sim/app/api/table/[tableId]/route.ts | 4 + .../app/api/table/import-async/route.test.ts | 123 + apps/sim/app/api/table/import-async/route.ts | 115 + .../app/api/table/import-csv/route.test.ts | 209 +- apps/sim/app/api/table/import-csv/route.ts | 243 +- apps/sim/app/api/table/route.ts | 4 + apps/sim/app/api/table/utils.ts | 34 + .../[tableId]/hooks/use-table-event-stream.ts | 50 +- .../[workspaceId]/tables/[tableId]/table.tsx | 18 +- .../import-csv-dialog/import-csv-dialog.tsx | 123 +- .../import-progress-menu.tsx | 76 + .../import-progress-menu/import-stage.ts | 58 + .../components/import-progress-menu/index.ts | 1 + .../use-workspace-imports.ts | 118 + .../[workspaceId]/tables/components/index.ts | 1 + .../workspace/[workspaceId]/tables/tables.tsx | 107 +- apps/sim/components/emcn/components/index.ts | 1 + .../progress-item/progress-item.tsx | 106 + apps/sim/hooks/queries/tables.ts | 135 +- apps/sim/lib/api/contracts/tables.ts | 78 + .../tools/handlers/materialize-file.test.ts | 177 + .../tools/handlers/materialize-file.ts | 128 +- .../copilot/tools/server/table/user-table.ts | 72 +- apps/sim/lib/core/utils/multipart.test.ts | 203 + apps/sim/lib/core/utils/multipart.ts | 256 + apps/sim/lib/table/constants.ts | 7 + apps/sim/lib/table/events.ts | 15 + apps/sim/lib/table/import-runner.ts | 296 + apps/sim/lib/table/import.test.ts | 42 + apps/sim/lib/table/import.ts | 139 +- apps/sim/lib/table/service.ts | 270 +- apps/sim/lib/table/types.ts | 13 + apps/sim/lib/uploads/core/storage-service.ts | 29 + apps/sim/lib/uploads/providers/blob/client.ts | 44 + apps/sim/lib/uploads/providers/s3/client.ts | 19 + apps/sim/package.json | 2 + apps/sim/stores/table/import-tray/store.ts | 109 + bun.lock | 12 + .../migrations/0224_table_import_columns.sql | 92 + .../db/migrations/meta/0224_snapshot.json | 17223 ++++++++++++++++ packages/db/migrations/meta/_journal.json | 7 + packages/db/schema.ts | 10 + scripts/check-api-validation-contracts.ts | 4 +- 50 files changed, 21020 insertions(+), 399 deletions(-) create mode 100644 apps/sim/app/api/table/[tableId]/import-async/route.test.ts create mode 100644 apps/sim/app/api/table/[tableId]/import-async/route.ts create mode 100644 apps/sim/app/api/table/[tableId]/import/cancel/route.test.ts create mode 100644 apps/sim/app/api/table/[tableId]/import/cancel/route.ts create mode 100644 apps/sim/app/api/table/import-async/route.test.ts create mode 100644 apps/sim/app/api/table/import-async/route.ts create mode 100644 apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/import-progress-menu.tsx create mode 100644 apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/import-stage.ts create mode 100644 apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/index.ts create mode 100644 apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/use-workspace-imports.ts create mode 100644 apps/sim/components/emcn/components/progress-item/progress-item.tsx create mode 100644 apps/sim/lib/copilot/tools/handlers/materialize-file.test.ts create mode 100644 apps/sim/lib/core/utils/multipart.test.ts create mode 100644 apps/sim/lib/core/utils/multipart.ts create mode 100644 apps/sim/lib/table/import-runner.ts create mode 100644 apps/sim/stores/table/import-tray/store.ts create mode 100644 packages/db/migrations/0224_table_import_columns.sql create mode 100644 packages/db/migrations/meta/0224_snapshot.json diff --git a/apps/sim/app/api/cron/cleanup-stale-executions/route.ts b/apps/sim/app/api/cron/cleanup-stale-executions/route.ts index 52c9420916..99c395d644 100644 --- a/apps/sim/app/api/cron/cleanup-stale-executions/route.ts +++ b/apps/sim/app/api/cron/cleanup-stale-executions/route.ts @@ -1,5 +1,5 @@ import { asyncJobs, db } from '@sim/db' -import { workflowExecutionLogs } from '@sim/db/schema' +import { userTableDefinitions, workflowExecutionLogs } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' import { and, eq, inArray, lt, sql } from 'drizzle-orm' @@ -110,6 +110,37 @@ export const GET = withRouteHandler(async (request: NextRequest) => { }) } + // Mark stale table imports as failed. Imports run detached on the web container and + // are lost if the pod is killed mid-load. `updatedAt` is bumped by progress updates, so + // an `importing` table with no recent update has stalled (not merely slow). Rows are + // left in place (no rollback); the user re-imports. + let staleImportsMarkedFailed = 0 + try { + const staleImports = await db + .update(userTableDefinitions) + .set({ + importStatus: 'failed', + importError: `Import terminated: no progress for more than ${STALE_THRESHOLD_MINUTES} minutes (worker timeout or crash)`, + updatedAt: new Date(), + }) + .where( + and( + eq(userTableDefinitions.importStatus, 'importing'), + lt(userTableDefinitions.updatedAt, staleThreshold) + ) + ) + .returning({ id: userTableDefinitions.id }) + + staleImportsMarkedFailed = staleImports.length + if (staleImportsMarkedFailed > 0) { + logger.info(`Marked ${staleImportsMarkedFailed} stale table imports as failed`) + } + } catch (error) { + logger.error('Failed to clean up stale table imports:', { + error: toError(error).message, + }) + } + // Clean up stale pending jobs (never started, e.g., due to server crash before startJob()) let stalePendingJobsMarkedFailed = 0 @@ -179,6 +210,9 @@ export const GET = withRouteHandler(async (request: NextRequest) => { staleThresholdMinutes: STALE_THRESHOLD_MINUTES, retentionHours: JOB_RETENTION_HOURS, }, + tableImports: { + staleMarkedFailed: staleImportsMarkedFailed, + }, }) } catch (error) { logger.error('Error in stale execution cleanup job:', error) diff --git a/apps/sim/app/api/table/[tableId]/import-async/route.test.ts b/apps/sim/app/api/table/[tableId]/import-async/route.test.ts new file mode 100644 index 0000000000..18fa93aca8 --- /dev/null +++ b/apps/sim/app/api/table/[tableId]/import-async/route.test.ts @@ -0,0 +1,144 @@ +/** + * @vitest-environment node + */ +import { hybridAuthMockFns } from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' +import type { TableDefinition } from '@/lib/table' + +const { mockCheckAccess, mockMarkTableImporting, mockRunTableImport } = vi.hoisted(() => ({ + mockCheckAccess: vi.fn(), + mockMarkTableImporting: vi.fn(), + mockRunTableImport: vi.fn(), +})) + +vi.mock('@sim/utils/id', () => ({ + generateId: vi.fn().mockReturnValue('import-id-xyz'), + generateShortId: vi.fn().mockReturnValue('short-id'), +})) +vi.mock('@/lib/table/service', () => ({ markTableImporting: mockMarkTableImporting })) +vi.mock('@/lib/table/import-runner', () => ({ runTableImport: mockRunTableImport })) +vi.mock('@/lib/core/utils/background', () => ({ + runDetached: (_label: string, work: () => Promise) => { + void work() + }, +})) +vi.mock('@/app/api/table/utils', async () => { + const { NextResponse } = await import('next/server') + return { + checkAccess: mockCheckAccess, + accessError: (result: { status: number }) => + NextResponse.json({ error: 'denied' }, { status: result.status }), + } +}) + +import { POST } from '@/app/api/table/[tableId]/import-async/route' + +function buildTable(overrides: Partial = {}): TableDefinition { + return { + id: 'tbl_1', + name: 'People', + description: null, + schema: { columns: [{ name: 'name', type: 'string' }] }, + metadata: null, + rowCount: 0, + maxRows: 1_000_000, + workspaceId: 'workspace-1', + createdBy: 'user-1', + archivedAt: null, + createdAt: new Date(), + updatedAt: new Date(), + ...overrides, + } +} + +function makeRequest(body: unknown, tableId = 'tbl_1') { + const req = new NextRequest(`http://localhost:3000/api/table/${tableId}/import-async`, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(body), + }) + return POST(req, { params: Promise.resolve({ tableId }) }) +} + +const validBody = { + workspaceId: 'workspace-1', + fileKey: 'workspace/workspace-1/123-data.csv', + fileName: 'data.csv', + mode: 'append', +} + +describe('POST /api/table/[tableId]/import-async', () => { + beforeEach(() => { + vi.clearAllMocks() + hybridAuthMockFns.mockCheckSessionOrInternalAuth.mockResolvedValue({ + success: true, + userId: 'user-1', + authType: 'session', + }) + mockCheckAccess.mockResolvedValue({ ok: true, table: buildTable() }) + mockMarkTableImporting.mockResolvedValue(true) + mockRunTableImport.mockResolvedValue(undefined) + }) + + it('marks the table importing and kicks off the worker with mode + mapping', async () => { + const response = await makeRequest({ + ...validBody, + mode: 'replace', + mapping: { Name: 'name' }, + createColumns: ['Extra'], + }) + const data = await response.json() + + expect(response.status).toBe(200) + expect(data.data).toEqual({ tableId: 'tbl_1', importId: 'import-id-xyz' }) + expect(mockMarkTableImporting).toHaveBeenCalledWith('tbl_1', 'import-id-xyz') + expect(mockRunTableImport).toHaveBeenCalledWith( + expect.objectContaining({ + tableId: 'tbl_1', + mode: 'replace', + delimiter: ',', + mapping: { Name: 'name' }, + createColumns: ['Extra'], + }) + ) + }) + + it('returns 409 when the table is already importing (claim lost)', async () => { + mockMarkTableImporting.mockResolvedValue(false) + const response = await makeRequest(validBody) + expect(response.status).toBe(409) + expect(mockRunTableImport).not.toHaveBeenCalled() + }) + + it('returns 401 when unauthenticated', async () => { + hybridAuthMockFns.mockCheckSessionOrInternalAuth.mockResolvedValue({ success: false }) + const response = await makeRequest(validBody) + expect(response.status).toBe(401) + expect(mockMarkTableImporting).not.toHaveBeenCalled() + }) + + it('returns the access error status when access is denied', async () => { + mockCheckAccess.mockResolvedValue({ ok: false, status: 403 }) + const response = await makeRequest(validBody) + expect(response.status).toBe(403) + expect(mockRunTableImport).not.toHaveBeenCalled() + }) + + it('returns 400 when the target table is archived', async () => { + mockCheckAccess.mockResolvedValue({ ok: true, table: buildTable({ archivedAt: new Date() }) }) + const response = await makeRequest(validBody) + expect(response.status).toBe(400) + expect(mockRunTableImport).not.toHaveBeenCalled() + }) + + it('returns 400 on workspace mismatch', async () => { + const response = await makeRequest({ ...validBody, workspaceId: 'other-ws' }) + expect(response.status).toBe(400) + }) + + it('returns 400 for an invalid mode', async () => { + const response = await makeRequest({ ...validBody, mode: 'bogus' }) + expect(response.status).toBe(400) + }) +}) diff --git a/apps/sim/app/api/table/[tableId]/import-async/route.ts b/apps/sim/app/api/table/[tableId]/import-async/route.ts new file mode 100644 index 0000000000..46190cbfb0 --- /dev/null +++ b/apps/sim/app/api/table/[tableId]/import-async/route.ts @@ -0,0 +1,92 @@ +import { createLogger } from '@sim/logger' +import { generateId } from '@sim/utils/id' +import { type NextRequest, NextResponse } from 'next/server' +import { importIntoTableAsyncContract } from '@/lib/api/contracts/tables' +import { parseRequest } from '@/lib/api/server' +import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid' +import { runDetached } from '@/lib/core/utils/background' +import { generateRequestId } from '@/lib/core/utils/request' +import { withRouteHandler } from '@/lib/core/utils/with-route-handler' +import { runTableImport } from '@/lib/table/import-runner' +import { markTableImporting } from '@/lib/table/service' +import { accessError, checkAccess } from '@/app/api/table/utils' + +const logger = createLogger('TableImportIntoAsync') + +export const runtime = 'nodejs' +export const dynamic = 'force-dynamic' + +interface RouteParams { + params: Promise<{ tableId: string }> +} + +export const POST = withRouteHandler(async (request: NextRequest, { params }: RouteParams) => { + const requestId = generateRequestId() + + const authResult = await checkSessionOrInternalAuth(request, { requireWorkflowId: false }) + if (!authResult.success || !authResult.userId) { + return NextResponse.json({ error: 'Authentication required' }, { status: 401 }) + } + const userId = authResult.userId + + const parsed = await parseRequest(importIntoTableAsyncContract, request, { params }) + if (!parsed.success) return parsed.response + const { tableId } = parsed.data.params + const { workspaceId, fileKey, fileName, mode, mapping, createColumns } = parsed.data.body + + const access = await checkAccess(tableId, userId, 'write') + if (!access.ok) return accessError(access, requestId, tableId) + const { table } = access + + if (table.workspaceId !== workspaceId) { + return NextResponse.json({ error: 'Invalid workspace ID' }, { status: 400 }) + } + // The fileKey is client-supplied — ensure it points at this workspace's storage prefix so a + // caller can't import another workspace's uploaded object. + if (!fileKey.startsWith(`workspace/${workspaceId}/`)) { + return NextResponse.json({ error: 'Invalid file key for workspace' }, { status: 400 }) + } + if (table.archivedAt) { + return NextResponse.json({ error: 'Cannot import into an archived table' }, { status: 400 }) + } + + const ext = fileName.split('.').pop()?.toLowerCase() + if (ext !== 'csv' && ext !== 'tsv') { + return NextResponse.json({ error: 'Only CSV and TSV files are supported' }, { status: 400 }) + } + const delimiter = ext === 'tsv' ? '\t' : ',' + + // Atomically claim the table — the single concurrency gate. If another import already holds it, + // this returns false (no overlapping workers writing colliding row positions). + const importId = generateId() + const claimed = await markTableImporting(tableId, importId) + if (!claimed) { + return NextResponse.json( + { error: 'An import is already in progress for this table' }, + { status: 409 } + ) + } + + runDetached('table-import', () => + runTableImport({ + importId, + tableId, + workspaceId, + userId, + fileKey, + fileName, + delimiter, + mode, + mapping, + createColumns, + }) + ) + + logger.info(`[${requestId}] Async CSV import into existing table started`, { + tableId, + importId, + mode, + fileName, + }) + return NextResponse.json({ success: true, data: { tableId, importId } }) +}) diff --git a/apps/sim/app/api/table/[tableId]/import/cancel/route.test.ts b/apps/sim/app/api/table/[tableId]/import/cancel/route.test.ts new file mode 100644 index 0000000000..d45baae77e --- /dev/null +++ b/apps/sim/app/api/table/[tableId]/import/cancel/route.test.ts @@ -0,0 +1,110 @@ +/** + * @vitest-environment node + */ +import { hybridAuthMockFns } from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' +import type { TableDefinition } from '@/lib/table' + +const { mockCheckAccess, mockMarkImportCanceled, mockAppendTableEvent } = vi.hoisted(() => ({ + mockCheckAccess: vi.fn(), + mockMarkImportCanceled: vi.fn(), + mockAppendTableEvent: vi.fn(), +})) + +vi.mock('@/lib/table/service', () => ({ markImportCanceled: mockMarkImportCanceled })) +vi.mock('@/lib/table/events', () => ({ appendTableEvent: mockAppendTableEvent })) +vi.mock('@/app/api/table/utils', async () => { + const { NextResponse } = await import('next/server') + return { + checkAccess: mockCheckAccess, + accessError: (result: { status: number }) => + NextResponse.json({ error: 'denied' }, { status: result.status }), + } +}) + +import { POST } from '@/app/api/table/[tableId]/import/cancel/route' + +function buildTable(overrides: Partial = {}): TableDefinition { + return { + id: 'tbl_1', + name: 'People', + description: null, + schema: { columns: [{ name: 'name', type: 'string' }] }, + metadata: null, + rowCount: 0, + maxRows: 1_000_000, + workspaceId: 'workspace-1', + createdBy: 'user-1', + archivedAt: null, + createdAt: new Date(), + updatedAt: new Date(), + ...overrides, + } +} + +function makeRequest(body: unknown, tableId = 'tbl_1') { + const req = new NextRequest(`http://localhost:3000/api/table/${tableId}/import/cancel`, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(body), + }) + return POST(req, { params: Promise.resolve({ tableId }) }) +} + +const validBody = { workspaceId: 'workspace-1', importId: 'import-id-xyz' } + +describe('POST /api/table/[tableId]/import/cancel', () => { + beforeEach(() => { + vi.clearAllMocks() + hybridAuthMockFns.mockCheckSessionOrInternalAuth.mockResolvedValue({ + success: true, + userId: 'user-1', + authType: 'session', + }) + mockCheckAccess.mockResolvedValue({ ok: true, table: buildTable() }) + mockMarkImportCanceled.mockResolvedValue(true) + }) + + it('cancels the import and emits a canceled event', async () => { + const response = await makeRequest(validBody) + const data = await response.json() + + expect(response.status).toBe(200) + expect(data.data).toEqual({ canceled: true }) + expect(mockMarkImportCanceled).toHaveBeenCalledWith('tbl_1', 'import-id-xyz') + expect(mockAppendTableEvent).toHaveBeenCalledWith( + expect.objectContaining({ kind: 'import', status: 'canceled', importId: 'import-id-xyz' }) + ) + }) + + it('does not emit an event when nothing was importing', async () => { + mockMarkImportCanceled.mockResolvedValue(false) + const response = await makeRequest(validBody) + const data = await response.json() + + expect(response.status).toBe(200) + expect(data.data).toEqual({ canceled: false }) + expect(mockAppendTableEvent).not.toHaveBeenCalled() + }) + + it('returns 401 when unauthenticated', async () => { + hybridAuthMockFns.mockCheckSessionOrInternalAuth.mockResolvedValue({ success: false }) + const response = await makeRequest(validBody) + expect(response.status).toBe(401) + expect(mockMarkImportCanceled).not.toHaveBeenCalled() + }) + + it('returns the access error status when access is denied', async () => { + mockCheckAccess.mockResolvedValue({ ok: false, status: 403 }) + const response = await makeRequest(validBody) + expect(response.status).toBe(403) + }) + + it('returns 400 on workspace mismatch', async () => { + mockCheckAccess.mockResolvedValue({ ok: true, table: buildTable({ workspaceId: 'other-ws' }) }) + const response = await makeRequest(validBody) + expect(response.status).toBe(400) + expect(mockMarkImportCanceled).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/app/api/table/[tableId]/import/cancel/route.ts b/apps/sim/app/api/table/[tableId]/import/cancel/route.ts new file mode 100644 index 0000000000..62ab7310f4 --- /dev/null +++ b/apps/sim/app/api/table/[tableId]/import/cancel/route.ts @@ -0,0 +1,54 @@ +import { createLogger } from '@sim/logger' +import { type NextRequest, NextResponse } from 'next/server' +import { cancelTableImportContract } from '@/lib/api/contracts/tables' +import { parseRequest } from '@/lib/api/server' +import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid' +import { generateRequestId } from '@/lib/core/utils/request' +import { withRouteHandler } from '@/lib/core/utils/with-route-handler' +import { appendTableEvent } from '@/lib/table/events' +import { markImportCanceled } from '@/lib/table/service' +import { accessError, checkAccess } from '@/app/api/table/utils' + +const logger = createLogger('TableImportCancelAPI') + +export const runtime = 'nodejs' +export const dynamic = 'force-dynamic' + +interface RouteParams { + params: Promise<{ tableId: string }> +} + +/** + * POST /api/table/[tableId]/import/cancel + * + * Cancels an in-flight async CSV import. Flips the table's import status to `canceled`, which makes + * the detached worker's next ownership check fail so it stops inserting. Committed rows are left in + * place (no rollback) — the user can delete the table. No-op if the import already finished. + */ +export const POST = withRouteHandler(async (request: NextRequest, { params }: RouteParams) => { + const requestId = generateRequestId() + + const authResult = await checkSessionOrInternalAuth(request, { requireWorkflowId: false }) + if (!authResult.success || !authResult.userId) { + return NextResponse.json({ error: 'Authentication required' }, { status: 401 }) + } + + const parsed = await parseRequest(cancelTableImportContract, request, { params }) + if (!parsed.success) return parsed.response + const { tableId } = parsed.data.params + const { workspaceId, importId } = parsed.data.body + + const access = await checkAccess(tableId, authResult.userId, 'write') + if (!access.ok) return accessError(access, requestId, tableId) + if (access.table.workspaceId !== workspaceId) { + return NextResponse.json({ error: 'Invalid workspace ID' }, { status: 400 }) + } + + const canceled = await markImportCanceled(tableId, importId) + if (canceled) { + void appendTableEvent({ kind: 'import', tableId, importId, status: 'canceled' }) + } + logger.info(`[${requestId}] Import cancel requested`, { tableId, importId, canceled }) + + return NextResponse.json({ success: true, data: { canceled } }) +}) diff --git a/apps/sim/app/api/table/[tableId]/import/route.test.ts b/apps/sim/app/api/table/[tableId]/import/route.test.ts index 1a55174540..a77da89c52 100644 --- a/apps/sim/app/api/table/[tableId]/import/route.test.ts +++ b/apps/sim/app/api/table/[tableId]/import/route.test.ts @@ -12,12 +12,16 @@ const { mockReplaceTableRowsWithTx, mockAddTableColumnsWithTx, mockDispatchAfterBatchInsert, + mockMarkTableImporting, + mockReleaseImportClaim, } = vi.hoisted(() => ({ mockCheckAccess: vi.fn(), mockBatchInsertRowsWithTx: vi.fn(), mockReplaceTableRowsWithTx: vi.fn(), mockAddTableColumnsWithTx: vi.fn(), mockDispatchAfterBatchInsert: vi.fn(), + mockMarkTableImporting: vi.fn(), + mockReleaseImportClaim: vi.fn(), })) vi.mock('@sim/utils/id', () => ({ @@ -33,6 +37,12 @@ vi.mock('@/app/api/table/utils', async () => { const message = result.status === 404 ? 'Table not found' : 'Access denied' return NextResponse.json({ error: message }, { status: result.status }) }, + csvProxyBodyCapResponse: () => null, + multipartErrorResponse: (error: { code: string; message: string }) => + NextResponse.json( + { error: error.message }, + { status: error.code === 'FILE_TOO_LARGE' ? 413 : 400 } + ), } }) @@ -47,6 +57,8 @@ vi.mock('@/lib/table/service', () => ({ replaceTableRowsWithTx: mockReplaceTableRowsWithTx, addTableColumnsWithTx: mockAddTableColumnsWithTx, dispatchAfterBatchInsert: mockDispatchAfterBatchInsert, + markTableImporting: mockMarkTableImporting, + releaseImportClaim: mockReleaseImportClaim, })) import { POST } from '@/app/api/table/[tableId]/import/route' @@ -64,8 +76,8 @@ function createFormData( createColumns?: unknown } ): FormData { + // Text fields must precede the file part for the streaming parser. const form = new FormData() - form.append('file', file) if (options?.workspaceId !== null) { form.append('workspaceId', options?.workspaceId ?? 'workspace-1') } @@ -86,6 +98,7 @@ function createFormData( : JSON.stringify(options.createColumns) ) } + form.append('file', file) return form } @@ -113,9 +126,10 @@ function buildTable(overrides: Partial = {}): TableDefinition { } async function callPost(form: FormData, { tableId }: { tableId: string } = { tableId: 'tbl_1' }) { + // Building the request from a FormData body gives a real multipart stream and + // boundary, exercising the streaming `readMultipart` parser end-to-end. const req = new NextRequest(`http://localhost:3000/api/table/${tableId}/import`, { method: 'POST', - headers: { 'content-length': '1024' }, body: form, }) return POST(req, { params: Promise.resolve({ tableId }) }) @@ -134,6 +148,8 @@ describe('POST /api/table/[tableId]/import', () => { data.rows.map((_, i) => ({ id: `row_${i}` })) ) mockReplaceTableRowsWithTx.mockResolvedValue({ deletedCount: 0, insertedCount: 0 }) + mockMarkTableImporting.mockResolvedValue(true) + mockReleaseImportClaim.mockResolvedValue(undefined) mockAddTableColumnsWithTx.mockImplementation( async ( _trx, @@ -160,6 +176,22 @@ describe('POST /api/table/[tableId]/import', () => { expect(response.status).toBe(401) }) + it('returns 409 when a background import already holds the table (claim lost)', async () => { + mockMarkTableImporting.mockResolvedValueOnce(false) + const response = await callPost(createFormData(createCsvFile('name,age\nAlice,30'))) + expect(response.status).toBe(409) + expect(mockBatchInsertRowsWithTx).not.toHaveBeenCalled() + expect(mockReplaceTableRowsWithTx).not.toHaveBeenCalled() + expect(mockReleaseImportClaim).not.toHaveBeenCalled() + }) + + it('releases the import claim after a successful write', async () => { + const response = await callPost(createFormData(createCsvFile('name,age\nAlice,30'))) + expect(response.status).toBe(200) + expect(mockMarkTableImporting).toHaveBeenCalledWith('tbl_1', 'deadbeefcafef00d') + expect(mockReleaseImportClaim).toHaveBeenCalledWith('tbl_1', 'deadbeefcafef00d') + }) + it('returns 400 when the mode is invalid', async () => { const response = await callPost( createFormData(createCsvFile('name,age\nAlice,30'), { mode: 'bogus' }) @@ -186,22 +218,30 @@ describe('POST /api/table/[tableId]/import', () => { expect(data.error).toMatch(/archived/i) }) - it('returns 413 for oversized CSV files before reading their contents', async () => { - const file = createCsvFile('name,age\nAlice,30') - Object.defineProperty(file, 'size', { - value: 26 * 1024 * 1024, - }) - const arrayBufferSpy = vi.spyOn(file, 'arrayBuffer') - + it('returns 400 when the file part precedes the required fields', async () => { + // Build a raw multipart body with the file BEFORE workspaceId. + const boundary = '----orderboundary' + const body = Buffer.concat([ + Buffer.from( + `--${boundary}\r\nContent-Disposition: form-data; name="file"; filename="data.csv"\r\nContent-Type: text/csv\r\n\r\nname,age\nAlice,30\r\n` + ), + Buffer.from(`--${boundary}\r\nContent-Disposition: form-data; name="workspaceId"\r\n\r\n`), + Buffer.from('workspace-1\r\n'), + Buffer.from(`--${boundary}--\r\n`), + ]) const req = { - formData: async () => createFormData(file), + headers: new Headers({ 'content-type': `multipart/form-data; boundary=${boundary}` }), + body: new ReadableStream({ + start(controller) { + controller.enqueue(new Uint8Array(body)) + controller.close() + }, + }), + signal: undefined, } as unknown as NextRequest const response = await POST(req, { params: Promise.resolve({ tableId: 'tbl_1' }) }) - expect(response.status).toBe(413) - const data = await response.json() - expect(data.error).toMatch(/CSV import file exceeds maximum size/) - expect(arrayBufferSpy).not.toHaveBeenCalled() + expect(response.status).toBe(400) expect(mockBatchInsertRowsWithTx).not.toHaveBeenCalled() expect(mockReplaceTableRowsWithTx).not.toHaveBeenCalled() }) diff --git a/apps/sim/app/api/table/[tableId]/import/route.ts b/apps/sim/app/api/table/[tableId]/import/route.ts index e097723c02..3d16cd636e 100644 --- a/apps/sim/app/api/table/[tableId]/import/route.ts +++ b/apps/sim/app/api/table/[tableId]/import/route.ts @@ -1,3 +1,4 @@ +import type { Readable } from 'node:stream' import { db } from '@sim/db' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' @@ -13,12 +14,8 @@ import { } from '@/lib/api/contracts/tables' import { getValidationErrorMessage } from '@/lib/api/server' import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid' +import { isMultipartError, readMultipart } from '@/lib/core/utils/multipart' import { generateRequestId } from '@/lib/core/utils/request' -import { - isPayloadSizeLimitError, - readFileToBufferWithLimit, - readFormDataWithLimit, -} from '@/lib/core/utils/stream-limits' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { addTableColumnsWithTx, @@ -29,9 +26,11 @@ import { type CsvHeaderMapping, CsvImportValidationError, coerceRowsForTable, + createCsvParser, dispatchAfterBatchInsert, inferColumnType, - parseCsvBuffer, + markTableImporting, + releaseImportClaim, replaceTableRowsWithTx, sanitizeName, type TableDefinition, @@ -39,10 +38,18 @@ import { type TableSchema, validateMapping, } from '@/lib/table' -import { accessError, checkAccess } from '@/app/api/table/utils' +import { + accessError, + checkAccess, + csvProxyBodyCapResponse, + multipartErrorResponse, +} from '@/app/api/table/utils' const logger = createLogger('TableImportCSVExisting') -const MAX_MULTIPART_OVERHEAD_BYTES = 1024 * 1024 + +export const runtime = 'nodejs' +export const dynamic = 'force-dynamic' +export const maxDuration = 300 interface RouteParams { params: Promise<{ tableId: string }> @@ -51,6 +58,8 @@ interface RouteParams { export const POST = withRouteHandler(async (request: NextRequest, { params }: RouteParams) => { const requestId = generateRequestId() const { tableId } = tableIdParamsSchema.parse(await params) + let fileStream: Readable | undefined + let claimedImportId: string | null = null try { const authResult = await checkSessionOrInternalAuth(request, { requireWorkflowId: false }) @@ -58,29 +67,37 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro return NextResponse.json({ error: 'Authentication required' }, { status: 401 }) } - const formData = await readFormDataWithLimit(request, { - maxBytes: CSV_MAX_FILE_SIZE_BYTES + MAX_MULTIPART_OVERHEAD_BYTES, - label: 'CSV import body', - }) - const formValidation = csvImportFormSchema.safeParse({ - file: formData.get('file'), - workspaceId: formData.get('workspaceId'), - }) - const rawMode = formData.get('mode') ?? 'append' - const rawMapping = formData.get('mapping') - const rawCreateColumns = formData.get('createColumns') + const oversize = csvProxyBodyCapResponse(request) + if (oversize) return oversize - if (!formValidation.success) { - const message = getValidationErrorMessage(formValidation.error) - const isSizeLimit = message.includes('File exceeds maximum allowed size') - return NextResponse.json( - { error: isSizeLimit ? 'CSV import file exceeds maximum size' : message }, - { status: isSizeLimit ? 413 : 400 } - ) + let parsed: Awaited> + try { + parsed = await readMultipart(request, { + maxFileBytes: CSV_MAX_FILE_SIZE_BYTES, + requiredFieldsBeforeFile: ['workspaceId'], + signal: request.signal, + }) + } catch (err) { + if (isMultipartError(err)) return multipartErrorResponse(err) + throw err } - const { file, workspaceId } = formValidation.data + const { fields, file } = parsed + if (!file) { + return NextResponse.json({ error: 'CSV file is required' }, { status: 400 }) + } + fileStream = file.stream + const workspaceIdResult = csvImportFormSchema.shape.workspaceId.safeParse(fields.workspaceId) + if (!workspaceIdResult.success) { + return NextResponse.json( + { error: getValidationErrorMessage(workspaceIdResult.error) }, + { status: 400 } + ) + } + const workspaceId = workspaceIdResult.data + + const rawMode = fields.mode ?? 'append' const modeValidation = csvImportModeSchema.safeParse(rawMode) if (!modeValidation.success) { return NextResponse.json( @@ -90,7 +107,7 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro } const mode = modeValidation.data - const ext = file.name.split('.').pop()?.toLowerCase() + const ext = file.filename.split('.').pop()?.toLowerCase() const extensionValidation = csvExtensionSchema.safeParse(ext) if (!extensionValidation.success) { return NextResponse.json( @@ -114,10 +131,18 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro if (table.archivedAt) { return NextResponse.json({ error: 'Cannot import into an archived table' }, { status: 400 }) } + // Don't run a sync import on top of an in-flight background import — concurrent writers + // would insert at colliding row positions. + if (table.importStatus === 'importing') { + return NextResponse.json( + { error: 'An import is already in progress for this table' }, + { status: 409 } + ) + } let mapping: CsvHeaderMapping | undefined - if (rawMapping) { - const mappingValidation = csvImportMappingSchema.safeParse(rawMapping) + if (fields.mapping) { + const mappingValidation = csvImportMappingSchema.safeParse(fields.mapping) if (!mappingValidation.success) { return NextResponse.json( { error: getValidationErrorMessage(mappingValidation.error) }, @@ -128,8 +153,8 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro } let createColumns: string[] | undefined - if (rawCreateColumns) { - const createColumnsValidation = csvImportCreateColumnsSchema.safeParse(rawCreateColumns) + if (fields.createColumns) { + const createColumnsValidation = csvImportCreateColumnsSchema.safeParse(fields.createColumns) if (!createColumnsValidation.success) { return NextResponse.json( { error: getValidationErrorMessage(createColumnsValidation.error) }, @@ -139,12 +164,19 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro createColumns = createColumnsValidation.data } - const buffer = await readFileToBufferWithLimit(file, { - maxBytes: CSV_MAX_FILE_SIZE_BYTES, - label: 'CSV import file', - }) const delimiter = extensionValidation.data === 'tsv' ? '\t' : ',' - const { headers, rows } = await parseCsvBuffer(buffer, delimiter) + const parser = createCsvParser(delimiter) + // `.pipe` doesn't forward source errors; forward them so the iterator throws. + file.stream.on('error', (streamErr) => parser.destroy(streamErr)) + file.stream.pipe(parser) + const rows: Record[] = [] + for await (const record of parser as AsyncIterable>) { + rows.push(record) + } + if (rows.length === 0) { + return NextResponse.json({ error: 'CSV file has no data rows' }, { status: 400 }) + } + const headers = Object.keys(rows[0]) let effectiveMapping = mapping ?? buildAutoMapping(headers, table.schema) let prospectiveTable: TableDefinition = table @@ -218,6 +250,19 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro const coerced = coerceRowsForTable(rows, prospectiveTable.schema, validation.effectiveMap) + // Atomically claim the table before writing. The pre-check above reads a checkAccess snapshot + // taken before the parse/validation; a background import could claim the table in that window. + // markTableImporting is the single atomic gate (same one the async kickoff uses) — released in + // the finally so a sync import can't write concurrently with a background one (corrupts replace). + const syncImportId = generateId() + if (!(await markTableImporting(tableId, syncImportId))) { + return NextResponse.json( + { error: 'An import is already in progress for this table' }, + { status: 409 } + ) + } + claimedImportId = syncImportId + if (mode === 'append') { if (prospectiveTable.rowCount + coerced.length > prospectiveTable.maxRows) { const deficit = prospectiveTable.rowCount + coerced.length - prospectiveTable.maxRows @@ -263,7 +308,7 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro logger.info(`[${requestId}] Append CSV imported`, { tableId: table.id, - fileName: file.name, + fileName: file.filename, mode, inserted, createdColumns: additions.length, @@ -280,7 +325,7 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro mappedColumns: validation.mappedHeaders, skippedHeaders: validation.skippedHeaders, unmappedColumns: validation.unmappedColumns, - sourceFile: file.name, + sourceFile: file.filename, }, }) } catch (err) { @@ -325,7 +370,7 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro logger.info(`[${requestId}] Replace CSV imported`, { tableId: table.id, - fileName: file.name, + fileName: file.filename, mode, deleted: result.deletedCount, inserted: result.insertedCount, @@ -343,7 +388,7 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro mappedColumns: validation.mappedHeaders, skippedHeaders: validation.skippedHeaders, unmappedColumns: validation.unmappedColumns, - sourceFile: file.name, + sourceFile: file.filename, }, }) } catch (err) { @@ -362,22 +407,23 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro throw err } } catch (error) { + if (isMultipartError(error)) return multipartErrorResponse(error) + const message = toError(error).message logger.error(`[${requestId}] CSV import into existing table failed:`, error) - const isSizeLimitError = - isPayloadSizeLimitError(error) || message.includes('CSV import file exceeds maximum size') const isClientError = message.includes('CSV file has no') || message.includes('already exists') || - message.includes('Invalid column name') || - isSizeLimitError + message.includes('Invalid column name') return NextResponse.json( { error: isClientError ? message : 'Failed to import CSV' }, - { - status: isSizeLimitError ? 413 : isClientError ? 400 : 500, - } + { status: isClientError ? 400 : 500 } ) + } finally { + fileStream?.destroy() + // Release before the response returns, so a client refetch never observes the transient claim. + if (claimedImportId) await releaseImportClaim(tableId, claimedImportId).catch(() => {}) } }) diff --git a/apps/sim/app/api/table/[tableId]/route.ts b/apps/sim/app/api/table/[tableId]/route.ts index 0e73ecaaeb..c0b018f854 100644 --- a/apps/sim/app/api/table/[tableId]/route.ts +++ b/apps/sim/app/api/table/[tableId]/route.ts @@ -68,6 +68,10 @@ export const GET = withRouteHandler(async (request: NextRequest, { params }: Tab table.updatedAt instanceof Date ? table.updatedAt.toISOString() : String(table.updatedAt), + importStatus: table.importStatus ?? null, + importId: table.importId ?? null, + importError: table.importError ?? null, + importRowsProcessed: table.importRowsProcessed ?? 0, }, }, }) diff --git a/apps/sim/app/api/table/import-async/route.test.ts b/apps/sim/app/api/table/import-async/route.test.ts new file mode 100644 index 0000000000..8ecdd2a923 --- /dev/null +++ b/apps/sim/app/api/table/import-async/route.test.ts @@ -0,0 +1,123 @@ +/** + * @vitest-environment node + */ +import { hybridAuthMockFns, permissionsMock, permissionsMockFns } from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { + mockCreateTable, + mockGetLimits, + mockListTables, + mockRunTableImport, + mockRunDetached, + MockTableConflictError, +} = vi.hoisted(() => ({ + mockCreateTable: vi.fn(), + mockGetLimits: vi.fn(), + mockListTables: vi.fn(), + mockRunTableImport: vi.fn(), + mockRunDetached: vi.fn(), + MockTableConflictError: class extends Error { + readonly code = 'TABLE_EXISTS' as const + }, +})) + +vi.mock('@sim/utils/id', () => ({ + generateId: vi.fn().mockReturnValue('import-id-123'), + generateShortId: vi.fn().mockReturnValue('short-id'), +})) + +vi.mock('@/lib/table', () => ({ + createTable: mockCreateTable, + getWorkspaceTableLimits: mockGetLimits, + listTables: mockListTables, + sanitizeName: (name: string) => name.replace(/[^a-zA-Z0-9_]/g, '_'), + TABLE_LIMITS: { MAX_TABLE_NAME_LENGTH: 128 }, + TableConflictError: MockTableConflictError, +})) +vi.mock('@/lib/table/import-runner', () => ({ runTableImport: mockRunTableImport })) +vi.mock('@/lib/core/utils/background', () => ({ + runDetached: mockRunDetached.mockImplementation( + (_label: string, work: () => Promise) => { + void work() + } + ), +})) +vi.mock('@/lib/workspaces/permissions/utils', () => permissionsMock) + +import { POST } from '@/app/api/table/import-async/route' + +function makeRequest(body: unknown): NextRequest { + return new NextRequest('http://localhost:3000/api/table/import-async', { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(body), + }) +} + +const validBody = { + workspaceId: 'workspace-1', + fileKey: 'workspace/workspace-1/123-data.csv', + fileName: 'data.csv', +} + +describe('POST /api/table/import-async', () => { + beforeEach(() => { + vi.clearAllMocks() + hybridAuthMockFns.mockCheckSessionOrInternalAuth.mockResolvedValue({ + success: true, + userId: 'user-1', + authType: 'session', + }) + permissionsMockFns.mockGetUserEntityPermissions.mockResolvedValue('write') + mockGetLimits.mockResolvedValue({ maxRowsPerTable: 1_000_000, maxTables: 50 }) + mockListTables.mockResolvedValue([]) + mockCreateTable.mockResolvedValue({ id: 'tbl_async', name: 'data' }) + mockRunTableImport.mockResolvedValue(undefined) + }) + + it('creates an importing table and kicks off the background import', async () => { + const response = await POST(makeRequest(validBody)) + const data = await response.json() + + expect(response.status).toBe(200) + expect(data.data).toEqual({ tableId: 'tbl_async', importId: 'import-id-123' }) + expect(mockCreateTable).toHaveBeenCalledWith( + expect.objectContaining({ importStatus: 'importing', importId: 'import-id-123' }), + expect.any(String) + ) + expect(mockRunTableImport).toHaveBeenCalledWith( + expect.objectContaining({ tableId: 'tbl_async', mode: 'create', delimiter: ',' }) + ) + }) + + it('uses a tab delimiter for .tsv files', async () => { + await POST(makeRequest({ ...validBody, fileName: 'data.tsv' })) + expect(mockRunTableImport).toHaveBeenCalledWith(expect.objectContaining({ delimiter: '\t' })) + }) + + it('returns 400 for unsupported extensions', async () => { + const response = await POST(makeRequest({ ...validBody, fileName: 'data.json' })) + expect(response.status).toBe(400) + expect(mockCreateTable).not.toHaveBeenCalled() + }) + + it('returns 401 when unauthenticated', async () => { + hybridAuthMockFns.mockCheckSessionOrInternalAuth.mockResolvedValue({ success: false }) + const response = await POST(makeRequest(validBody)) + expect(response.status).toBe(401) + }) + + it('returns 403 without write permission', async () => { + permissionsMockFns.mockGetUserEntityPermissions.mockResolvedValue('read') + const response = await POST(makeRequest(validBody)) + expect(response.status).toBe(403) + expect(mockCreateTable).not.toHaveBeenCalled() + }) + + it('returns 400 when the body is missing required fields', async () => { + const response = await POST(makeRequest({ workspaceId: 'workspace-1' })) + expect(response.status).toBe(400) + }) +}) diff --git a/apps/sim/app/api/table/import-async/route.ts b/apps/sim/app/api/table/import-async/route.ts new file mode 100644 index 0000000000..43fefeca9a --- /dev/null +++ b/apps/sim/app/api/table/import-async/route.ts @@ -0,0 +1,115 @@ +import { createLogger } from '@sim/logger' +import { generateId } from '@sim/utils/id' +import { type NextRequest, NextResponse } from 'next/server' +import { importTableAsyncContract } from '@/lib/api/contracts/tables' +import { parseRequest } from '@/lib/api/server' +import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid' +import { runDetached } from '@/lib/core/utils/background' +import { generateRequestId } from '@/lib/core/utils/request' +import { withRouteHandler } from '@/lib/core/utils/with-route-handler' +import { + createTable, + getWorkspaceTableLimits, + listTables, + sanitizeName, + TABLE_LIMITS, + TableConflictError, +} from '@/lib/table' +import { runTableImport } from '@/lib/table/import-runner' +import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils' + +const logger = createLogger('TableImportAsync') + +export const runtime = 'nodejs' +export const dynamic = 'force-dynamic' + +export const POST = withRouteHandler(async (request: NextRequest) => { + const requestId = generateRequestId() + + const authResult = await checkSessionOrInternalAuth(request, { requireWorkflowId: false }) + if (!authResult.success || !authResult.userId) { + return NextResponse.json({ error: 'Authentication required' }, { status: 401 }) + } + const userId = authResult.userId + + const parsed = await parseRequest(importTableAsyncContract, request, {}) + if (!parsed.success) return parsed.response + const { workspaceId, fileKey, fileName } = parsed.data.body + + const permission = await getUserEntityPermissions(userId, 'workspace', workspaceId) + if (permission !== 'write' && permission !== 'admin') { + return NextResponse.json({ error: 'Access denied' }, { status: 403 }) + } + // The fileKey is client-supplied — ensure it points at this workspace's storage prefix so a + // caller can't import another workspace's uploaded object. + if (!fileKey.startsWith(`workspace/${workspaceId}/`)) { + return NextResponse.json({ error: 'Invalid file key for workspace' }, { status: 400 }) + } + + const ext = fileName.split('.').pop()?.toLowerCase() + if (ext !== 'csv' && ext !== 'tsv') { + return NextResponse.json({ error: 'Only CSV and TSV files are supported' }, { status: 400 }) + } + const delimiter = ext === 'tsv' ? '\t' : ',' + + const planLimits = await getWorkspaceTableLimits(workspaceId) + const baseName = sanitizeName(fileName.replace(/\.[^.]+$/, ''), 'imported_table').slice( + 0, + TABLE_LIMITS.MAX_TABLE_NAME_LENGTH + ) + // Re-importing the same file shouldn't fail on a name collision — pick the next free + // `name_2`, `name_3`, … (matching how "New table" auto-names), keeping under the cap. + const existingNames = new Set( + (await listTables(workspaceId, { scope: 'all' })).map((t) => t.name.toLowerCase()) + ) + let tableName = baseName + for (let n = 2; existingNames.has(tableName.toLowerCase()); n++) { + const suffix = `_${n}` + tableName = `${baseName.slice(0, TABLE_LIMITS.MAX_TABLE_NAME_LENGTH - suffix.length)}${suffix}` + } + const importId = generateId() + + // Placeholder schema satisfies createTable's validation; the import worker infers the + // real columns from the file and overwrites it before any rows become visible. + let table: Awaited> + try { + table = await createTable( + { + name: tableName, + description: `Imported from ${fileName}`, + schema: { columns: [{ name: 'column_1', type: 'string' }] }, + workspaceId, + userId, + maxRows: planLimits.maxRowsPerTable, + maxTables: planLimits.maxTables, + importStatus: 'importing', + importId, + }, + requestId + ) + } catch (error) { + if (error instanceof TableConflictError) { + return NextResponse.json({ error: error.message }, { status: 409 }) + } + if (error instanceof Error && error.message.includes('maximum table limit')) { + return NextResponse.json({ error: error.message }, { status: 400 }) + } + throw error + } + + runDetached('table-import', () => + runTableImport({ + importId, + tableId: table.id, + workspaceId, + userId, + fileKey, + fileName, + delimiter, + mode: 'create', + }) + ) + + logger.info(`[${requestId}] Async CSV import started`, { tableId: table.id, importId, fileName }) + return NextResponse.json({ success: true, data: { tableId: table.id, importId } }) +}) diff --git a/apps/sim/app/api/table/import-csv/route.test.ts b/apps/sim/app/api/table/import-csv/route.test.ts index 9844bf6966..dc0bb0a53a 100644 --- a/apps/sim/app/api/table/import-csv/route.test.ts +++ b/apps/sim/app/api/table/import-csv/route.test.ts @@ -5,10 +5,11 @@ import { hybridAuthMockFns, permissionsMock, permissionsMockFns } from '@sim/tes import type { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mockCreateTable, mockParseCsvBuffer, mockGetWorkspaceTableLimits } = vi.hoisted(() => ({ +const { mockCreateTable, mockBatchInsertRows, mockDeleteTable, mockGetLimits } = vi.hoisted(() => ({ mockCreateTable: vi.fn(), - mockParseCsvBuffer: vi.fn(), - mockGetWorkspaceTableLimits: vi.fn(), + mockBatchInsertRows: vi.fn(), + mockDeleteTable: vi.fn(), + mockGetLimits: vi.fn(), })) vi.mock('@sim/utils/id', () => ({ @@ -16,46 +17,83 @@ vi.mock('@sim/utils/id', () => ({ generateShortId: vi.fn().mockReturnValue('short-id'), })) -vi.mock('@/lib/table', () => ({ - batchInsertRows: vi.fn(), - CSV_MAX_BATCH_SIZE: 1000, - CSV_MAX_FILE_SIZE_BYTES: 25 * 1024 * 1024, - coerceRowsForTable: vi.fn(), +// Mock only the DB-backed service/billing functions; the real `./import` helpers +// (createCsvParser, inferSchemaFromCsv, coerceRowsForTable, …) run for real so the +// streaming multipart + CSV pipeline is exercised end-to-end. +vi.mock('@/lib/table/service', () => ({ createTable: mockCreateTable, - deleteTable: vi.fn(), - getWorkspaceTableLimits: mockGetWorkspaceTableLimits, - inferSchemaFromCsv: vi.fn(), - parseCsvBuffer: mockParseCsvBuffer, - sanitizeName: vi.fn((name: string) => name), - TABLE_LIMITS: { - MAX_TABLE_NAME_LENGTH: 64, - }, + batchInsertRows: mockBatchInsertRows, + deleteTable: mockDeleteTable, })) - -vi.mock('@/app/api/table/utils', () => ({ - normalizeColumn: vi.fn((column) => column), -})) - +vi.mock('@/lib/table/billing', () => ({ getWorkspaceTableLimits: mockGetLimits })) +vi.mock('@/app/api/table/utils', async () => { + const { NextResponse } = await import('next/server') + return { + normalizeColumn: (column: unknown) => column, + csvProxyBodyCapResponse: () => null, + multipartErrorResponse: (error: { code: string; message: string }) => + NextResponse.json( + { error: error.message }, + { status: error.code === 'FILE_TOO_LARGE' ? 413 : 400 } + ), + } +}) vi.mock('@/lib/workspaces/permissions/utils', () => permissionsMock) import { POST } from '@/app/api/table/import-csv/route' -function createCsvFile(contents: string, name = 'data.csv', type = 'text/csv'): File { - return new File([contents], name, { type }) +type Part = + | { name: string; value: string } + | { name: string; filename: string; value: string; contentType?: string } + +const BOUNDARY = '----testboundaryCSV' + +function buildBody(parts: Part[]): Buffer { + const segments: Buffer[] = [] + for (const part of parts) { + let header = `--${BOUNDARY}\r\nContent-Disposition: form-data; name="${part.name}"` + if ('filename' in part) { + header += `; filename="${part.filename}"\r\nContent-Type: ${part.contentType ?? 'text/csv'}` + } + header += '\r\n\r\n' + segments.push(Buffer.from(header, 'utf8'), Buffer.from(part.value, 'utf8'), Buffer.from('\r\n')) + } + segments.push(Buffer.from(`--${BOUNDARY}--\r\n`, 'utf8')) + return Buffer.concat(segments) } -function createFormData(file: File): FormData { - const form = new FormData() - form.append('file', file) - form.append('workspaceId', 'workspace-1') - return form -} - -async function callPost(form: FormData) { - const req = { - formData: async () => form, +function makeRequest(parts: Part[], chunkSize?: number): NextRequest { + const body = buildBody(parts) + const stream = new ReadableStream({ + start(controller) { + if (chunkSize) { + for (let i = 0; i < body.length; i += chunkSize) { + controller.enqueue(new Uint8Array(body.subarray(i, i + chunkSize))) + } + } else { + controller.enqueue(new Uint8Array(body)) + } + controller.close() + }, + }) + return { + headers: new Headers({ 'content-type': `multipart/form-data; boundary=${BOUNDARY}` }), + body: stream, + signal: undefined, } as unknown as NextRequest - return POST(req) +} + +function csvWithRows(count: number): string { + const lines = ['name,age'] + for (let i = 0; i < count; i++) lines.push(`Person${i},${20 + (i % 50)}`) + return `${lines.join('\n')}\n` +} + +function uploadParts(csv: string): Part[] { + return [ + { name: 'workspaceId', value: 'workspace-1' }, + { name: 'file', filename: 'data.csv', value: csv }, + ] } describe('POST /api/table/import-csv', () => { @@ -67,38 +105,93 @@ describe('POST /api/table/import-csv', () => { authType: 'session', }) permissionsMockFns.mockGetUserEntityPermissions.mockResolvedValue('write') - mockGetWorkspaceTableLimits.mockResolvedValue({ - maxRowsPerTable: 1000, - maxTables: 10, - }) + mockGetLimits.mockResolvedValue({ maxRowsPerTable: 1_000_000, maxTables: 50 }) + mockCreateTable.mockImplementation(async (data) => ({ + id: 'tbl_1', + name: data.name, + description: data.description ?? null, + schema: data.schema, + workspaceId: data.workspaceId, + maxRows: data.maxRows, + rowCount: 0, + createdBy: 'user-1', + archivedAt: null, + createdAt: new Date(), + updatedAt: new Date(), + })) + mockBatchInsertRows.mockImplementation(async ({ rows }: { rows: unknown[] }) => + rows.map((_, i) => ({ id: `row-${i}` })) + ) + mockDeleteTable.mockResolvedValue(undefined) }) - it('returns 413 for oversized CSV files before reading their contents or creating a table', async () => { - const file = createCsvFile('name,age\nAlice,30') - Object.defineProperty(file, 'size', { - value: 26 * 1024 * 1024, - }) - const arrayBufferSpy = vi.spyOn(file, 'arrayBuffer') - - const response = await callPost(createFormData(file)) + it('streams a CSV upload into a new table and reports the row count', async () => { + const response = await POST(makeRequest(uploadParts(csvWithRows(250)))) const data = await response.json() - expect(response.status).toBe(413) - expect(data.error).toMatch(/CSV import file exceeds maximum size/) - expect(arrayBufferSpy).not.toHaveBeenCalled() - expect(mockParseCsvBuffer).not.toHaveBeenCalled() + expect(response.status).toBe(200) + expect(mockCreateTable).toHaveBeenCalledTimes(1) + expect(data.data.table.id).toBe('tbl_1') + expect(data.data.table.rowCount).toBe(250) + // 250 rows = a 100-row schema-sample batch + a 150-row remainder batch. + expect(mockBatchInsertRows).toHaveBeenCalledTimes(2) + }) + + it('parses a body delivered in tiny chunks (regression: missing final boundary)', async () => { + const response = await POST(makeRequest(uploadParts(csvWithRows(5)), 7)) + const data = await response.json() + + expect(response.status).toBe(200) + expect(data.data.table.rowCount).toBe(5) + }) + + it('returns 400 for a CSV with no data rows', async () => { + const response = await POST(makeRequest(uploadParts('name,age\n'))) + const data = await response.json() + + expect(response.status).toBe(400) + expect(data.error).toMatch(/no data rows/i) expect(mockCreateTable).not.toHaveBeenCalled() }) - it('accepts chunked multipart requests without a content-length header', async () => { - const req = { - headers: new Headers({ 'transfer-encoding': 'chunked' }), - formData: vi.fn(async () => createFormData(createCsvFile('name\nAlice'))), - } as unknown as NextRequest + it('returns 400 when the file precedes required fields', async () => { + const response = await POST( + makeRequest([ + { name: 'file', filename: 'data.csv', value: csvWithRows(3) }, + { name: 'workspaceId', value: 'workspace-1' }, + ]) + ) - const response = await POST(req) + expect(response.status).toBe(400) + expect(mockCreateTable).not.toHaveBeenCalled() + }) - expect(response.status).not.toBe(411) - expect(req.formData).toHaveBeenCalled() + it('returns 400 when no file part is present', async () => { + const response = await POST(makeRequest([{ name: 'workspaceId', value: 'workspace-1' }])) + expect(response.status).toBe(400) + expect(mockCreateTable).not.toHaveBeenCalled() + }) + + it('rolls back the created table when a batch insert fails mid-stream', async () => { + mockBatchInsertRows + .mockResolvedValueOnce(Array.from({ length: 100 }, () => ({ id: 'row' }))) + .mockRejectedValueOnce(new Error('insert boom')) + + const response = await POST(makeRequest(uploadParts(csvWithRows(250)))) + + expect(response.status).toBe(500) + expect(mockDeleteTable).toHaveBeenCalledWith('tbl_1', expect.any(String)) + }) + + it('returns 401 when unauthenticated', async () => { + hybridAuthMockFns.mockCheckSessionOrInternalAuth.mockResolvedValue({ success: false }) + const response = await POST(makeRequest(uploadParts(csvWithRows(3)))) + expect(response.status).toBe(401) + }) + + it('returns 403 without write permission', async () => { + permissionsMockFns.mockGetUserEntityPermissions.mockResolvedValue('read') + const response = await POST(makeRequest(uploadParts(csvWithRows(3)))) + expect(response.status).toBe(403) }) }) diff --git a/apps/sim/app/api/table/import-csv/route.ts b/apps/sim/app/api/table/import-csv/route.ts index 3192788920..4ab4d26920 100644 --- a/apps/sim/app/api/table/import-csv/route.ts +++ b/apps/sim/app/api/table/import-csv/route.ts @@ -1,3 +1,4 @@ +import type { Readable } from 'node:stream' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' @@ -5,163 +6,213 @@ import { type NextRequest, NextResponse } from 'next/server' import { csvExtensionSchema, csvImportFormSchema } from '@/lib/api/contracts/tables' import { getValidationErrorMessage } from '@/lib/api/server' import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid' +import { isMultipartError, readMultipart } from '@/lib/core/utils/multipart' import { generateRequestId } from '@/lib/core/utils/request' -import { - isPayloadSizeLimitError, - readFileToBufferWithLimit, - readFormDataWithLimit, -} from '@/lib/core/utils/stream-limits' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { batchInsertRows, CSV_MAX_BATCH_SIZE, CSV_MAX_FILE_SIZE_BYTES, + CSV_SCHEMA_SAMPLE_SIZE, coerceRowsForTable, + createCsvParser, createTable, deleteTable, getWorkspaceTableLimits, inferSchemaFromCsv, - parseCsvBuffer, sanitizeName, TABLE_LIMITS, + type TableDefinition, type TableSchema, } from '@/lib/table' import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils' -import { normalizeColumn } from '@/app/api/table/utils' +import { + csvProxyBodyCapResponse, + multipartErrorResponse, + normalizeColumn, +} from '@/app/api/table/utils' const logger = createLogger('TableImportCSV') -const MAX_MULTIPART_OVERHEAD_BYTES = 1024 * 1024 + +export const runtime = 'nodejs' +export const dynamic = 'force-dynamic' +export const maxDuration = 300 export const POST = withRouteHandler(async (request: NextRequest) => { const requestId = generateRequestId() + let fileStream: Readable | undefined try { const authResult = await checkSessionOrInternalAuth(request, { requireWorkflowId: false }) if (!authResult.success || !authResult.userId) { return NextResponse.json({ error: 'Authentication required' }, { status: 401 }) } + const userId = authResult.userId - const formData = await readFormDataWithLimit(request, { - maxBytes: CSV_MAX_FILE_SIZE_BYTES + MAX_MULTIPART_OVERHEAD_BYTES, - label: 'CSV import body', - }) - const validation = csvImportFormSchema.safeParse({ - file: formData.get('file'), - workspaceId: formData.get('workspaceId'), - }) + const oversize = csvProxyBodyCapResponse(request) + if (oversize) return oversize - if (!validation.success) { - const message = getValidationErrorMessage(validation.error) - const isSizeLimit = message.includes('File exceeds maximum allowed size') - return NextResponse.json( - { error: isSizeLimit ? 'CSV import file exceeds maximum size' : message }, - { status: isSizeLimit ? 413 : 400 } - ) + let parsed: Awaited> + try { + parsed = await readMultipart(request, { + maxFileBytes: CSV_MAX_FILE_SIZE_BYTES, + requiredFieldsBeforeFile: ['workspaceId'], + signal: request.signal, + }) + } catch (err) { + if (isMultipartError(err)) return multipartErrorResponse(err) + throw err } - const { file, workspaceId } = validation.data + const { fields, file } = parsed + if (!file) { + return NextResponse.json({ error: 'CSV file is required' }, { status: 400 }) + } + fileStream = file.stream - const permission = await getUserEntityPermissions(authResult.userId, 'workspace', workspaceId) + const workspaceIdResult = csvImportFormSchema.shape.workspaceId.safeParse(fields.workspaceId) + if (!workspaceIdResult.success) { + return NextResponse.json( + { error: getValidationErrorMessage(workspaceIdResult.error) }, + { status: 400 } + ) + } + const workspaceId = workspaceIdResult.data + + const permission = await getUserEntityPermissions(userId, 'workspace', workspaceId) if (permission !== 'write' && permission !== 'admin') { return NextResponse.json({ error: 'Access denied' }, { status: 403 }) } - const ext = file.name.split('.').pop()?.toLowerCase() - const extensionValidation = csvExtensionSchema.safeParse(ext) - if (!extensionValidation.success) { + const ext = file.filename.split('.').pop()?.toLowerCase() + const extensionResult = csvExtensionSchema.safeParse(ext) + if (!extensionResult.success) { return NextResponse.json( - { error: getValidationErrorMessage(extensionValidation.error) }, + { error: getValidationErrorMessage(extensionResult.error) }, { status: 400 } ) } + const delimiter = extensionResult.data === 'tsv' ? '\t' : ',' - const buffer = await readFileToBufferWithLimit(file, { - maxBytes: CSV_MAX_FILE_SIZE_BYTES, - label: 'CSV import file', - }) - const delimiter = extensionValidation.data === 'tsv' ? '\t' : ',' - const { headers, rows } = await parseCsvBuffer(buffer, delimiter) + const parser = createCsvParser(delimiter) + // `.pipe` doesn't forward source errors; forward them so the iterator throws. + file.stream.on('error', (err) => parser.destroy(err)) + file.stream.pipe(parser) - const { columns, headerToColumn } = inferSchemaFromCsv(headers, rows) - const tableName = sanitizeName(file.name.replace(/\.[^.]+$/, ''), 'imported_table').slice( - 0, - TABLE_LIMITS.MAX_TABLE_NAME_LENGTH - ) - const planLimits = await getWorkspaceTableLimits(workspaceId) - - const normalizedSchema: TableSchema = { - columns: columns.map(normalizeColumn), + interface ImportState { + table: TableDefinition + schema: TableSchema + headerToColumn: Map } - const table = await createTable( - { - name: tableName, - description: `Imported from ${file.name}`, - schema: normalizedSchema, - workspaceId, - userId: authResult.userId, - maxRows: planLimits.maxRowsPerTable, - maxTables: planLimits.maxTables, - }, - requestId - ) + const insertRows = async (rows: Record[], state: ImportState) => { + if (rows.length === 0) return 0 + const coerced = coerceRowsForTable(rows, state.schema, state.headerToColumn) + const result = await batchInsertRows( + { tableId: state.table.id, rows: coerced, workspaceId, userId }, + state.table, + generateId().slice(0, 8) + ) + return result.length + } + + /** Infer the schema from the buffered sample and create the (empty) table. */ + const buildTable = async (sampleRows: Record[]): Promise => { + const inferred = inferSchemaFromCsv(Object.keys(sampleRows[0]), sampleRows) + const schema: TableSchema = { columns: inferred.columns.map(normalizeColumn) } + const planLimits = await getWorkspaceTableLimits(workspaceId) + const tableName = sanitizeName(file.filename.replace(/\.[^.]+$/, ''), 'imported_table').slice( + 0, + TABLE_LIMITS.MAX_TABLE_NAME_LENGTH + ) + const table = await createTable( + { + name: tableName, + description: `Imported from ${file.filename}`, + schema, + workspaceId, + userId, + maxRows: planLimits.maxRowsPerTable, + maxTables: planLimits.maxTables, + }, + requestId + ) + return { table, schema, headerToColumn: inferred.headerToColumn } + } + + let state: ImportState | null = null + let inserted = 0 + const sample: Record[] = [] + let batch: Record[] = [] try { - const coerced = coerceRowsForTable(rows, normalizedSchema, headerToColumn) - let inserted = 0 - for (let i = 0; i < coerced.length; i += CSV_MAX_BATCH_SIZE) { - const batch = coerced.slice(i, i + CSV_MAX_BATCH_SIZE) - const batchRequestId = generateId().slice(0, 8) - const result = await batchInsertRows( - { tableId: table.id, rows: batch, workspaceId, userId: authResult.userId }, - table, - batchRequestId - ) - inserted += result.length + for await (const record of parser as AsyncIterable>) { + if (!state) { + sample.push(record) + if (sample.length >= CSV_SCHEMA_SAMPLE_SIZE) { + state = await buildTable(sample) + inserted += await insertRows(sample, state) + } + continue + } + batch.push(record) + if (batch.length >= CSV_MAX_BATCH_SIZE) { + inserted += await insertRows(batch, state) + batch = [] + } } - logger.info(`[${requestId}] CSV imported`, { - tableId: table.id, - fileName: file.name, - columns: columns.length, - rows: inserted, - }) - - return NextResponse.json({ - success: true, - data: { - table: { - id: table.id, - name: table.name, - description: table.description, - schema: normalizedSchema, - rowCount: inserted, - }, - }, - }) - } catch (insertError) { - await deleteTable(table.id, requestId).catch(() => {}) - throw insertError + if (!state) { + if (sample.length === 0) { + return NextResponse.json({ error: 'CSV file has no data rows' }, { status: 400 }) + } + state = await buildTable(sample) + inserted += await insertRows(sample, state) + } else { + inserted += await insertRows(batch, state) + } + } catch (streamError) { + if (state) await deleteTable(state.table.id, requestId).catch(() => {}) + throw streamError } + + logger.info(`[${requestId}] CSV imported`, { + tableId: state.table.id, + fileName: file.filename, + columns: state.schema.columns.length, + rows: inserted, + }) + + return NextResponse.json({ + success: true, + data: { + table: { + id: state.table.id, + name: state.table.name, + description: state.table.description, + schema: state.schema, + rowCount: inserted, + }, + }, + }) } catch (error) { + if (isMultipartError(error)) return multipartErrorResponse(error) + const message = toError(error).message logger.error(`[${requestId}] CSV import failed:`, error) - const isSizeLimitError = - isPayloadSizeLimitError(error) || message.includes('CSV import file exceeds maximum size') const isClientError = message.includes('maximum table limit') || message.includes('CSV file has no') || message.includes('Invalid table name') || message.includes('Invalid schema') || - message.includes('already exists') || - isSizeLimitError + message.includes('already exists') return NextResponse.json( { error: isClientError ? message : 'Failed to import CSV' }, - { - status: isSizeLimitError ? 413 : isClientError ? 400 : 500, - } + { status: isClientError ? 400 : 500 } ) + } finally { + fileStream?.destroy() } }) diff --git a/apps/sim/app/api/table/route.ts b/apps/sim/app/api/table/route.ts index 89a48b8089..2d97dc4f63 100644 --- a/apps/sim/app/api/table/route.ts +++ b/apps/sim/app/api/table/route.ts @@ -203,6 +203,10 @@ export const GET = withRouteHandler(async (request: NextRequest) => { : t.archivedAt ? String(t.archivedAt) : null, + importStatus: t.importStatus ?? null, + importId: t.importId ?? null, + importError: t.importError ?? null, + importRowsProcessed: t.importRowsProcessed ?? 0, } }) diff --git a/apps/sim/app/api/table/utils.ts b/apps/sim/app/api/table/utils.ts index 114271a940..eef507c94b 100644 --- a/apps/sim/app/api/table/utils.ts +++ b/apps/sim/app/api/table/utils.ts @@ -5,12 +5,46 @@ import { deleteTableColumnBodySchema, updateTableColumnBodySchema, } from '@/lib/api/contracts/tables' +import type { MultipartError } from '@/lib/core/utils/multipart' import type { ColumnDefinition, TableDefinition } from '@/lib/table' import { getTableById } from '@/lib/table' import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils' const logger = createLogger('TableUtils') +/** + * Next.js buffers the request body for the proxy and silently truncates it past this + * size (`experimental.proxyClientMaxBodySize`, default 10MB). The synchronous CSV + * import routes reject bodies over the cap up front; larger files use the async + * direct-to-storage path instead. + */ +export const CSV_IMPORT_PROXY_BODY_CAP_BYTES = 10 * 1024 * 1024 + +/** 413 response when a synchronous CSV upload would exceed (and be truncated at) the proxy cap; `null` otherwise. */ +export function csvProxyBodyCapResponse(request: { headers: Headers }): NextResponse | null { + const contentLength = Number(request.headers.get('content-length') ?? 0) + if (contentLength > CSV_IMPORT_PROXY_BODY_CAP_BYTES) { + return NextResponse.json( + { + error: + 'File too large to import through the server. Files over 10MB import in the background.', + }, + { status: 413 } + ) + } + return null +} + +/** Maps a {@link MultipartError} from the streaming CSV parser to its HTTP response. */ +export function multipartErrorResponse(error: MultipartError): NextResponse { + if (error.code === 'FILE_TOO_LARGE') { + return NextResponse.json({ error: 'CSV import file exceeds maximum size' }, { status: 413 }) + } + const message = + error.code === 'NO_FILE' ? 'CSV file is required' : `Invalid CSV upload: ${error.message}` + return NextResponse.json({ error: message }, { status: 400 }) +} + interface TableAccessResult { hasAccess: true table: TableDefinition diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/hooks/use-table-event-stream.ts b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/hooks/use-table-event-stream.ts index 78eb7e9610..29cfbfd947 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/hooks/use-table-event-stream.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/hooks/use-table-event-stream.ts @@ -4,7 +4,7 @@ import { useEffect, useRef } from 'react' import { createLogger } from '@sim/logger' import { useQueryClient } from '@tanstack/react-query' import type { ActiveDispatch } from '@/lib/api/contracts/tables' -import type { RowData, RowExecutionMetadata, RowExecutions } from '@/lib/table' +import type { RowData, RowExecutionMetadata, RowExecutions, TableDefinition } from '@/lib/table' import { isExecInFlight } from '@/lib/table/deps' import type { TableEvent, TableEventEntry } from '@/lib/table/events' import { snapshotAndMutateRows, type TableRunState, tableKeys } from '@/hooks/queries/tables' @@ -92,6 +92,17 @@ export function useTableEventStream({ }, DISPATCH_INVALIDATE_DEBOUNCE_MS) } + // Live-fill: import progress ticks arrive every N rows; coalesce the row + // refetches into one per debounce window instead of refetching per tick. + let importInvalidateTimer: ReturnType | null = null + const scheduleRowsInvalidate = (): void => { + if (importInvalidateTimer !== null) clearTimeout(importInvalidateTimer) + importInvalidateTimer = setTimeout(() => { + importInvalidateTimer = null + void queryClient.invalidateQueries({ queryKey: tableKeys.rowsRoot(tableId) }) + }, DISPATCH_INVALIDATE_DEBOUNCE_MS) + } + // Keeps the per-row gutter (`runningByRowId`) live between dispatch events. // `runningCellCount` (the "X running" badge) is NOT touched here — it's the // server's dispatch-scope count, seeded optimistically on click and @@ -213,6 +224,41 @@ export function useTableEventStream({ scheduleDispatchInvalidate() } + const applyImport = (event: Extract): void => { + const { status, progress, error, importId } = event + const isTerminal = status === 'ready' || status === 'failed' || status === 'canceled' + + // The SSE buffer replays on (re)connect and can hold a *prior* import's events for this + // table. Ignore anything from a superseded run, and don't trust a replayed terminal before + // we know the active run's id. + const prev = queryClient.getQueryData(tableKeys.detail(tableId)) + const lockedId = prev?.importId + if (lockedId && importId && importId !== lockedId) return + if (!lockedId && isTerminal) return + + queryClient.setQueryData(tableKeys.detail(tableId), (p) => + p + ? { + ...p, + importStatus: status, + importId: importId ?? p.importId, + importRowsProcessed: progress ?? p.importRowsProcessed, + importError: error ?? null, + } + : p + ) + // The header tray + completion toast are owned by `useImportTrayPoll`. Here we only keep the + // detail cache + grid in sync: live-fill rows per batch (debounced), and on the terminal + // event refetch rows + the definition (the worker may have rewritten the schema). + if (isTerminal) { + if (importInvalidateTimer !== null) clearTimeout(importInvalidateTimer) + void queryClient.invalidateQueries({ queryKey: tableKeys.rowsRoot(tableId) }) + void queryClient.invalidateQueries({ queryKey: tableKeys.detail(tableId) }) + } else { + scheduleRowsInvalidate() + } + } + const applyUsageLimit = (event: Extract): void => { // Drop the halted dispatch from the overlay so the "running" UI clears // immediately (the dispatcher was marked complete server-side). Cascade / @@ -283,6 +329,7 @@ export function useTableEventStream({ savePointer(tableId, lastEventId) if (entry.event?.kind === 'cell') applyCell(entry.event) else if (entry.event?.kind === 'dispatch') applyDispatch(entry.event) + else if (entry.event?.kind === 'import') applyImport(entry.event) else if (entry.event?.kind === 'usageLimitReached') applyUsageLimit(entry.event) } catch (err) { logger.warn('Failed to parse table event', { tableId, err }) @@ -317,6 +364,7 @@ export function useTableEventStream({ cancelled = true if (reconnectTimer !== null) clearTimeout(reconnectTimer) if (dispatchInvalidateTimer !== null) clearTimeout(dispatchInvalidateTimer) + if (importInvalidateTimer !== null) clearTimeout(importInvalidateTimer) eventSource?.close() eventSource = null } diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/table.tsx b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/table.tsx index 5ba7f380d0..55993d1fa6 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/table.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/table.tsx @@ -27,6 +27,7 @@ import { import { LogDetails } from '@/app/workspace/[workspaceId]/logs/components' import { useUserPermissionsContext } from '@/app/workspace/[workspaceId]/providers/workspace-permissions-provider' import { ImportCsvDialog } from '@/app/workspace/[workspaceId]/tables/components/import-csv-dialog' +import { ImportProgressMenu } from '@/app/workspace/[workspaceId]/tables/components/import-progress-menu' import { useLogByExecutionId } from '@/hooks/queries/logs' import { downloadTableExport, @@ -519,13 +520,16 @@ export function Table({ createTrigger={createTrigger} actions={headerActions} leadingActions={ - selection.totalRunning > 0 || selection.hasActiveDispatch ? ( - - ) : null + <> + + {selection.totalRunning > 0 || selection.hasActiveDispatch ? ( + + ) : null} + } /> )} diff --git a/apps/sim/app/workspace/[workspaceId]/tables/components/import-csv-dialog/import-csv-dialog.tsx b/apps/sim/app/workspace/[workspaceId]/tables/components/import-csv-dialog/import-csv-dialog.tsx index b6f7e5beca..b104f5c6c3 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/components/import-csv-dialog/import-csv-dialog.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/components/import-csv-dialog/import-csv-dialog.tsx @@ -25,14 +25,24 @@ import { toast, } from '@/components/emcn' import { cn } from '@/lib/core/utils/cn' +import { CSV_ASYNC_IMPORT_THRESHOLD_BYTES } from '@/lib/table/constants' import { buildAutoMapping, parseCsvBuffer } from '@/lib/table/import' import type { TableDefinition } from '@/lib/table/types' -import { type CsvImportMode, useImportCsvIntoTable } from '@/hooks/queries/tables' +import { + type CsvImportMode, + cancelTableImport, + useImportCsvIntoTable, + useImportCsvIntoTableAsync, +} from '@/hooks/queries/tables' +import { useImportTrayStore } from '@/stores/table/import-tray/store' const logger = createLogger('ImportCsvDialog') const MAX_SAMPLE_ROWS = 5 const MAX_EXAMPLES_IN_ERROR = 3 +/** Bytes read for the preview/mapping. We never parse the whole file client-side — the importer + * streams it server-side and the DB trigger enforces the row limit. */ +const CSV_PREVIEW_BYTES = 512 * 1024 /** * Sentinel value for the "Do not import" option in the mapping combobox. The * whitespace is intentional: valid column names must match `NAME_PATTERN` @@ -94,7 +104,18 @@ interface ParsedCsv { file: File headers: string[] sampleRows: Record[] - totalRows: number +} + +/** Parses the head of a CSV/TSV for the mapping + sample, dropping any truncated final line. */ +async function parseCsvPreview(file: File, delimiter: ',' | '\t') { + const sliced = file.size > CSV_PREVIEW_BYTES + const blob = sliced ? file.slice(0, CSV_PREVIEW_BYTES) : file + let bytes = new Uint8Array(await blob.arrayBuffer()) + if (sliced) { + const lastNewline = bytes.lastIndexOf(0x0a) + if (lastNewline > 0) bytes = bytes.subarray(0, lastNewline + 1) + } + return parseCsvBuffer(bytes, delimiter) } export function ImportCsvDialog({ @@ -114,6 +135,7 @@ export function ImportCsvDialog({ const [isDragging, setIsDragging] = useState(false) const fileInputRef = useRef(null) const importMutation = useImportCsvIntoTable() + const importAsyncMutation = useImportCsvIntoTableAsync() function resetState() { setParsed(null) @@ -161,15 +183,13 @@ export function ImportCsvDialog({ setParsing(true) setParseError(null) try { - const arrayBuffer = await file.arrayBuffer() - const delimiter = ext === 'tsv' ? '\t' : ',' - const { headers, rows } = await parseCsvBuffer(new Uint8Array(arrayBuffer), delimiter) + const delimiter: ',' | '\t' = ext === 'tsv' ? '\t' : ',' + const { headers, rows } = await parseCsvPreview(file, delimiter) const autoMapping = buildAutoMapping(headers, table.schema) setParsed({ file, headers, sampleRows: rows.slice(0, MAX_SAMPLE_ROWS), - totalRows: rows.length, }) setMapping(autoMapping) } catch (err) { @@ -283,28 +303,69 @@ export function ImportCsvDialog({ } }, [mapping, parsed?.headers, table.schema.columns, createHeaders]) - const appendCapacityDeficit = - parsed && mode === 'append' && table.rowCount + parsed.totalRows > table.maxRows - ? table.rowCount + parsed.totalRows - table.maxRows - : 0 - - const replaceCapacityDeficit = - parsed && mode === 'replace' && parsed.totalRows > table.maxRows - ? parsed.totalRows - table.maxRows - : 0 - const canSubmit = parsed !== null && !importMutation.isPending && + !importAsyncMutation.isPending && missingRequired.length === 0 && duplicateTargets.length === 0 && - mappedCount + createCount > 0 && - appendCapacityDeficit === 0 && - replaceCapacityDeficit === 0 + mappedCount + createCount > 0 async function handleSubmit() { if (!parsed || !canSubmit) return setSubmitError(null) + const createColumns = createHeaders.size > 0 ? [...createHeaders] : undefined + + // Large files can't be POSTed through the server (request-body cap) — upload them + // straight to storage and import in the background instead. Seed the header tray and + // close the dialog immediately so the indicator is visible during the upload, then run + // the upload + kickoff in the background (don't block the dialog on it). + if (parsed.file.size >= CSV_ASYNC_IMPORT_THRESHOLD_BYTES) { + useImportTrayStore.getState().startUpload({ + uploadId: table.id, + workspaceId, + title: parsed.file.name, + }) + onOpenChange(false) + toast({ + message: `Importing "${parsed.file.name}" into "${table.name}"…`, + action: { + label: 'View', + onClick: () => useImportTrayStore.getState().setMenuOpen(true), + }, + }) + importAsyncMutation.mutate( + { + workspaceId, + tableId: table.id, + file: parsed.file, + mode, + mapping, + createColumns, + onProgress: (percent) => { + useImportTrayStore.getState().setUploadPercent(table.id, percent) + }, + }, + { + onSuccess: (data) => { + useImportTrayStore.getState().endUpload(table.id) + // The server row drives the tray once the list refetches. If canceled mid-upload, flag + // the id so it's not shown and cancel the worker server-side. + if (useImportTrayStore.getState().consumeCanceled(table.id) && data?.importId) { + useImportTrayStore.getState().cancel(table.id) + void cancelTableImport(workspaceId, table.id, data.importId).catch(() => {}) + } + }, + onError: (err) => { + useImportTrayStore.getState().endUpload(table.id) + toast.error(getErrorMessage(err, 'Failed to start import')) + logger.error('Async CSV import failed to start', err) + }, + } + ) + return + } + try { const result = await importMutation.mutateAsync({ workspaceId, @@ -312,7 +373,7 @@ export function ImportCsvDialog({ file: parsed.file, mode, mapping, - createColumns: createHeaders.size > 0 ? [...createHeaders] : undefined, + createColumns, }) const data = result.data if (mode === 'append') { @@ -334,11 +395,7 @@ export function ImportCsvDialog({ } } - const hasWarning = - missingRequired.length > 0 || - duplicateTargets.length > 0 || - appendCapacityDeficit > 0 || - replaceCapacityDeficit > 0 + const hasWarning = missingRequired.length > 0 || duplicateTargets.length > 0 return ( @@ -397,7 +454,7 @@ export function ImportCsvDialog({ {parsed.file.name} - {parsed.totalRows.toLocaleString()} rows · {parsed.headers.length} columns + {parsed.headers.length} columns + + + {imports.map((row) => { + const stage = getImportStage(row) + return ( + cancel(row) : undefined} + onDismiss={stage.dismissible ? () => dismiss(row.id) : undefined} + /> + ) + })} + + + ) +} diff --git a/apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/import-stage.ts b/apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/import-stage.ts new file mode 100644 index 0000000000..56e0fb7773 --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/import-stage.ts @@ -0,0 +1,58 @@ +import type { ImportRow } from './use-workspace-imports' + +type ProgressStatus = 'pending' | 'success' | 'error' + +/** Uniform view model for a tray entry — every stage fills the same slots. */ +export interface ImportStageView { + status: ProgressStatus + /** Primary line: `{status} {name}`, e.g. `Processing data.csv`. */ + title: string + /** Right-aligned on the title row: the percent (when known). */ + meta?: string + /** Secondary line: the row count, or the error message on failure. */ + detail?: string + dismissible: boolean +} + +/** + * Maps a tray entry to the stage shown in the import dropdown. The single place the import + * stages (Uploading → Processing → Imported / Failed) are defined; the row component just + * renders the returned slots, so every stage looks consistent: `{status} {name}`. While + * uploading, the right slot shows the byte-based upload percent (from the client XHR). Once the + * server is processing we only know the committed row count (polled from the table row), so the + * detail line reads `{rows} rows` with no percent. + */ +export function getImportStage(entry: ImportRow): ImportStageView { + const rows = entry.rowsProcessed.toLocaleString() + const name = entry.title + const meta = typeof entry.percent === 'number' ? `${entry.percent}%` : undefined + + if (entry.phase === 'failed') { + return { + status: 'error', + title: `Failed ${name}`, + detail: entry.error ?? 'Something went wrong', + dismissible: true, + } + } + + if (entry.phase === 'ready') { + return { + status: 'success', + title: `Imported ${name}`, + detail: `${rows} rows`, + dismissible: true, + } + } + + // importing: rows only start arriving once the worker is processing; before that it's the upload. + if (entry.rowsProcessed > 0) { + return { + status: 'pending', + title: `Processing ${name}`, + detail: `${rows} rows`, + dismissible: false, + } + } + return { status: 'pending', title: `Uploading ${name}`, meta, dismissible: false } +} diff --git a/apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/index.ts b/apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/index.ts new file mode 100644 index 0000000000..b7ade906b1 --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/index.ts @@ -0,0 +1 @@ +export * from './import-progress-menu' diff --git a/apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/use-workspace-imports.ts b/apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/use-workspace-imports.ts new file mode 100644 index 0000000000..a4f1acb25e --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/tables/components/import-progress-menu/use-workspace-imports.ts @@ -0,0 +1,118 @@ +'use client' + +import { useEffect, useMemo, useRef } from 'react' +import { useShallow } from 'zustand/react/shallow' +import { toast } from '@/components/emcn' +import { useTablesList } from '@/hooks/queries/tables' +import { useImportTrayStore } from '@/stores/table/import-tray/store' + +const READY_AUTO_CLEAR_MS = 6000 +const POLL_INTERVAL_MS = 2000 + +export type ImportPhase = 'importing' | 'ready' | 'failed' + +/** A row rendered in the import tray. Importing rows come live from the table list; uploads are + * client-only until their server row exists. */ +export interface ImportRow { + id: string + workspaceId: string + title: string + phase: ImportPhase + rowsProcessed: number + /** Upload byte percent (upload phase only). */ + percent?: number + error?: string + importId?: string +} + +/** + * Single source for the import tray. Importing rows are derived live from the table list (polled + * while any import is in flight) rather than mirrored into a store; the store only supplies + * optimistic uploads and which terminal completions to surface this session. Also fires the + * completion toasts on the importing → terminal transition. + */ +export function useWorkspaceImports( + workspaceId: string | undefined, + scopeTableId?: string +): ImportRow[] { + const { data: tables } = useTablesList(workspaceId, 'active', { + refetchInterval: (list) => + list?.some((t) => t.importStatus === 'importing') ? POLL_INTERVAL_MS : false, + }) + + const prevStatus = useRef>(new Map()) + useEffect(() => { + if (!tables) return + const store = useImportTrayStore.getState() + for (const table of tables) { + const before = prevStatus.current.get(table.id) + const now = table.importStatus ?? 'none' + if (before === 'importing' && now === 'ready') { + const rows = (table.importRowsProcessed ?? 0).toLocaleString() + toast.success(`Imported ${rows} rows into "${table.name}"`) + store.notify(table.id) + setTimeout(() => useImportTrayStore.getState().dismiss(table.id), READY_AUTO_CLEAR_MS) + } else if (before === 'importing' && now === 'failed') { + toast.error(table.importError || `Import failed for "${table.name}"`) + store.notify(table.id) + } + if (now !== 'importing' && store.isCanceled(table.id)) store.consumeCanceled(table.id) + prevStatus.current.set(table.id, now) + } + }, [tables]) + + const uploads = useImportTrayStore(useShallow((s) => Object.values(s.uploads))) + const notified = useImportTrayStore((s) => s.notified) + const canceledIds = useImportTrayStore((s) => s.canceledIds) + + return useMemo(() => { + const rows: ImportRow[] = [] + const seen = new Set() + + for (const table of tables ?? []) { + if (scopeTableId && table.id !== scopeTableId) continue + if (table.importStatus === 'importing') { + if (canceledIds[table.id]) continue + rows.push({ + id: table.id, + workspaceId: table.workspaceId, + title: table.name, + phase: 'importing', + rowsProcessed: table.importRowsProcessed ?? 0, + importId: table.importId ?? undefined, + }) + seen.add(table.id) + } else if ( + (table.importStatus === 'ready' || table.importStatus === 'failed') && + notified[table.id] + ) { + rows.push({ + id: table.id, + workspaceId: table.workspaceId, + title: table.name, + phase: table.importStatus, + rowsProcessed: table.importRowsProcessed ?? 0, + error: table.importError ?? undefined, + }) + seen.add(table.id) + } + } + + for (const upload of uploads) { + if (upload.workspaceId !== workspaceId) continue + if (scopeTableId && upload.uploadId !== scopeTableId) continue + if (canceledIds[upload.uploadId] || seen.has(upload.uploadId)) continue + rows.push({ + id: upload.uploadId, + workspaceId: upload.workspaceId, + title: upload.title, + phase: 'importing', + rowsProcessed: 0, + percent: upload.percent, + }) + } + + rows.sort((a, b) => (a.phase === b.phase ? 0 : a.phase === 'importing' ? -1 : 1)) + return rows + }, [tables, uploads, notified, canceledIds, workspaceId, scopeTableId]) +} diff --git a/apps/sim/app/workspace/[workspaceId]/tables/components/index.ts b/apps/sim/app/workspace/[workspaceId]/tables/components/index.ts index 4a74ec9548..ea88eac2fd 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/components/index.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/components/index.ts @@ -1,3 +1,4 @@ export * from './import-csv-dialog' +export * from './import-progress-menu' export * from './table-context-menu' export * from './tables-list-context-menu' diff --git a/apps/sim/app/workspace/[workspaceId]/tables/tables.tsx b/apps/sim/app/workspace/[workspaceId]/tables/tables.tsx index 5cf881a2f4..14639a60d2 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/tables.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/tables.tsx @@ -2,6 +2,7 @@ import { useCallback, useEffect, useMemo, useRef, useState } from 'react' import { createLogger } from '@sim/logger' +import { generateId } from '@sim/utils/id' import { useParams, useRouter } from 'next/navigation' import type { ComboboxOption } from '@/components/emcn' import { @@ -18,7 +19,7 @@ import { } from '@/components/emcn' import { Columns3, Rows3, Table as TableIcon } from '@/components/emcn/icons' import type { TableDefinition } from '@/lib/table' -import { generateUniqueTableName } from '@/lib/table/constants' +import { CSV_ASYNC_IMPORT_THRESHOLD_BYTES, generateUniqueTableName } from '@/lib/table/constants' import type { FilterTag, ResourceColumn, @@ -30,20 +31,24 @@ import { ownerCell, Resource, timeCell } from '@/app/workspace/[workspaceId]/com import { useUserPermissionsContext } from '@/app/workspace/[workspaceId]/providers/workspace-permissions-provider' import { ImportCsvDialog, + ImportProgressMenu, TablesListContextMenu, } from '@/app/workspace/[workspaceId]/tables/components' import { TableContextMenu } from '@/app/workspace/[workspaceId]/tables/components/table-context-menu' import { useContextMenu } from '@/app/workspace/[workspaceId]/w/components/sidebar/hooks' import { + cancelTableImport, downloadTableExport, useCreateTable, useDeleteTable, + useImportCsvAsync, useTablesList, useUploadCsvToTable, } from '@/hooks/queries/tables' import { useWorkspaceMembersQuery } from '@/hooks/queries/workspace' import { useDebounce } from '@/hooks/use-debounce' import { usePermissionConfig } from '@/hooks/use-permission-config' +import { useImportTrayStore } from '@/stores/table/import-tray/store' const logger = createLogger('Tables') @@ -79,6 +84,7 @@ export function Tables() { const deleteTable = useDeleteTable(workspaceId) const createTable = useCreateTable(workspaceId) const uploadCsv = useUploadCsvToTable() + const importCsvAsync = useImportCsvAsync() const [isDeleteDialogOpen, setIsDeleteDialogOpen] = useState(false) const [isImportDialogOpen, setIsImportDialogOpen] = useState(false) @@ -91,8 +97,6 @@ export function Tables() { } | null>(null) const [rowCountFilter, setRowCountFilter] = useState([]) const [ownerFilter, setOwnerFilter] = useState([]) - const [uploading, setUploading] = useState(false) - const [uploadProgress, setUploadProgress] = useState({ completed: 0, total: 0 }) const csvInputRef = useRef(null) const { @@ -386,25 +390,65 @@ export function Tables() { const list = e.target.files if (!list || list.length === 0 || !workspaceId) return + // Reset the input up front so the user can immediately pick another CSV (even the same + // file) while this batch is still uploading in the background — imports never block. + const csvFiles = Array.from(list).filter((f) => { + const ext = f.name.split('.').pop()?.toLowerCase() + return ext === 'csv' || ext === 'tsv' + }) + if (e.target) e.target.value = '' + + if (csvFiles.length === 0) { + toast.error('No CSV or TSV files selected') + return + } + try { - setUploading(true) - - const csvFiles = Array.from(list).filter((f) => { - const ext = f.name.split('.').pop()?.toLowerCase() - return ext === 'csv' || ext === 'tsv' - }) - - if (csvFiles.length === 0) { - toast.error('No CSV or TSV files selected') - return - } - - setUploadProgress({ completed: 0, total: csvFiles.length }) - for (let i = 0; i < csvFiles.length; i++) { + const file = csvFiles[i] try { - const result = await uploadCsv.mutateAsync({ workspaceId, file: csvFiles[i] }) + // Large files can't be POSTed through the server (request-body cap) — upload + // them straight to storage and import in the background instead. Show the + // indicator immediately under a temporary id (the real table id doesn't exist + // until kickoff returns), then swap to the real id. Don't redirect — the table + // is still empty/importing, so stay on the list and let the indicator track it. + if (file.size >= CSV_ASYNC_IMPORT_THRESHOLD_BYTES) { + const pendingId = `pending_${generateId()}` + useImportTrayStore + .getState() + .startUpload({ uploadId: pendingId, workspaceId, title: file.name }) + toast({ + message: `Importing "${file.name}"…`, + action: { + label: 'View', + onClick: () => useImportTrayStore.getState().setMenuOpen(true), + }, + }) + try { + const result = await importCsvAsync.mutateAsync({ + workspaceId, + file, + onProgress: (percent) => { + useImportTrayStore.getState().setUploadPercent(pendingId, percent) + }, + }) + useImportTrayStore.getState().endUpload(pendingId) + // The server row drives the tray once the list refetches (mutation invalidates it). + // If canceled mid-upload, flag the real id so it's not shown and cancel server-side. + if (result?.tableId && useImportTrayStore.getState().consumeCanceled(pendingId)) { + useImportTrayStore.getState().cancel(result.tableId) + void cancelTableImport(workspaceId, result.tableId, result.importId).catch( + () => {} + ) + } + } catch (err) { + useImportTrayStore.getState().endUpload(pendingId) + throw err + } + continue + } + const result = await uploadCsv.mutateAsync({ workspaceId, file }) if (csvFiles.length === 1) { const tableId = result?.data?.table?.id if (tableId) { @@ -413,21 +457,13 @@ export function Tables() { } } catch (err) { logger.error('Error uploading CSV:', err) - } finally { - setUploadProgress({ completed: i + 1, total: csvFiles.length }) } } } catch (err) { logger.error('Error uploading CSV:', err) - } finally { - setUploading(false) - setUploadProgress({ completed: 0, total: 0 }) - if (csvInputRef.current) { - csvInputRef.current.value = '' - } } }, - [workspaceId, router, uploadCsv] + [workspaceId, router, uploadCsv, importCsvAsync] ) const handleListUploadCsv = useCallback(() => { @@ -435,13 +471,6 @@ export function Tables() { closeListContextMenu() }, [closeListContextMenu]) - const uploadButtonLabel = - uploading && uploadProgress.total > 0 - ? `${uploadProgress.completed}/${uploadProgress.total}` - : uploading - ? 'Uploading...' - : 'Import CSV' - const handleCreateTable = useCallback(async () => { const existingNames = tables.map((t) => t.name) const name = generateUniqueTableName(existingNames) @@ -470,7 +499,7 @@ export function Tables() { create={{ label: 'New table', onClick: handleCreateTable, - disabled: uploading || userPermissions.canEdit !== true || createTable.isPending, + disabled: userPermissions.canEdit !== true || createTable.isPending, }} search={searchConfig} sort={sortConfig} @@ -478,12 +507,13 @@ export function Tables() { filterTags={filterTags} headerActions={[ { - label: uploadButtonLabel, + label: 'Import CSV', icon: Upload, onClick: () => csvInputRef.current?.click(), - disabled: uploading || userPermissions.canEdit !== true, + disabled: userPermissions.canEdit !== true, }, ]} + leadingActions={} columns={COLUMNS} rows={rows} onRowClick={handleRowClick} @@ -497,7 +527,6 @@ export function Tables() { type='file' className='hidden' onChange={handleCsvChange} - disabled={uploading} accept='.csv,.tsv' multiple /> @@ -509,7 +538,7 @@ export function Tables() { onCreateTable={handleCreateTable} onUploadCsv={handleListUploadCsv} disableCreate={userPermissions.canEdit !== true || createTable.isPending} - disableUpload={uploading || userPermissions.canEdit !== true} + disableUpload={userPermissions.canEdit !== true} /> ['status']> + +const ICON_CLASS = 'mt-px size-[14px] shrink-0' + +function StatusIcon({ status }: { status: ProgressStatus }) { + if (status === 'success') + return + if (status === 'error') + return + return +} + +export interface ProgressItemProps + extends Omit, 'title'>, + VariantProps { + status: ProgressStatus + /** Primary line (truncated). */ + title: React.ReactNode + /** Right-aligned status on the title row, e.g. `Processing · 45%`. */ + meta?: React.ReactNode + /** Secondary line under the title. */ + detail?: React.ReactNode + /** Renders a dismiss button when provided (terminal rows). */ + onDismiss?: () => void + /** Accessible label for the dismiss button. */ + dismissLabel?: string + /** Renders a cancel button when provided (active rows); takes precedence over `onDismiss`. */ + onCancel?: () => void +} + +/** + * A single status/progress row: a leading status icon (spinner / check / alert), a primary + * title, an optional right-aligned `meta` (status + percent), an optional secondary `detail` + * line, and an optional dismiss button. Every status renders through the same fixed layout — + * only the values change — so rows stay visually consistent across stages. + * + * @example + * ```tsx + * + * + * + * ``` + */ +const ProgressItem = forwardRef(function ProgressItem( + { className, status, title, meta, detail, onDismiss, dismissLabel, onCancel, ...props }, + ref +) { + const trailingAction = onCancel ?? onDismiss + const trailingLabel = onCancel ? 'Cancel' : (dismissLabel ?? 'Dismiss') + return ( +
+ +
+
+ + {title} + + {meta != null && ( + {meta} + )} +
+ {detail != null && ( + + {detail} + + )} +
+ {trailingAction && ( + + )} +
+ ) +}) +ProgressItem.displayName = 'ProgressItem' + +export { ProgressItem, progressItemVariants } diff --git a/apps/sim/hooks/queries/tables.ts b/apps/sim/hooks/queries/tables.ts index 13ab83fda0..6d20d1d1e9 100644 --- a/apps/sim/hooks/queries/tables.ts +++ b/apps/sim/hooks/queries/tables.ts @@ -29,6 +29,7 @@ import { batchUpdateTableRowsContract, type CreateTableBodyInput, type CreateTableColumnBodyInput, + cancelTableImportContract, cancelTableRunsContract, createTableContract, createTableRowContract, @@ -39,6 +40,8 @@ import { deleteWorkflowGroupContract, getTableContract, type InsertTableRowBodyInput, + importIntoTableAsyncContract, + importTableAsyncContract, listActiveDispatchesContract, listTableRowsContract, listTablesContract, @@ -78,6 +81,7 @@ import { isExecInFlight, optimisticallyScheduleNewlyEligibleGroups, } from '@/lib/table/deps' +import { runUploadStrategy } from '@/lib/uploads/client/direct-upload' const logger = createLogger('TableQueries') @@ -178,7 +182,15 @@ function invalidateTableSchema(queryClient: ReturnType, t /** * Fetch all tables for a workspace. */ -export function useTablesList(workspaceId?: string, scope: TableQueryScope = 'active') { +export function useTablesList( + workspaceId?: string, + scope: TableQueryScope = 'active', + options?: { + /** Poll cadence, or a predicate over the current list that returns a cadence (or `false`). */ + refetchInterval?: number | false | ((tables: TableDefinition[] | undefined) => number | false) + } +) { + const refetchInterval = options?.refetchInterval return useQuery({ queryKey: tableKeys.list(workspaceId, scope), queryFn: async ({ signal }) => { @@ -193,6 +205,10 @@ export function useTablesList(workspaceId?: string, scope: TableQueryScope = 'ac enabled: Boolean(workspaceId), staleTime: 30 * 1000, placeholderData: keepPreviousData, + refetchInterval: + typeof refetchInterval === 'function' + ? (query) => refetchInterval(query.state.data) + : (refetchInterval ?? false), }) } @@ -1087,9 +1103,11 @@ export function useUploadCsvToTable() { return useMutation({ mutationFn: async ({ workspaceId, file }: UploadCsvParams) => { + // Text fields must precede the file part: the server parses the body as a + // stream and needs workspaceId before it reaches the (large) file. const formData = new FormData() - formData.append('file', file) formData.append('workspaceId', workspaceId) + formData.append('file', file) // boundary-raw-fetch: multipart/form-data CSV upload, requestJson only supports JSON bodies const response = await fetch('/api/table/import-csv', { @@ -1114,8 +1132,102 @@ export function useUploadCsvToTable() { }) } +interface ImportCsvAsyncParams { + workspaceId: string + file: File + onProgress?: (percent: number) => void +} + +/** + * Uploads a CSV/TSV straight to workspace storage (bypassing the server's request-body + * cap) and returns its storage key. Shared by the async-import kickoff hooks. + */ +async function uploadCsvToWorkspaceStorage( + file: File, + workspaceId: string, + onProgress?: (percent: number) => void +): Promise { + const upload = await runUploadStrategy({ + file, + workspaceId, + context: 'workspace', + presignedEndpoint: `/api/workspaces/${workspaceId}/files/presigned`, + onProgress: onProgress ? (event) => onProgress(event.percent) : undefined, + }) + return upload.key +} + +/** + * Uploads a large CSV/TSV straight to storage, then kicks off a background import into a + * new table. Resolves with `{ tableId, importId }` immediately — load progress and the + * terminal state arrive over the table-events SSE stream (see `useTableEventStream`). + */ +export function useImportCsvAsync() { + const queryClient = useQueryClient() + return useMutation({ + mutationFn: async ({ workspaceId, file, onProgress }: ImportCsvAsyncParams) => { + const fileKey = await uploadCsvToWorkspaceStorage(file, workspaceId, onProgress) + const response = await requestJson(importTableAsyncContract, { + body: { workspaceId, fileKey, fileName: file.name }, + }) + return response.data + }, + onError: (error) => { + logger.error('Failed to start async CSV import:', error) + toast.error(error.message, { duration: 5000 }) + }, + onSettled: () => { + queryClient.invalidateQueries({ queryKey: tableKeys.lists() }) + }, + }) +} + export type CsvImportMode = 'append' | 'replace' +interface ImportCsvIntoTableAsyncParams { + workspaceId: string + tableId: string + file: File + mode: CsvImportMode + mapping?: CsvHeaderMapping + createColumns?: string[] + onProgress?: (percent: number) => void +} + +/** + * Async append/replace import into an existing table for large files: uploads straight to + * storage (bypassing the server's request-body cap), then kicks off the background worker. + * Resolves immediately; progress + completion arrive over the table-events SSE stream. + */ +export function useImportCsvIntoTableAsync() { + const queryClient = useQueryClient() + return useMutation({ + mutationFn: async ({ + workspaceId, + tableId, + file, + mode, + mapping, + createColumns, + onProgress, + }: ImportCsvIntoTableAsyncParams) => { + const fileKey = await uploadCsvToWorkspaceStorage(file, workspaceId, onProgress) + const response = await requestJson(importIntoTableAsyncContract, { + params: { tableId }, + body: { workspaceId, fileKey, fileName: file.name, mode, mapping, createColumns }, + }) + return response.data + }, + onError: (error) => { + logger.error('Failed to start async CSV import:', error) + toast.error(error.message, { duration: 5000 }) + }, + onSettled: (_data, _error, variables) => { + invalidateRowCount(queryClient, variables.tableId) + }, + }) +} + interface ImportCsvIntoTableParams { workspaceId: string tableId: string @@ -1157,8 +1269,9 @@ export function useImportCsvIntoTable() { mapping, createColumns, }: ImportCsvIntoTableParams): Promise => { + // Text fields must precede the file part: the server parses the body as a + // stream and needs these fields before it reaches the (large) file. const formData = new FormData() - formData.append('file', file) formData.append('workspaceId', workspaceId) formData.append('mode', mode) if (mapping) { @@ -1167,6 +1280,7 @@ export function useImportCsvIntoTable() { if (createColumns && createColumns.length > 0) { formData.append('createColumns', JSON.stringify(createColumns)) } + formData.append('file', file) // boundary-raw-fetch: multipart/form-data CSV upload, requestJson only supports JSON bodies const response = await fetch(`/api/table/${tableId}/import`, { @@ -1195,6 +1309,21 @@ export function useImportCsvIntoTable() { * Downloads the full contents of a table to the user's device by streaming * `/api/table/[tableId]/export`. Defaults to CSV; pass `'json'` for JSON. */ +/** + * Cancels an in-flight async import. Plain function (not a hook) because the import dropdown lists + * multiple tables and cancels a chosen one by id rather than binding to a single table. + */ +export async function cancelTableImport( + workspaceId: string, + tableId: string, + importId: string +): Promise { + await requestJson(cancelTableImportContract, { + params: { tableId }, + body: { workspaceId, importId }, + }) +} + export async function downloadTableExport( tableId: string, fileName: string, diff --git a/apps/sim/lib/api/contracts/tables.ts b/apps/sim/lib/api/contracts/tables.ts index f56c22a122..dcd78e1c47 100644 --- a/apps/sim/lib/api/contracts/tables.ts +++ b/apps/sim/lib/api/contracts/tables.ts @@ -348,6 +348,34 @@ export const createTableContract = defineRouteContract({ }, }) +/** + * Kickoff body for an asynchronous large-CSV import into a NEW table. The file is + * already uploaded to storage (the client sends its `fileKey`); the route creates an + * `importing` table and runs the load in the background. + */ +export const importTableAsyncBodySchema = z.object({ + workspaceId: z.string().min(1, 'Workspace ID is required'), + fileKey: z.string().min(1, 'fileKey is required'), + fileName: z.string().min(1, 'fileName is required'), +}) + +export type ImportTableAsyncBody = z.input + +export const importTableAsyncContract = defineRouteContract({ + method: 'POST', + path: '/api/table/import-async', + body: importTableAsyncBodySchema, + response: { + mode: 'json', + schema: successResponseSchema( + z.object({ + tableId: z.string(), + importId: z.string(), + }) + ), + }, +}) + export const getTableContract = defineRouteContract({ method: 'GET', path: '/api/table/[tableId]', @@ -565,6 +593,38 @@ export const csvExtensionSchema = z.enum(['csv', 'tsv'], { error: 'Only CSV and TSV files are supported', }) +/** + * Kickoff body for an asynchronous CSV import into an EXISTING table (append/replace). + * The file is already uploaded to storage; `mapping`/`createColumns` are the client's + * resolved column mapping (the dialog computes them from its preview). + */ +export const importIntoTableAsyncBodySchema = z.object({ + workspaceId: z.string().min(1, 'Workspace ID is required'), + fileKey: z.string().min(1, 'fileKey is required'), + fileName: z.string().min(1, 'fileName is required'), + mode: csvImportModeSchema, + mapping: z.record(z.string(), z.string().nullable()).optional(), + createColumns: z.array(z.string()).optional(), +}) + +export type ImportIntoTableAsyncBody = z.input + +export const importIntoTableAsyncContract = defineRouteContract({ + method: 'POST', + path: '/api/table/[tableId]/import-async', + params: tableIdParamsSchema, + body: importIntoTableAsyncBodySchema, + response: { + mode: 'json', + schema: successResponseSchema( + z.object({ + tableId: z.string(), + importId: z.string(), + }) + ), + }, +}) + /** * `createColumns` form field — a JSON-encoded array of CSV header names that * the import should auto-create as new columns on the target table. @@ -891,6 +951,24 @@ export const cancelTableRunsContract = defineRouteContract({ }, }) +export const cancelTableImportBodySchema = z.object({ + workspaceId: z.string().min(1, 'Workspace ID is required'), + importId: z.string().min(1, 'Import ID is required'), +}) + +/** Cancel an in-flight async CSV import. The worker stops; committed rows are left in place. */ +export const cancelTableImportContract = defineRouteContract({ + method: 'POST', + path: '/api/table/[tableId]/import/cancel', + params: tableIdParamsSchema, + body: cancelTableImportBodySchema, + response: { + mode: 'json', + schema: successResponseSchema(z.object({ canceled: z.boolean() })), + }, +}) +export type CancelTableImportBody = z.input + /** * Run modes for `POST /api/table/[tableId]/columns/run`: * - `all` — every dep-satisfied row not already running/pending diff --git a/apps/sim/lib/copilot/tools/handlers/materialize-file.test.ts b/apps/sim/lib/copilot/tools/handlers/materialize-file.test.ts new file mode 100644 index 0000000000..84184b0e35 --- /dev/null +++ b/apps/sim/lib/copilot/tools/handlers/materialize-file.test.ts @@ -0,0 +1,177 @@ +/** + * @vitest-environment node + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { + mockFindUpload, + mockFetchBuffer, + mockParseFileRows, + mockInferSchema, + mockCoerceRows, + mockCreateTable, + mockBatchInsertRows, + mockDeleteTable, + mockGetLimits, +} = vi.hoisted(() => ({ + mockFindUpload: vi.fn(), + mockFetchBuffer: vi.fn(), + mockParseFileRows: vi.fn(), + mockInferSchema: vi.fn(), + mockCoerceRows: vi.fn(), + mockCreateTable: vi.fn(), + mockBatchInsertRows: vi.fn(), + mockDeleteTable: vi.fn(), + mockGetLimits: vi.fn(), +})) + +vi.mock('@/lib/copilot/tools/handlers/upload-file-reader', () => ({ + findMothershipUploadRowByChatAndName: mockFindUpload, +})) + +vi.mock('@/lib/uploads', () => ({ + getServePathPrefix: () => '/api/files/serve/', +})) + +vi.mock('@/lib/uploads/contexts/workspace/workspace-file-manager', () => ({ + fetchWorkspaceFileBuffer: mockFetchBuffer, +})) + +vi.mock('@/lib/table', () => ({ + CSV_MAX_BATCH_SIZE: 1000, + TABLE_LIMITS: { MAX_TABLE_NAME_LENGTH: 100 }, + parseFileRows: mockParseFileRows, + inferSchemaFromCsv: mockInferSchema, + coerceRowsForTable: mockCoerceRows, + createTable: mockCreateTable, + batchInsertRows: mockBatchInsertRows, + deleteTable: mockDeleteTable, + getWorkspaceTableLimits: mockGetLimits, + sanitizeName: (raw: string) => raw.replace(/[^a-zA-Z0-9_]/g, '_'), +})) + +vi.mock('@/lib/workflows/operations/import-export', () => ({ parseWorkflowJson: vi.fn() })) +vi.mock('@/lib/workflows/persistence/utils', () => ({ saveWorkflowToNormalizedTables: vi.fn() })) +vi.mock('@/lib/workflows/utils', () => ({ deduplicateWorkflowName: vi.fn() })) +vi.mock('@/app/api/v1/admin/types', () => ({ extractWorkflowMetadata: vi.fn() })) + +import type { ExecutionContext } from '@/lib/copilot/request/types' +import { executeMaterializeFile } from '@/lib/copilot/tools/handlers/materialize-file' + +const context = { + chatId: 'chat-1', + workspaceId: 'ws-1', + userId: 'user-1', + workflowId: 'wf-1', +} as ExecutionContext + +const uploadRow = { + id: 'file-1', + workspaceId: 'ws-1', + displayName: 'data.csv', + originalName: 'data.csv', + key: 'uploads/data.csv', + size: 123, + contentType: 'text/csv', + userId: 'user-1', + deletedAt: null, + uploadedAt: new Date(), + updatedAt: new Date(), +} + +describe('executeMaterializeFile - table operation', () => { + beforeEach(() => { + vi.clearAllMocks() + mockFindUpload.mockResolvedValue(uploadRow) + mockFetchBuffer.mockResolvedValue(Buffer.from('name\nAlice')) + mockParseFileRows.mockResolvedValue({ headers: ['name'], rows: [{ name: 'Alice' }] }) + mockInferSchema.mockReturnValue({ + columns: [{ name: 'name', type: 'string' }], + headerToColumn: new Map([['name', 'name']]), + }) + mockCoerceRows.mockReturnValue([{ name: 'Alice' }]) + mockGetLimits.mockResolvedValue({ maxRowsPerTable: 1_000_000, maxTables: 50 }) + mockCreateTable.mockResolvedValue({ id: 'tbl_abc', name: 'data', schema: { columns: [] } }) + mockBatchInsertRows.mockResolvedValue([{ id: 'row-1' }]) + mockDeleteTable.mockResolvedValue(undefined) + }) + + it('creates a table and returns a table resource', async () => { + const result = await executeMaterializeFile( + { fileNames: ['data.csv'], operation: 'table' }, + context + ) + + expect(result.success).toBe(true) + expect(mockCreateTable).toHaveBeenCalledTimes(1) + expect(mockCreateTable).toHaveBeenCalledWith( + expect.objectContaining({ + name: 'data', + workspaceId: 'ws-1', + userId: 'user-1', + maxRows: 1_000_000, + maxTables: 50, + }), + expect.any(String) + ) + expect(result.resources).toEqual([{ type: 'table', id: 'tbl_abc', title: 'data' }]) + expect((result.output as { succeeded: string[] }).succeeded).toEqual(['data.csv']) + }) + + it('honors an explicit tableName', async () => { + await executeMaterializeFile( + { fileNames: ['data.csv'], operation: 'table', tableName: 'My Customers' }, + context + ) + expect(mockCreateTable).toHaveBeenCalledWith( + expect.objectContaining({ name: 'My_Customers' }), + expect.any(String) + ) + }) + + it('deletes the table and fails when row insertion throws', async () => { + mockBatchInsertRows.mockRejectedValueOnce(new Error('insert exploded')) + + const result = await executeMaterializeFile( + { fileNames: ['data.csv'], operation: 'table' }, + context + ) + + expect(result.success).toBe(false) + expect(mockDeleteTable).toHaveBeenCalledWith('tbl_abc', expect.any(String)) + expect((result.output as { failed: Array<{ error: string }> }).failed[0].error).toContain( + 'insert exploded' + ) + }) + + it('fails fast (no table created) when the upload is missing', async () => { + mockFindUpload.mockResolvedValue(null) + + const result = await executeMaterializeFile( + { fileNames: ['missing.csv'], operation: 'table' }, + context + ) + + expect(result.success).toBe(false) + expect(mockCreateTable).not.toHaveBeenCalled() + expect((result.output as { failed: Array<{ error: string }> }).failed[0].error).toContain( + 'Upload not found' + ) + }) +}) + +describe('executeMaterializeFile - unsupported operation', () => { + beforeEach(() => vi.clearAllMocks()) + + it('rejects an unimplemented operation instead of silently saving', async () => { + const result = await executeMaterializeFile( + { fileNames: ['data.csv'], operation: 'knowledge_base' }, + context + ) + + expect(result.success).toBe(false) + expect(result.error).toContain('not implemented') + expect(mockFindUpload).not.toHaveBeenCalled() + expect(mockCreateTable).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/copilot/tools/handlers/materialize-file.ts b/apps/sim/lib/copilot/tools/handlers/materialize-file.ts index 7aa2b88c72..2d2ff8db0b 100644 --- a/apps/sim/lib/copilot/tools/handlers/materialize-file.ts +++ b/apps/sim/lib/copilot/tools/handlers/materialize-file.ts @@ -7,6 +7,19 @@ import { generateId } from '@sim/utils/id' import { and, eq, isNull } from 'drizzle-orm' import type { ExecutionContext, ToolCallResult } from '@/lib/copilot/request/types' import { findMothershipUploadRowByChatAndName } from '@/lib/copilot/tools/handlers/upload-file-reader' +import { + batchInsertRows, + CSV_MAX_BATCH_SIZE, + coerceRowsForTable, + createTable, + deleteTable, + getWorkspaceTableLimits, + inferSchemaFromCsv, + parseFileRows, + sanitizeName, + TABLE_LIMITS, + type TableSchema, +} from '@/lib/table' import { getServePathPrefix } from '@/lib/uploads' import { fetchWorkspaceFileBuffer } from '@/lib/uploads/contexts/workspace/workspace-file-manager' import { parseWorkflowJson } from '@/lib/workflows/operations/import-export' @@ -184,6 +197,88 @@ async function executeImport( } } +async function executeTable( + fileName: string, + chatId: string, + workspaceId: string, + userId: string, + requestedTableName?: string +): Promise { + const row = await findMothershipUploadRowByChatAndName(chatId, fileName) + if (!row) { + return { + success: false, + error: `Upload not found: "${fileName}". Use glob("uploads/*") to list available uploads.`, + } + } + + const fileRecord = toFileRecord(row) + const buffer = await fetchWorkspaceFileBuffer(fileRecord) + const { headers, rows } = await parseFileRows(buffer, fileRecord.name, fileRecord.type) + if (rows.length === 0) { + return { success: false, error: `"${fileName}" contains no data rows.` } + } + + const { columns, headerToColumn } = inferSchemaFromCsv(headers, rows) + const baseName = requestedTableName?.trim() || fileName.replace(/\.[^.]+$/, '') + const tableName = sanitizeName(baseName, 'imported_table').slice( + 0, + TABLE_LIMITS.MAX_TABLE_NAME_LENGTH + ) + const schema: TableSchema = { columns } + const planLimits = await getWorkspaceTableLimits(workspaceId) + const requestId = generateId().slice(0, 8) + + const table = await createTable( + { + name: tableName, + description: `Imported from ${fileName}`, + schema, + workspaceId, + userId, + maxRows: planLimits.maxRowsPerTable, + maxTables: planLimits.maxTables, + }, + requestId + ) + + try { + const coerced = coerceRowsForTable(rows, schema, headerToColumn) + let inserted = 0 + for (let i = 0; i < coerced.length; i += CSV_MAX_BATCH_SIZE) { + const batch = coerced.slice(i, i + CSV_MAX_BATCH_SIZE) + const result = await batchInsertRows( + { tableId: table.id, rows: batch, workspaceId, userId }, + table, + generateId().slice(0, 8) + ) + inserted += result.length + } + + logger.info('Created table from upload', { + fileName, + tableId: table.id, + columns: columns.length, + rows: inserted, + chatId, + }) + + return { + success: true, + output: { + message: `File "${fileName}" imported as table "${table.name}" with ${columns.length} columns and ${inserted} rows.`, + tableId: table.id, + tableName: table.name, + rowCount: inserted, + }, + resources: [{ type: 'table', id: table.id, title: table.name }], + } + } catch (insertError) { + await deleteTable(table.id, requestId).catch(() => {}) + throw insertError + } +} + export async function executeMaterializeFile( params: Record, context: ExecutionContext @@ -205,17 +300,43 @@ export async function executeMaterializeFile( } const operation = (params.operation as string | undefined) || 'save' + + const supportedOperations = new Set(['save', 'import', 'table']) + if (!supportedOperations.has(operation)) { + return { + success: false, + error: `materialize_file operation "${operation}" is not implemented. Supported operations: ${[...supportedOperations].join(', ')}.`, + } + } + + const requestedTableName = params.tableName as string | undefined const succeeded: string[] = [] const failed: Array<{ fileName: string; error: string }> = [] + const resources: NonNullable = [] for (const fileName of fileNames) { try { + let result: ToolCallResult if (operation === 'import') { - await executeImport(fileName, context.chatId, context.workspaceId, context.userId) + result = await executeImport(fileName, context.chatId, context.workspaceId, context.userId) + } else if (operation === 'table') { + result = await executeTable( + fileName, + context.chatId, + context.workspaceId, + context.userId, + requestedTableName + ) } else { - await executeSave(fileName, context.chatId) + result = await executeSave(fileName, context.chatId) + } + + if (result.success) { + succeeded.push(fileName) + if (result.resources) resources.push(...result.resources) + } else { + failed.push({ fileName, error: result.error ?? 'Failed to materialize file' }) } - succeeded.push(fileName) } catch (err) { logger.error('materialize_file failed', { fileName, @@ -237,5 +358,6 @@ export async function executeMaterializeFile( failed.length > 0 ? `Failed to materialize: ${failed.map((f) => f.fileName).join(', ')}` : undefined, + resources: resources.length > 0 ? resources : undefined, } } diff --git a/apps/sim/lib/copilot/tools/server/table/user-table.ts b/apps/sim/lib/copilot/tools/server/table/user-table.ts index d20d172711..a6e77164dd 100644 --- a/apps/sim/lib/copilot/tools/server/table/user-table.ts +++ b/apps/sim/lib/copilot/tools/server/table/user-table.ts @@ -16,8 +16,7 @@ import { coerceRowsForTable, getWorkspaceTableLimits, inferSchemaFromCsv, - parseCsvBuffer, - sanitizeName, + parseFileRows, validateMapping, } from '@/lib/table' import { columnTypeForLeaf, deriveOutputColumnName } from '@/lib/table/column-naming' @@ -98,39 +97,6 @@ async function resolveWorkspaceFile( return { buffer, name: record.name, type: record.type } } -/** - * Sanitizes raw JSON headers/rows so they conform to the same rules as CSV - * imports (so `inferSchemaFromCsv` and friends can be reused). - */ -function sanitizeJsonHeaders( - headers: string[], - rows: Record[] -): { headers: string[]; rows: Record[] } { - const renamed = new Map() - const seen = new Set() - - for (const raw of headers) { - let safe = sanitizeName(raw) - while (seen.has(safe)) safe = `${safe}_` - seen.add(safe) - renamed.set(raw, safe) - } - - const noChange = headers.every((h) => renamed.get(h) === h) - if (noChange) return { headers, rows } - - return { - headers: headers.map((h) => renamed.get(h)!), - rows: rows.map((row) => { - const out: Record = {} - for (const [raw, safe] of renamed) { - if (raw in row) out[safe] = row[raw] - } - return out - }), - } -} - /** * Loads the live workflow state and flattens it into pickable outputs. Used * to validate `(blockId, path)` pairs the AI passes to add/update_workflow_group @@ -173,42 +139,6 @@ function validateOutputsAgainstWorkflow( return `Invalid output(s) for workflow ${workflowId}:\n${invalidList}\n\nValid options${flattened.length > 12 ? ' (first 12)' : ''}:\n${sample}\n\nCall list_workflow_outputs with workflowId="${workflowId}" to see all valid (blockId, path) picks.` } -async function parseJsonRows( - buffer: Buffer -): Promise<{ headers: string[]; rows: Record[] }> { - const parsed = JSON.parse(buffer.toString('utf-8')) - if (!Array.isArray(parsed)) { - throw new Error('JSON file must contain an array of objects') - } - if (parsed.length === 0) { - throw new Error('JSON file contains an empty array') - } - const headerSet = new Set() - for (const row of parsed) { - if (typeof row !== 'object' || row === null || Array.isArray(row)) { - throw new Error('Each element in the JSON array must be a plain object') - } - for (const key of Object.keys(row)) headerSet.add(key) - } - return sanitizeJsonHeaders([...headerSet], parsed) -} - -async function parseFileRows( - buffer: Buffer, - fileName: string, - contentType: string -): Promise<{ headers: string[]; rows: Record[] }> { - const ext = fileName.split('.').pop()?.toLowerCase() - if (ext === 'json' || contentType === 'application/json') { - return parseJsonRows(buffer) - } - if (ext === 'csv' || ext === 'tsv' || contentType === 'text/csv') { - const delimiter = ext === 'tsv' ? '\t' : ',' - return parseCsvBuffer(buffer, delimiter) - } - throw new Error(`Unsupported file format: "${ext}". Supported: csv, tsv, json`) -} - async function batchInsertAll( tableId: string, rows: RowData[], diff --git a/apps/sim/lib/core/utils/multipart.test.ts b/apps/sim/lib/core/utils/multipart.test.ts new file mode 100644 index 0000000000..81c0961961 --- /dev/null +++ b/apps/sim/lib/core/utils/multipart.test.ts @@ -0,0 +1,203 @@ +/** + * @vitest-environment node + */ +import type { Readable } from 'node:stream' +import { describe, expect, it } from 'vitest' +import { isMultipartError, type MultipartError, readMultipart } from '@/lib/core/utils/multipart' + +type Part = + | { name: string; value: string } + | { name: string; filename: string; value: string; contentType?: string } + +const BOUNDARY = '----testboundary1234' + +function buildBody(parts: Part[], boundary = BOUNDARY): Buffer { + const segments: Buffer[] = [] + for (const part of parts) { + let header = `--${boundary}\r\nContent-Disposition: form-data; name="${part.name}"` + if ('filename' in part) { + header += `; filename="${part.filename}"\r\nContent-Type: ${part.contentType ?? 'text/csv'}` + } + header += '\r\n\r\n' + segments.push(Buffer.from(header, 'utf8'), Buffer.from(part.value, 'utf8'), Buffer.from('\r\n')) + } + segments.push(Buffer.from(`--${boundary}--\r\n`, 'utf8')) + return Buffer.concat(segments) +} + +function toWebStream(body: Buffer, chunkSize?: number): ReadableStream { + return new ReadableStream({ + start(controller) { + if (chunkSize) { + for (let i = 0; i < body.length; i += chunkSize) { + controller.enqueue(new Uint8Array(body.subarray(i, i + chunkSize))) + } + } else { + controller.enqueue(new Uint8Array(body)) + } + controller.close() + }, + }) +} + +function makeRequest( + parts: Part[], + opts?: { chunkSize?: number; contentType?: string; boundary?: string } +) { + const boundary = opts?.boundary ?? BOUNDARY + return { + headers: new Headers({ + 'content-type': opts?.contentType ?? `multipart/form-data; boundary=${boundary}`, + }), + body: toWebStream(buildBody(parts, boundary), opts?.chunkSize), + } +} + +async function readStream(stream: Readable): Promise { + const chunks: Buffer[] = [] + for await (const chunk of stream) chunks.push(Buffer.from(chunk)) + return Buffer.concat(chunks).toString('utf8') +} + +function expectCode(error: unknown, code: MultipartError['code']) { + expect(isMultipartError(error)).toBe(true) + expect((error as MultipartError).code).toBe(code) +} + +describe('readMultipart', () => { + it('parses text fields (before the file) and exposes the file stream', async () => { + const csv = 'name,age\nAlice,30\n' + const request = makeRequest([ + { name: 'workspaceId', value: 'ws-1' }, + { name: 'file', filename: 'data.csv', value: csv }, + ]) + + const { fields, file } = await readMultipart(request, { + maxFileBytes: 1024, + requiredFieldsBeforeFile: ['workspaceId'], + }) + + expect(fields.workspaceId).toBe('ws-1') + expect(file?.filename).toBe('data.csv') + expect(file?.fieldName).toBe('file') + expect(await readStream(file!.stream)).toBe(csv) + }) + + it('handles a body delivered in tiny chunks (split mid-boundary)', async () => { + const csv = 'name,age\nAlice,30\nBob,40\n' + const request = makeRequest( + [ + { name: 'workspaceId', value: 'ws-1' }, + { name: 'file', filename: 'data.csv', value: csv }, + ], + { chunkSize: 3 } + ) + + const { file } = await readMultipart(request, { maxFileBytes: 1024 }) + expect(await readStream(file!.stream)).toBe(csv) + }) + + it('rejects FIELD_AFTER_FILE when a required field comes after the file', async () => { + const request = makeRequest([ + { name: 'file', filename: 'data.csv', value: 'name\nAlice\n' }, + { name: 'workspaceId', value: 'ws-1' }, + ]) + + await readMultipart(request, { + maxFileBytes: 1024, + requiredFieldsBeforeFile: ['workspaceId'], + }).then( + () => { + throw new Error('expected rejection') + }, + (err) => expectCode(err, 'FIELD_AFTER_FILE') + ) + }) + + it('rejects NO_FILE when the body has no file part', async () => { + const request = makeRequest([{ name: 'workspaceId', value: 'ws-1' }]) + await readMultipart(request, { maxFileBytes: 1024 }).then( + () => { + throw new Error('expected rejection') + }, + (err) => expectCode(err, 'NO_FILE') + ) + }) + + it('rejects NOT_MULTIPART for a non-multipart content type', async () => { + const request = { + headers: new Headers({ 'content-type': 'application/json' }), + body: toWebStream(Buffer.from('{}')), + } + await readMultipart(request, { maxFileBytes: 1024 }).then( + () => { + throw new Error('expected rejection') + }, + (err) => expectCode(err, 'NOT_MULTIPART') + ) + }) + + it('errors the file stream with FILE_TOO_LARGE when the cap is exceeded', async () => { + const request = makeRequest([ + { name: 'workspaceId', value: 'ws-1' }, + { name: 'file', filename: 'big.csv', value: 'x'.repeat(500) }, + ]) + + const { file } = await readMultipart(request, { maxFileBytes: 50 }) + await readStream(file!.stream).then( + () => { + throw new Error('expected stream error') + }, + (err) => expectCode(err, 'FILE_TOO_LARGE') + ) + }) + + it('rejects when the signal is already aborted', async () => { + const controller = new AbortController() + controller.abort() + const request = makeRequest([ + { name: 'workspaceId', value: 'ws-1' }, + { name: 'file', filename: 'data.csv', value: 'name\nAlice\n' }, + ]) + + await expect( + readMultipart(request, { maxFileBytes: 1024, signal: controller.signal }) + ).rejects.toBeTruthy() + }) + + it('destroys the file stream when the signal aborts mid-upload (after resolve)', async () => { + const controller = new AbortController() + // A body that delivers the file-part header but never closes, so the file stream stays open + // after readMultipart resolves — mimicking a client still uploading. + let enqueue!: (b: Buffer) => void + const body = new ReadableStream({ + start(c) { + enqueue = (b) => c.enqueue(new Uint8Array(b)) + }, + }) + const head = Buffer.concat([ + Buffer.from( + `--${BOUNDARY}\r\nContent-Disposition: form-data; name="workspaceId"\r\n\r\nws-1\r\n` + ), + Buffer.from( + `--${BOUNDARY}\r\nContent-Disposition: form-data; name="file"; filename="data.csv"\r\nContent-Type: text/csv\r\n\r\n` + ), + Buffer.from('name,age\n'), + ]) + const request = { + headers: new Headers({ 'content-type': `multipart/form-data; boundary=${BOUNDARY}` }), + body, + } + enqueue(head) + + const parsed = await readMultipart(request, { + maxFileBytes: 1024, + requiredFieldsBeforeFile: ['workspaceId'], + signal: controller.signal, + }) + expect(parsed.file).toBeTruthy() + + controller.abort() + await expect(readStream(parsed.file!.stream)).rejects.toBeTruthy() + }) +}) diff --git a/apps/sim/lib/core/utils/multipart.ts b/apps/sim/lib/core/utils/multipart.ts new file mode 100644 index 0000000000..f23969def5 --- /dev/null +++ b/apps/sim/lib/core/utils/multipart.ts @@ -0,0 +1,256 @@ +import { Readable } from 'node:stream' +import type { ReadableStream as NodeReadableStream } from 'node:stream/web' +import busboy from 'busboy' + +/** + * Streaming multipart/form-data reader built on `busboy`. + * + * Unlike `request.formData()` (undici), this never buffers the whole request + * body in memory and does not depend on a correct `content-length`/boundary — + * it parses the request as it streams off the socket. The single file part is + * surfaced as an un-drained Node {@link Readable} so the caller can run auth / + * create-table work BEFORE consuming the (potentially huge) file bytes. + * + * @see readMultipart + */ + +/** Error codes surfaced by {@link readMultipart} and the returned file stream. */ +export type MultipartErrorCode = + | 'NOT_MULTIPART' + | 'NO_BODY' + | 'FILE_TOO_LARGE' + | 'FIELD_AFTER_FILE' + | 'NO_FILE' + | 'PARSE_ERROR' + +/** + * Error thrown by {@link readMultipart} (for pre-file failures) or emitted on + * the returned file stream (for failures during consumption, e.g. + * `FILE_TOO_LARGE`). Callers map `code` to an HTTP status. + */ +export class MultipartError extends Error { + readonly code: MultipartErrorCode + + constructor(code: MultipartErrorCode, message: string) { + super(message) + this.name = 'MultipartError' + this.code = code + } +} + +export function isMultipartError(error: unknown): error is MultipartError { + return error instanceof MultipartError +} + +export interface MultipartFilePart { + /** The multipart field name that carried the file (expected: `file`). */ + fieldName: string + filename: string + mimeType: string + /** + * The file bytes. The caller MUST fully consume or `destroy()` this stream + * (use a `finally`) or the request will hang. On overflow of `maxFileBytes` + * the stream is destroyed with a {@link MultipartError} (`FILE_TOO_LARGE`). + */ + stream: Readable +} + +export interface ParsedMultipart { + /** Text fields that arrived before the file part, keyed by field name. */ + fields: Record + /** The single file part, or `null` if the body had no file part. */ + file: MultipartFilePart | null +} + +export interface ReadMultipartOptions { + /** Per-file byte cap. Overflow destroys the file stream with `FILE_TOO_LARGE`. */ + maxFileBytes: number + /** + * Field names that must arrive before the file part. If the file part is + * seen while any are still missing, the parse rejects with `FIELD_AFTER_FILE`. + */ + requiredFieldsBeforeFile?: string[] + /** Field name expected to carry the file. Defaults to `file`. */ + fileFieldName?: string + /** Abort signal — cancels parsing and destroys the underlying stream. */ + signal?: AbortSignal +} + +interface MultipartRequest { + headers: Headers + body: ReadableStream | null +} + +/** + * Parse a `multipart/form-data` request as a stream. Resolves as soon as the + * file-part header is seen (text fields collected up to that point are in + * `fields`); the file bytes are NOT yet consumed — the caller drives + * `result.file.stream`. + * + * Pre-file failures reject the returned promise; failures that happen while the + * file streams (size limit, mid-body parse errors, abort) are surfaced as an + * error on `result.file.stream`. + */ +export function readMultipart( + request: MultipartRequest, + options: ReadMultipartOptions +): Promise { + const { maxFileBytes, requiredFieldsBeforeFile = [], fileFieldName = 'file', signal } = options + + return new Promise((resolve, reject) => { + const contentType = request.headers.get('content-type') + if (!contentType || !contentType.toLowerCase().includes('multipart/form-data')) { + reject(new MultipartError('NOT_MULTIPART', 'Expected multipart/form-data request')) + return + } + if (!request.body) { + reject(new MultipartError('NO_BODY', 'Request has no body')) + return + } + + let bb: busboy.Busboy + try { + bb = busboy({ + headers: { 'content-type': contentType }, + limits: { fileSize: maxFileBytes, files: 1 }, + }) + } catch (err) { + reject( + new MultipartError( + 'NOT_MULTIPART', + err instanceof Error ? err.message : 'Invalid multipart request' + ) + ) + return + } + + // double-cast-allowed: the web ReadableStream on request.body isn't structurally assignable to the Node type Readable.fromWeb expects + const nodeStream = Readable.fromWeb(request.body as unknown as NodeReadableStream) + const fields: Record = {} + let settled = false + let fileSeen = false + + const onAbort = () => { + const reason = signal?.reason instanceof Error ? signal.reason : new Error('Aborted') + nodeStream.destroy(reason) + bb.destroy() + if (!settled) { + settled = true + reject(reason) + } + } + + const cleanup = () => { + signal?.removeEventListener('abort', onAbort) + } + + const settle = (fn: () => void) => { + if (settled) return + settled = true + cleanup() + fn() + } + + if (signal?.aborted) { + // `destroy()` with no reason emits 'close', not an unhandled 'error'. + nodeStream.destroy() + settled = true + reject(signal.reason instanceof Error ? signal.reason : new Error('Aborted')) + return + } + signal?.addEventListener('abort', onAbort, { once: true }) + + bb.on('field', (name, value) => { + fields[name] = value + }) + + bb.on('file', (name, stream, info) => { + if (settled || fileSeen) { + stream.resume() + return + } + fileSeen = true + + if (name !== fileFieldName) { + stream.resume() + nodeStream.destroy() + settle(() => + reject( + new MultipartError('NO_FILE', `Expected file field "${fileFieldName}", got "${name}"`) + ) + ) + return + } + + const missing = requiredFieldsBeforeFile.filter((field) => !(field in fields)) + if (missing.length > 0) { + stream.resume() + nodeStream.destroy() + settle(() => + reject( + new MultipartError( + 'FIELD_AFTER_FILE', + `Field(s) must precede the file in the request body: ${missing.join(', ')}` + ) + ) + ) + return + } + + stream.once('limit', () => { + stream.destroy( + new MultipartError('FILE_TOO_LARGE', `File exceeds maximum size of ${maxFileBytes} bytes`) + ) + }) + + settle(() => { + // settle() detached the pre-file abort handler. Re-arm one scoped to the file stream so a + // client disconnect mid-upload tears it down — otherwise the caller's consume loop hangs + // until maxDuration. Detach when the stream closes so it can't fire afterward. + if (signal) { + const onStreamAbort = () => { + const reason = signal.reason instanceof Error ? signal.reason : new Error('Aborted') + stream.destroy(reason) + nodeStream.destroy(reason) + bb.destroy() + } + if (signal.aborted) onStreamAbort() + else { + signal.addEventListener('abort', onStreamAbort, { once: true }) + stream.once('close', () => signal.removeEventListener('abort', onStreamAbort)) + } + } + resolve({ + fields, + file: { fieldName: name, filename: info.filename, mimeType: info.mimeType, stream }, + }) + }) + }) + + bb.on('error', (err) => { + const message = err instanceof Error ? err.message : 'Failed to parse multipart body' + settle(() => reject(new MultipartError('PARSE_ERROR', message))) + }) + + bb.on('close', () => { + if (!fileSeen) { + settle(() => reject(new MultipartError('NO_FILE', 'No file part in multipart body'))) + } + }) + + nodeStream.on('error', (err) => { + settle(() => + reject( + err instanceof MultipartError + ? err + : new MultipartError( + 'PARSE_ERROR', + err instanceof Error ? err.message : 'Failed to read request body' + ) + ) + ) + }) + + nodeStream.pipe(bb) + }) +} diff --git a/apps/sim/lib/table/constants.ts b/apps/sim/lib/table/constants.ts index 00597130b7..04084ed821 100644 --- a/apps/sim/lib/table/constants.ts +++ b/apps/sim/lib/table/constants.ts @@ -108,6 +108,13 @@ export const NAME_PATTERN = /^[a-z_][a-z0-9_]*$/i export const USER_TABLE_ROWS_SQL_NAME = 'user_table_rows' +/** + * CSV/TSV uploads at or above this size import in the background (direct-to-storage + * upload + async worker) instead of being POSTed through the server. Kept safely under + * the Next.js proxy request-body cap (10MB) so a synchronous upload is never truncated. + */ +export const CSV_ASYNC_IMPORT_THRESHOLD_BYTES = 8 * 1024 * 1024 + const TABLE_NAME_ADJECTIVES = [ 'Radiant', 'Luminous', diff --git a/apps/sim/lib/table/events.ts b/apps/sim/lib/table/events.ts index dd35e3a799..24156409a1 100644 --- a/apps/sim/lib/table/events.ts +++ b/apps/sim/lib/table/events.ts @@ -113,6 +113,21 @@ export type TableEvent = * skip capped dispatches (see `resolveCellExec`). */ limit?: { type: 'rows'; max: number } } + | { + /** Async large-import progress. The background import worker emits + * `importing` ticks as batches commit, then a terminal `ready`/`failed`. + * The client reveals the (hidden) rows on `ready` and shows a failure + * badge on `failed`. See `apps/sim/lib/table/import-runner.ts`. */ + kind: 'import' + tableId: string + importId: string + status: 'importing' | 'ready' | 'failed' | 'canceled' + /** Rows committed so far (importing) or in total (ready). */ + progress?: number + /** Byte-based completion percent (0–100) — exact and monotonic, for the determinate bar. */ + percent?: number + error?: string + } | { /** A dispatch was stopped because the billed account is over its usage * limit. The client surfaces an upgrade prompt and redirects to billing. diff --git a/apps/sim/lib/table/import-runner.ts b/apps/sim/lib/table/import-runner.ts new file mode 100644 index 0000000000..d654a086cc --- /dev/null +++ b/apps/sim/lib/table/import-runner.ts @@ -0,0 +1,296 @@ +import { type Readable, Transform } from 'node:stream' +import { createLogger } from '@sim/logger' +import { getErrorMessage } from '@sim/utils/errors' +import { generateId } from '@sim/utils/id' +import { + buildAutoMapping, + CSV_MAX_BATCH_SIZE, + CSV_SCHEMA_SAMPLE_SIZE, + type CsvHeaderMapping, + coerceRowsForTable, + createCsvParser, + inferColumnType, + inferSchemaFromCsv, + sanitizeName, + type TableSchema, + validateMapping, +} from '@/lib/table' +import { appendTableEvent } from '@/lib/table/events' +import { + addImportColumns, + bulkInsertImportBatch, + deleteAllTableRows, + getTableById, + markImportFailed, + markImportReady, + nextImportStartPosition, + setTableSchemaForImport, + updateImportProgress, +} from '@/lib/table/service' +import { deleteFile, downloadFileStream, headObject } from '@/lib/uploads/core/storage-service' +import { normalizeColumn } from '@/app/api/table/utils' + +const logger = createLogger('TableImportRunner') + +/** Emit a progress event / DB update at most every this many rows. */ +const PROGRESS_INTERVAL_ROWS = 5000 + +/** + * Thrown when this worker discovers it no longer owns the table's import (the stale-job janitor + * marked its run failed and a newer import took over). The worker stops inserting rather than + * writing into a table a second worker now owns. + */ +class ImportSupersededError extends Error {} + +/** `create` infers a schema for a new table; `append`/`replace` map onto an existing one. */ +export type TableImportMode = 'create' | 'append' | 'replace' + +export interface TableImportPayload { + importId: string + tableId: string + workspaceId: string + userId: string + /** Storage key of the already-uploaded CSV/TSV file. */ + fileKey: string + fileName: string + delimiter: ',' | '\t' + mode: TableImportMode + /** (append/replace) Explicit CSV-header → column mapping; auto-mapped when omitted. */ + mapping?: CsvHeaderMapping + /** (append/replace) CSV headers to auto-create as new columns (types inferred from the sample). */ + createColumns?: string[] +} + +/** + * Background worker for large CSV/TSV imports. Runs detached on the web container + * (see the kickoff routes). Streams the stored file through `createCsvParser`, resolves + * the target schema + header→column mapping from the first sample (inferring a new schema + * for `create`, mapping onto the existing schema for `append`/`replace`), then bulk-inserts + * in committed batches — **no rollback**: committed batches persist even if a later batch + * fails. Progress and the terminal state are surfaced via the table-events SSE stream. + */ +export async function runTableImport(payload: TableImportPayload): Promise { + const { importId, tableId, workspaceId, userId, fileKey, fileName, delimiter, mode } = payload + const requestId = generateId().slice(0, 8) + // Hoisted so `finally` can destroy it on any failure — otherwise the storage HTTP body leaks + // open until it times out. + let source: Readable | undefined + + try { + const loaded = await getTableById(tableId, { includeArchived: true }) + if (!loaded) throw new Error(`Import target table ${tableId} not found`) + const table = loaded + + // Total byte size for the progress estimate — a cheap HEAD, no download. May be null on + // the local dev provider, in which case the bar stays indeterminate (rows still show). + const totalBytes = (await headObject(fileKey, 'workspace'))?.size ?? 0 + + // Stream the file rather than buffering it — a ~1M-row import must never be held in memory. + source = await downloadFileStream({ key: fileKey, context: 'workspace' }) + + // Append must continue after the existing rows; create/replace start empty. Read once up + // front (the import is the table's sole writer) and assign contiguous positions from it. + const basePosition = mode === 'append' ? await nextImportStartPosition(tableId) : 0 + + // Count bytes as they flow so the row total can be extrapolated from byte progress. + let bytesRead = 0 + const byteCounter = new Transform({ + transform(chunk: Buffer, _enc, cb) { + bytesRead += chunk.length + cb(null, chunk) + }, + }) + + const parser = createCsvParser(delimiter) + // `.pipe` doesn't forward source errors; forward so the iterator throws. + source.on('error', (err) => parser.destroy(err)) + byteCounter.on('error', (err) => parser.destroy(err)) + source.pipe(byteCounter).pipe(parser) + + let schema: TableSchema | null = null + let headerToColumn: Map | null = null + let inserted = 0 + let lastReported = 0 + const sample: Record[] = [] + let batch: Record[] = [] + + /** + * Resolve the schema + header→column mapping from the buffered sample (runs once). + * `create` infers a fresh schema and overwrites the placeholder; `append`/`replace` + * map onto the existing schema, optionally auto-creating `createColumns` first. + */ + const resolveSetup = async () => { + const headers = Object.keys(sample[0]) + + if (mode === 'create') { + const inferred = inferSchemaFromCsv(headers, sample) + schema = { columns: inferred.columns.map(normalizeColumn) } + headerToColumn = inferred.headerToColumn + await setTableSchemaForImport(tableId, schema) + return + } + + // append / replace into an existing table. + let targetSchema = table.schema + let effectiveMapping: CsvHeaderMapping = + payload.mapping ?? buildAutoMapping(headers, table.schema) + + if (payload.createColumns && payload.createColumns.length > 0) { + const unknown = payload.createColumns.filter((h) => !headers.includes(h)) + if (unknown.length > 0) { + throw new Error(`Columns to create are not in the CSV: ${unknown.join(', ')}`) + } + const usedNames = new Set(table.schema.columns.map((c) => c.name.toLowerCase())) + const additions: { name: string; type: string }[] = [] + const updatedMapping: CsvHeaderMapping = { ...effectiveMapping } + for (const header of payload.createColumns) { + const base = sanitizeName(header) + let columnName = base + let suffix = 2 + while (usedNames.has(columnName.toLowerCase())) { + columnName = `${base}_${suffix}` + suffix++ + } + usedNames.add(columnName.toLowerCase()) + additions.push({ name: columnName, type: inferColumnType(sample.map((r) => r[header])) }) + updatedMapping[header] = columnName + } + const updated = await addImportColumns(table, additions, requestId) + targetSchema = updated.schema + effectiveMapping = updatedMapping + } + + const validation = validateMapping({ + csvHeaders: headers, + mapping: effectiveMapping, + tableSchema: targetSchema, + }) + schema = targetSchema + headerToColumn = validation.effectiveMap + + // Replace deletes existing rows only after schema/mapping validation passes, so an + // invalid or empty file fails the import with the old rows still intact (a mid-stream + // insert failure after this point leaves a partial replace — replace is destructive). + if (mode === 'replace') await deleteAllTableRows(tableId) + } + + const flush = async (rows: Record[]) => { + if (rows.length === 0 || !schema || !headerToColumn) return + // Ownership gate before every insert: once this run loses the table (cancel/supersede), + // updateImportProgress returns false and we stop before writing into a table a newer import + // may own. Runs per batch (not just at the emit cadence) so we stop within one batch. + const owns = await updateImportProgress(tableId, inserted, importId) + if (!owns) throw new ImportSupersededError() + const coerced = coerceRowsForTable(rows, schema, headerToColumn) + inserted += await bulkInsertImportBatch( + { tableId, workspaceId, userId, rows: coerced, startPosition: basePosition + inserted }, + { ...table, schema }, + requestId + ) + // Emit after the first batch, then every interval, so the bar appears early without flooding. + if ( + inserted - lastReported >= PROGRESS_INTERVAL_ROWS || + (lastReported === 0 && inserted > 0) + ) { + lastReported = inserted + // Exact, monotonic completion from bytes consumed — no wobbly row estimate. + const percent = + totalBytes > 0 ? Math.min(99, Math.round((bytesRead / totalBytes) * 100)) : undefined + void appendTableEvent({ + kind: 'import', + tableId, + importId, + status: 'importing', + progress: inserted, + percent, + }) + } + } + + let ready = false + for await (const record of parser as AsyncIterable>) { + if (!ready) { + sample.push(record) + if (sample.length >= CSV_SCHEMA_SAMPLE_SIZE) { + await resolveSetup() + await flush(sample) + ready = true + } + continue + } + batch.push(record) + if (batch.length >= CSV_MAX_BATCH_SIZE) { + await flush(batch) + batch = [] + } + } + + if (!ready) { + // Fewer than CSV_SCHEMA_SAMPLE_SIZE rows total (or zero). + if (sample.length === 0) { + // No data rows — fail rather than report a successful empty import (matches the sync route). + const message = 'CSV file has no data rows' + await markImportFailed(tableId, importId, message) + void appendTableEvent({ + kind: 'import', + tableId, + importId, + status: 'failed', + error: message, + }) + logger.warn(`[${requestId}] Import has no data rows`, { tableId, fileName }) + return + } + await resolveSetup() + await flush(sample) + } else { + await flush(batch) + } + + await updateImportProgress(tableId, inserted, importId) + // Only announce success if we actually won the transition — a cancel/supersede that landed + // right at the end makes this a no-op, and we must not emit a false `ready`. + const becameReady = await markImportReady(tableId, importId) + if (becameReady) { + void appendTableEvent({ + kind: 'import', + tableId, + importId, + status: 'ready', + progress: inserted, + percent: 100, + }) + logger.info(`[${requestId}] Import complete`, { tableId, fileName, mode, rows: inserted }) + } else { + logger.info( + `[${requestId}] Import finished but no longer owns the run (canceled/superseded)`, + { + tableId, + importId, + } + ) + } + } catch (err) { + if (err instanceof ImportSupersededError) { + // A newer import owns the table now — leave its status alone and just stop. + logger.info(`[${requestId}] Import superseded by a newer run; stopping`, { + tableId, + importId, + }) + } else { + const message = getErrorMessage(err, 'Import failed') + logger.error(`[${requestId}] Import failed for table ${tableId}:`, err) + // Scoped to importId — a no-op if a newer import has taken over. + await markImportFailed(tableId, importId, message).catch(() => {}) + void appendTableEvent({ kind: 'import', tableId, importId, status: 'failed', error: message }) + } + } finally { + // Release the storage stream so its HTTP connection doesn't leak on failure. + source?.destroy() + // The uploaded source file is single-use (a fresh upload per import) — delete it once the + // import is terminal so the workspace bucket doesn't accumulate. Best-effort. + await deleteFile({ key: fileKey, context: 'workspace' }).catch((err) => { + logger.warn(`[${requestId}] Failed to delete imported file`, { fileKey, err }) + }) + } +} diff --git a/apps/sim/lib/table/import.test.ts b/apps/sim/lib/table/import.test.ts index 65d1607301..d25ee031e0 100644 --- a/apps/sim/lib/table/import.test.ts +++ b/apps/sim/lib/table/import.test.ts @@ -1,12 +1,15 @@ /** * @vitest-environment node */ +import { Readable } from 'node:stream' import { describe, expect, it } from 'vitest' import { buildAutoMapping, CsvImportValidationError, coerceRowsForTable, coerceValue, + createCsvParser, + csvParseOptions, inferColumnType, inferSchemaFromCsv, parseCsvBuffer, @@ -274,4 +277,43 @@ describe('import', () => { expect(rows).toEqual([{ name: 'Alice' }]) }) }) + + describe('createCsvParser', () => { + async function parseViaStream(csv: string, delimiter = ',') { + const parser = createCsvParser(delimiter) + Readable.from([csv]).pipe(parser) + const rows: Record[] = [] + for await (const record of parser as AsyncIterable>) { + rows.push(record) + } + return rows + } + + it('streams records keyed by header, matching parseCsvBuffer', async () => { + const csv = 'name,age\nAlice,30\nBob,40\n' + const streamed = await parseViaStream(csv) + const { rows: buffered } = await parseCsvBuffer(csv) + expect(streamed).toEqual(buffered) + expect(streamed).toEqual([ + { name: 'Alice', age: '30' }, + { name: 'Bob', age: '40' }, + ]) + }) + + it('honors a TSV delimiter', async () => { + const rows = await parseViaStream('name\tage\nAlice\t30\n', '\t') + expect(rows).toEqual([{ name: 'Alice', age: '30' }]) + }) + + it('strips a leading UTF-8 BOM', async () => { + const rows = await parseViaStream('name,age\nAlice,30\n') + expect(Object.keys(rows[0])).toEqual(['name', 'age']) + }) + }) + + describe('csvParseOptions', () => { + it('sets columns, bom, and the delimiter', () => { + expect(csvParseOptions('\t')).toMatchObject({ columns: true, bom: true, delimiter: '\t' }) + }) + }) }) diff --git a/apps/sim/lib/table/import.ts b/apps/sim/lib/table/import.ts index 23566c145d..843edd3d7a 100644 --- a/apps/sim/lib/table/import.ts +++ b/apps/sim/lib/table/import.ts @@ -2,15 +2,47 @@ * Shared CSV import helpers for user-defined tables. * * Used by: - * - `POST /api/table/import-csv` (create new table from CSV) + * - `POST /api/table/import-csv` (create new table from CSV — streams via {@link createCsvParser}) * - `POST /api/table/[tableId]/import` (append/replace into existing table) - * - Copilot `user-table` tool (`create_from_file`, `import_file`) + * - Copilot `user-table` tool (`create_from_file`, `import_file` — buffers via {@link parseCsvBuffer}) * * Keeping a single implementation avoids drift between HTTP and agent code paths. + * Both the buffered ({@link parseCsvBuffer}) and streaming ({@link createCsvParser}) + * parsers share {@link csvParseOptions} so their behavior can't drift. */ +import { type Options as CsvParseOptions, type Parser, parse as parseCsvStream } from 'csv-parse' import type { ColumnDefinition, RowData, TableSchema } from '@/lib/table/types' +/** + * Single source of truth for the `csv-parse` options used by both the buffered + * sync parser and the streaming parser. `columns: true` emits each record as an + * object keyed by the (first-row) headers. + */ +export function csvParseOptions(delimiter = ','): CsvParseOptions { + return { + columns: true, + skip_empty_lines: true, + trim: true, + relax_column_count: true, + relax_quotes: true, + skip_records_with_error: true, + cast: false, + bom: true, + delimiter, + } +} + +/** + * Returns a streaming `csv-parse` parser (a `Transform`/async-iterable). Pipe a + * file stream into it and iterate records with `for await`; backpressure flows + * back to the source while each record is processed. Use this for HTTP uploads + * so the file is never fully buffered in memory. + */ +export function createCsvParser(delimiter = ','): Parser { + return parseCsvStream(csvParseOptions(delimiter)) +} + /** Narrower type than `COLUMN_TYPES` used internally for coercion. */ export type CsvColumnType = 'string' | 'number' | 'boolean' | 'date' | 'json' @@ -53,8 +85,10 @@ export class CsvImportValidationError extends Error { /** * Parses a CSV/TSV payload using `csv-parse/sync`. Accepts a Node `Buffer`, - * browser-friendly `Uint8Array`, or already-decoded string. Strips a leading - * UTF-8 BOM so headers are not silently prefixed with `\uFEFF`. + * browser-friendly `Uint8Array`, or already-decoded string. A leading UTF-8 BOM + * is stripped by csv-parse (`bom: true` in {@link csvParseOptions}). + * + * For HTTP uploads prefer {@link createCsvParser} so the file isn't buffered. */ export async function parseCsvBuffer( input: Buffer | Uint8Array | string, @@ -70,18 +104,10 @@ export async function parseCsvBuffer( } else { text = new TextDecoder('utf-8').decode(input as Uint8Array) } - text = text.replace(/^\uFEFF/, '') - const parsed = parse(text, { - columns: true, - skip_empty_lines: true, - trim: true, - relax_column_count: true, - relax_quotes: true, - skip_records_with_error: true, - cast: false, - delimiter, - }) as Record[] + // double-cast-allowed: shared csvParseOptions() loses the `columns: true` literal that drives + // csv-parse's record-vs-string[][] overload, but `columns: true` is always set so records are objects + const parsed = parse(text, csvParseOptions(delimiter)) as unknown as Record[] if (parsed.length === 0) { throw new Error('CSV file has no data rows') @@ -389,3 +415,86 @@ export function coerceRowsForTable( return coerced }) } + +/** + * Sanitizes raw JSON keys so they conform to the same column-name rules as CSV + * headers, letting `inferSchemaFromCsv` and `coerceRowsForTable` be reused for + * JSON imports. Collisions after sanitization are disambiguated with a trailing + * underscore. Returns the headers and rows untouched when no key needs renaming. + */ +export function sanitizeJsonHeaders( + headers: string[], + rows: Record[] +): { headers: string[]; rows: Record[] } { + const renamed = new Map() + const seen = new Set() + + for (const raw of headers) { + let safe = sanitizeName(raw) + while (seen.has(safe)) safe = `${safe}_` + seen.add(safe) + renamed.set(raw, safe) + } + + const noChange = headers.every((h) => renamed.get(h) === h) + if (noChange) return { headers, rows } + + return { + headers: headers.map((h) => renamed.get(h)!), + rows: rows.map((row) => { + const out: Record = {} + for (const [raw, safe] of renamed) { + if (raw in row) out[safe] = row[raw] + } + return out + }), + } +} + +/** + * Parses a JSON payload that must be an array of plain objects into the same + * `{ headers, rows }` shape produced by `parseCsvBuffer`. The header set is the + * union of all object keys, sanitized via {@link sanitizeJsonHeaders}. + */ +export function parseJsonRows(buffer: Buffer | string): { + headers: string[] + rows: Record[] +} { + const text = typeof buffer === 'string' ? buffer : buffer.toString('utf-8') + const parsed = JSON.parse(text) + if (!Array.isArray(parsed)) { + throw new Error('JSON file must contain an array of objects') + } + if (parsed.length === 0) { + throw new Error('JSON file contains an empty array') + } + const headerSet = new Set() + for (const row of parsed) { + if (typeof row !== 'object' || row === null || Array.isArray(row)) { + throw new Error('Each element in the JSON array must be a plain object') + } + for (const key of Object.keys(row)) headerSet.add(key) + } + return sanitizeJsonHeaders([...headerSet], parsed) +} + +/** + * Parses a tabular upload (CSV, TSV, or JSON array-of-objects) into a uniform + * `{ headers, rows }` shape, dispatching on file extension and falling back to + * the MIME content type. Throws on unsupported formats so callers fail fast. + */ +export async function parseFileRows( + buffer: Buffer, + fileName: string, + contentType?: string +): Promise<{ headers: string[]; rows: Record[] }> { + const ext = fileName.split('.').pop()?.toLowerCase() + if (ext === 'json' || contentType === 'application/json') { + return parseJsonRows(buffer) + } + if (ext === 'csv' || ext === 'tsv' || contentType === 'text/csv') { + const delimiter = ext === 'tsv' ? '\t' : ',' + return parseCsvBuffer(buffer, delimiter) + } + throw new Error(`Unsupported file format: "${ext ?? fileName}". Supported: csv, tsv, json`) +} diff --git a/apps/sim/lib/table/service.ts b/apps/sim/lib/table/service.ts index 280a845340..065c61d676 100644 --- a/apps/sim/lib/table/service.ts +++ b/apps/sim/lib/table/service.ts @@ -17,7 +17,7 @@ import { import { createLogger } from '@sim/logger' import { getPostgresErrorCode } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' -import { and, count, eq, gt, gte, inArray, isNull, type SQL, sql } from 'drizzle-orm' +import { and, count, eq, gt, gte, inArray, isNull, ne, or, type SQL, sql } from 'drizzle-orm' import { MATERIALIZE_CONCURRENCY, mapWithConcurrency } from '@/lib/core/utils/concurrency' import { generateRestoreName } from '@/lib/core/utils/restore-name' import type { DbOrTx } from '@/lib/db/types' @@ -176,6 +176,21 @@ async function nextAutoPosition(trx: DbTransaction, tableId: string): Promise { + const [{ maxPos }] = await db + .select({ + maxPos: sql`coalesce(max(${userTableRows.position}), -1)`.mapWith(Number), + }) + .from(userTableRows) + .where(eq(userTableRows.tableId, tableId)) + return maxPos + 1 +} + const TIMEOUT_CAP_MS = 10 * 60_000 /** @@ -252,6 +267,11 @@ export async function getTableById( createdAt: userTableDefinitions.createdAt, updatedAt: userTableDefinitions.updatedAt, rowCount: userTableDefinitions.rowCount, + importStatus: userTableDefinitions.importStatus, + importId: userTableDefinitions.importId, + importError: userTableDefinitions.importError, + importRowsProcessed: userTableDefinitions.importRowsProcessed, + importStartedAt: userTableDefinitions.importStartedAt, }) .from(userTableDefinitions) .where( @@ -278,6 +298,11 @@ export async function getTableById( archivedAt: table.archivedAt, createdAt: table.createdAt, updatedAt: table.updatedAt, + importStatus: table.importStatus as TableDefinition['importStatus'], + importId: table.importId, + importError: table.importError, + importRowsProcessed: table.importRowsProcessed, + importStartedAt: table.importStartedAt, } } @@ -319,6 +344,11 @@ export async function listTables( createdAt: userTableDefinitions.createdAt, updatedAt: userTableDefinitions.updatedAt, rowCount: userTableDefinitions.rowCount, + importStatus: userTableDefinitions.importStatus, + importId: userTableDefinitions.importId, + importError: userTableDefinitions.importError, + importRowsProcessed: userTableDefinitions.importRowsProcessed, + importStartedAt: userTableDefinitions.importStartedAt, }) .from(userTableDefinitions) .where( @@ -351,6 +381,11 @@ export async function listTables( archivedAt: t.archivedAt, createdAt: t.createdAt, updatedAt: t.updatedAt, + importStatus: t.importStatus as TableDefinition['importStatus'], + importId: t.importId, + importError: t.importError, + importRowsProcessed: t.importRowsProcessed, + importStartedAt: t.importStartedAt, } }) } @@ -397,6 +432,9 @@ export async function createTable( archivedAt: null, createdAt: now, updatedAt: now, + importStatus: data.importStatus ?? null, + importId: data.importId ?? null, + importStartedAt: data.importStatus ? now : null, } // Wrap count check, duplicate check, and insert in a transaction with FOR UPDATE @@ -477,6 +515,10 @@ export async function createTable( archivedAt: newTable.archivedAt, createdAt: newTable.createdAt, updatedAt: newTable.updatedAt, + importStatus: newTable.importStatus as TableDefinition['importStatus'], + importId: newTable.importId, + importRowsProcessed: 0, + importStartedAt: newTable.importStartedAt, } } @@ -1202,6 +1244,232 @@ export function dispatchAfterBatchInsert( }).catch((err) => logger.error(`[${requestId}] auto-dispatch (batchInsertRows) failed:`, err)) } +/** One batch of rows for a background import (see {@link bulkInsertImportBatch}). */ +export interface BulkImportBatch { + tableId: string + workspaceId: string + userId?: string + rows: RowData[] + /** Position of the first row in this batch; rows get contiguous positions from here. */ + startPosition: number +} + +/** + * Inserts one batch of rows for an async import in a single committed statement. + * + * Differs from {@link batchInsertRowsWithTx} for the bulk-load case: caller-supplied + * contiguous positions (no `acquireTablePositionLock` / `nextAutoPosition` scan — an + * import owns its hidden table as the sole writer), no `RETURNING`, and **no + * `fireTableTrigger` / `runWorkflowColumn`** (a 1M-row import must not dispatch a + * workflow run per row). `row_count` is maintained set-based by the statement-level + * trigger. There is no surrounding transaction and no rollback: each batch commits on + * its own, so committed batches persist even if a later batch fails. + * + * Throws on row-size/schema/unique violations or if the statement-level trigger rejects + * the batch for crossing `max_rows`; the caller marks the import failed. + */ +export async function bulkInsertImportBatch( + data: BulkImportBatch, + table: TableDefinition, + requestId: string +): Promise { + for (let i = 0; i < data.rows.length; i++) { + const sizeValidation = validateRowSize(data.rows[i]) + if (!sizeValidation.valid) { + throw new Error(`Row ${i + 1}: ${sizeValidation.errors.join(', ')}`) + } + const schemaValidation = coerceRowToSchema(data.rows[i], table.schema) + if (!schemaValidation.valid) { + throw new Error(`Row ${i + 1}: ${schemaValidation.errors.join(', ')}`) + } + } + + const uniqueColumns = getUniqueColumns(table.schema) + if (uniqueColumns.length > 0) { + const uniqueResult = await checkBatchUniqueConstraintsDb( + data.tableId, + data.rows, + table.schema, + db + ) + if (!uniqueResult.valid) { + throw new Error( + uniqueResult.errors.map((e) => `Row ${e.row + 1}: ${e.errors.join(', ')}`).join('; ') + ) + } + } + + const now = new Date() + const rowsToInsert = data.rows.map((rowData, i) => ({ + id: `row_${generateId().replace(/-/g, '')}`, + tableId: data.tableId, + workspaceId: data.workspaceId, + data: rowData, + position: data.startPosition + i, + createdAt: now, + updatedAt: now, + ...(data.userId ? { createdBy: data.userId } : {}), + })) + + await db.insert(userTableRows).values(rowsToInsert) + logger.info(`[${requestId}] Bulk-imported ${rowsToInsert.length} rows into table ${data.tableId}`) + return rowsToInsert.length +} + +/** Deletes every row of a table (set-based; the statement-level trigger zeroes `row_count`). */ +export async function deleteAllTableRows(tableId: string): Promise { + await db.delete(userTableRows).where(eq(userTableRows.tableId, tableId)) +} + +/** + * Adds columns to a table during an import (the `createColumns` flow), wrapping the + * tx-bound {@link addTableColumnsWithTx} in its own transaction. Returns the updated table. + */ +export async function addImportColumns( + table: TableDefinition, + additions: { name: string; type: string }[], + requestId: string +): Promise { + return db.transaction((trx) => addTableColumnsWithTx(trx, table, additions, requestId)) +} + +/** Overwrites a table's schema during an import (used when inferring columns from the file). */ +export async function setTableSchemaForImport(tableId: string, schema: TableSchema): Promise { + await db + .update(userTableDefinitions) + .set({ schema, updatedAt: new Date() }) + .where(eq(userTableDefinitions.id, tableId)) +} + +/** + * Atomically claims a table for an async import. The `import_status != 'importing'` guard makes + * this the single concurrency gate: of two racing kickoffs only one row-update matches, so only + * one wins (no TOCTOU between a separate status check and this write). Returns whether it claimed + * the table — the caller returns 409 when it didn't. + */ +export async function markTableImporting(tableId: string, importId: string): Promise { + const updated = await db + .update(userTableDefinitions) + .set({ + importStatus: 'importing', + importId, + importError: null, + importRowsProcessed: 0, + importStartedAt: new Date(), + updatedAt: new Date(), + }) + .where( + and( + eq(userTableDefinitions.id, tableId), + or( + isNull(userTableDefinitions.importStatus), + ne(userTableDefinitions.importStatus, 'importing') + ) + ) + ) + .returning({ id: userTableDefinitions.id }) + return updated.length > 0 +} + +/** + * Releases a claim taken by {@link markTableImporting} for a synchronous import — clears the + * import state back to idle. Scoped to `importId` so it only clears its own claim, never a newer + * run that may have taken over. A sync route claims, writes, then releases here in a `finally`. + */ +export async function releaseImportClaim(tableId: string, importId: string): Promise { + await db + .update(userTableDefinitions) + .set({ importStatus: null, importId: null, importStartedAt: null, updatedAt: new Date() }) + .where( + and( + eq(userTableDefinitions.id, tableId), + eq(userTableDefinitions.importId, importId), + eq(userTableDefinitions.importStatus, 'importing') + ) + ) +} + +/** + * Records import progress (rows processed so far). Also bumps `updatedAt` so the + * stale-import janitor (`cleanup-stale-executions`) sees a live heartbeat and doesn't mark a + * still-running import as failed. + * + * Scoped to `importId` AND `import_status = 'importing'`: a stale/superseded worker no longer + * matches (its write is a no-op), and once the import is terminal (e.g. canceled) the match fails + * too — so this returning `false` is also the worker's signal to stop. Returns whether this worker + * still owns an in-flight import. + */ +export async function updateImportProgress( + tableId: string, + rowsProcessed: number, + importId: string +): Promise { + const updated = await db + .update(userTableDefinitions) + .set({ importRowsProcessed: rowsProcessed, updatedAt: new Date() }) + .where( + and( + eq(userTableDefinitions.id, tableId), + eq(userTableDefinitions.importId, importId), + eq(userTableDefinitions.importStatus, 'importing') + ) + ) + .returning({ id: userTableDefinitions.id }) + return updated.length > 0 +} + +/** Shared WHERE for terminal transitions: this import run, and still in-flight (write-once). */ +function ownsActiveImport(tableId: string, importId: string) { + return and( + eq(userTableDefinitions.id, tableId), + eq(userTableDefinitions.importId, importId), + eq(userTableDefinitions.importStatus, 'importing') + ) +} + +/** + * Marks an import complete; rows become visible. No-op unless it's still this in-flight run. + * Returns whether it transitioned, so the worker only emits the `ready` event when it actually + * won (and not after a cancel / supersede). + */ +export async function markImportReady(tableId: string, importId: string): Promise { + const updated = await db + .update(userTableDefinitions) + .set({ importStatus: 'ready', importError: null, updatedAt: new Date() }) + .where(ownsActiveImport(tableId, importId)) + .returning({ id: userTableDefinitions.id }) + return updated.length > 0 +} + +/** + * Marks an import failed, leaving any already-committed rows in place. No-op unless it's still + * this in-flight run (so a stale worker can't clobber a newer import or a cancel). + */ +export async function markImportFailed( + tableId: string, + importId: string, + error: string +): Promise { + await db + .update(userTableDefinitions) + .set({ importStatus: 'failed', importError: error.slice(0, 2000), updatedAt: new Date() }) + .where(ownsActiveImport(tableId, importId)) +} + +/** + * Marks an in-flight import canceled (user-initiated). No-op unless it's still importing. The + * worker's next ownership check then returns `false` and it stops; committed rows are left in + * place (no rollback). Returns whether a running import was actually canceled. + */ +export async function markImportCanceled(tableId: string, importId: string): Promise { + const updated = await db + .update(userTableDefinitions) + .set({ importStatus: 'canceled', updatedAt: new Date() }) + .where(ownsActiveImport(tableId, importId)) + .returning({ id: userTableDefinitions.id }) + return updated.length > 0 +} + /** * Replaces all rows in a table with a new set of rows. Deletes existing rows * and inserts the provided rows inside a single transaction so the table is diff --git a/apps/sim/lib/table/types.ts b/apps/sim/lib/table/types.ts index 279d149c39..a20789a848 100644 --- a/apps/sim/lib/table/types.ts +++ b/apps/sim/lib/table/types.ts @@ -152,6 +152,9 @@ export interface TableMetadata { pinnedColumns?: string[] } +/** Async-import lifecycle state for a table. NULL/undefined = normal (no async import). */ +export type TableImportStatus = 'importing' | 'ready' | 'failed' | 'canceled' + export interface TableDefinition { id: string name: string @@ -165,6 +168,12 @@ export interface TableDefinition { archivedAt?: Date | string | null createdAt: Date | string updatedAt: Date | string + /** Async-import state (see `apps/sim/lib/table/import-runner.ts`). */ + importStatus?: TableImportStatus | null + importId?: string | null + importError?: string | null + importRowsProcessed?: number + importStartedAt?: Date | string | null } /** Minimal table info for UI components. */ @@ -296,6 +305,10 @@ export interface CreateTableData { maxTables?: number /** Number of empty rows to create with the table. Defaults to 0. */ initialRowCount?: number + /** When set, the table is created in this async-import state (rows hidden until ready). */ + importStatus?: TableImportStatus + /** Async-import id stamped on the table when `importStatus` is set. */ + importId?: string } export interface InsertRowData { diff --git a/apps/sim/lib/uploads/core/storage-service.ts b/apps/sim/lib/uploads/core/storage-service.ts index f730d49bea..d0973a5552 100644 --- a/apps/sim/lib/uploads/core/storage-service.ts +++ b/apps/sim/lib/uploads/core/storage-service.ts @@ -1,3 +1,4 @@ +import type { Readable } from 'node:stream' import { randomBytes } from 'crypto' import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' @@ -222,6 +223,34 @@ export async function downloadFile(options: DownloadFileOptions): Promise { + const { key, context } = options + const config = getStorageConfig(context) + + if (USE_BLOB_STORAGE) { + const { downloadFromBlobStream } = await import('@/lib/uploads/providers/blob/client') + return downloadFromBlobStream(key, createBlobConfig(config)) + } + + if (USE_S3_STORAGE) { + const { downloadFromS3Stream } = await import('@/lib/uploads/providers/s3/client') + return downloadFromS3Stream(key, createS3Config(config)) + } + + const { createReadStream } = await import('fs') + const { join } = await import('path') + const { UPLOAD_DIR_SERVER } = await import('./setup.server') + return createReadStream(join(UPLOAD_DIR_SERVER, sanitizeFileKey(key))) +} + /** * Delete a file from the configured storage provider */ diff --git a/apps/sim/lib/uploads/providers/blob/client.ts b/apps/sim/lib/uploads/providers/blob/client.ts index 5ff536bfb5..b517d9ed36 100644 --- a/apps/sim/lib/uploads/providers/blob/client.ts +++ b/apps/sim/lib/uploads/providers/blob/client.ts @@ -1,3 +1,4 @@ +import type { Readable } from 'node:stream' import type { BlobServiceClient as BlobServiceClientType } from '@azure/storage-blob' import { createLogger } from '@sim/logger' import { generateId } from '@sim/utils/id' @@ -341,6 +342,49 @@ export async function downloadFromBlob( return downloaded } +/** + * Stream a blob out of storage without buffering it. The caller MUST fully consume or + * `destroy()` the returned stream. Used by the large-CSV import worker. + */ +export async function downloadFromBlobStream( + key: string, + customConfig?: BlobConfig +): Promise { + const { BlobServiceClient, StorageSharedKeyCredential } = await import('@azure/storage-blob') + let blobServiceClient: BlobServiceClientType + let containerName: string + + if (customConfig) { + if (customConfig.connectionString) { + blobServiceClient = BlobServiceClient.fromConnectionString(customConfig.connectionString) + } else if (customConfig.accountName && customConfig.accountKey) { + const credential = new StorageSharedKeyCredential( + customConfig.accountName, + customConfig.accountKey + ) + blobServiceClient = new BlobServiceClient( + `https://${customConfig.accountName}.blob.core.windows.net`, + credential + ) + } else { + throw new Error('Invalid custom blob configuration') + } + containerName = customConfig.containerName + } else { + blobServiceClient = await getBlobServiceClient() + containerName = BLOB_CONFIG.containerName + } + + const containerClient = blobServiceClient.getContainerClient(containerName) + const blockBlobClient = containerClient.getBlockBlobClient(key) + + const downloadBlockBlobResponse = await blockBlobClient.download() + if (!downloadBlockBlobResponse.readableStreamBody) { + throw new Error('Failed to get readable stream from blob download') + } + return downloadBlockBlobResponse.readableStreamBody as Readable +} + /** * Check whether a blob exists (and return its size when it does). * Returns null when the blob is missing. diff --git a/apps/sim/lib/uploads/providers/s3/client.ts b/apps/sim/lib/uploads/providers/s3/client.ts index 7ddae7bbf8..cff13eee06 100644 --- a/apps/sim/lib/uploads/providers/s3/client.ts +++ b/apps/sim/lib/uploads/providers/s3/client.ts @@ -1,3 +1,4 @@ +import type { Readable } from 'node:stream' import { AbortMultipartUploadCommand, CompleteMultipartUploadCommand, @@ -223,6 +224,24 @@ export async function downloadFromS3( }) } +/** + * Stream an object out of S3 without buffering it. The caller MUST fully consume or + * `destroy()` the returned stream. Used by the large-CSV import worker so a 1M-row file is + * never resident in memory. + */ +export async function downloadFromS3Stream( + key: string, + customConfig?: S3Config +): Promise { + const config = customConfig || { bucket: S3_CONFIG.bucket, region: S3_CONFIG.region } + const command = new GetObjectCommand({ Bucket: config.bucket, Key: key }) + const response = await getS3Client().send(command) + if (!response.Body) { + throw new Error(`S3 object has no body: ${key}`) + } + return response.Body as Readable +} + /** * Check whether an object exists in S3 (and return its size when it does). * Returns null when the object is missing. diff --git a/apps/sim/package.json b/apps/sim/package.json index 48037fcbd7..d11921af52 100644 --- a/apps/sim/package.json +++ b/apps/sim/package.json @@ -114,6 +114,7 @@ "better-auth-harmony": "1.3.1", "binary-extensions": "3.1.0", "browser-image-compression": "^2.0.2", + "busboy": "1.6.0", "cheerio": "1.1.2", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", @@ -212,6 +213,7 @@ "@tailwindcss/typography": "0.5.19", "@testing-library/jest-dom": "^6.6.3", "@trigger.dev/build": "4.4.3", + "@types/busboy": "1.5.4", "@types/fluent-ffmpeg": "2.1.28", "@types/html-to-text": "9.0.4", "@types/js-yaml": "4.0.9", diff --git a/apps/sim/stores/table/import-tray/store.ts b/apps/sim/stores/table/import-tray/store.ts new file mode 100644 index 0000000000..66be508089 --- /dev/null +++ b/apps/sim/stores/table/import-tray/store.ts @@ -0,0 +1,109 @@ +import { create } from 'zustand' +import { devtools } from 'zustand/middleware' + +/** + * An in-flight client upload, shown optimistically before its server import row exists or the + * table list has refreshed. Keyed by `uploadId`: a `pending_*` id (creating a new table, no row + * yet) or the target tableId (append/replace into an existing table). + */ +export interface ImportUpload { + uploadId: string + workspaceId: string + title: string + /** Byte-based upload percent from the client XHR. */ + percent?: number +} + +/** + * Client-only state for the import tray. The importing/terminal rows themselves are derived from + * the table list (React Query) — this store holds only what the server doesn't: optimistic uploads, + * which terminal completions to surface this session, canceled ids, and the menu's open state. + */ +interface ImportTrayState { + uploads: Record + /** Terminal (`ready`/`failed`) table ids to surface as a card this session. */ + notified: Record + /** Ids (upload or table) canceled so callbacks/derivation don't resurrect them. */ + canceledIds: Record + menuOpen: boolean + + startUpload: (upload: ImportUpload) => void + setUploadPercent: (uploadId: string, percent: number) => void + endUpload: (uploadId: string) => void + /** Surface a terminal completion as a tray card. */ + notify: (tableId: string) => void + /** Remove a terminal card (manual dismiss or auto-clear). */ + dismiss: (tableId: string) => void + /** Flag an id canceled and drop any optimistic upload for it. */ + cancel: (id: string) => void + isCanceled: (id: string) => boolean + /** Returns whether the id was canceled and clears the flag (one-shot, for the kickoff handler). */ + consumeCanceled: (id: string) => boolean + setMenuOpen: (open: boolean) => void + reset: () => void +} + +const initialState = { + uploads: {} as Record, + notified: {} as Record, + canceledIds: {} as Record, + menuOpen: false, +} + +export const useImportTrayStore = create()( + devtools( + (set, get) => ({ + ...initialState, + + startUpload: (upload) => + set((state) => ({ uploads: { ...state.uploads, [upload.uploadId]: upload } })), + + setUploadPercent: (uploadId, percent) => + set((state) => { + const prev = state.uploads[uploadId] + if (!prev) return state + return { uploads: { ...state.uploads, [uploadId]: { ...prev, percent } } } + }), + + endUpload: (uploadId) => + set((state) => { + if (!state.uploads[uploadId]) return state + const { [uploadId]: _removed, ...rest } = state.uploads + return { uploads: rest } + }), + + notify: (tableId) => set((state) => ({ notified: { ...state.notified, [tableId]: true } })), + + dismiss: (tableId) => + set((state) => { + if (!state.notified[tableId]) return state + const { [tableId]: _removed, ...rest } = state.notified + return { notified: rest } + }), + + cancel: (id) => + set((state) => { + const { [id]: _removed, ...uploads } = state.uploads + return { uploads, canceledIds: { ...state.canceledIds, [id]: true } } + }), + + isCanceled: (id) => Boolean(get().canceledIds[id]), + + consumeCanceled: (id) => { + const was = Boolean(get().canceledIds[id]) + if (was) { + set((state) => { + const { [id]: _removed, ...rest } = state.canceledIds + return { canceledIds: rest } + }) + } + return was + }, + + setMenuOpen: (open) => set({ menuOpen: open }), + + reset: () => set(initialState), + }), + { name: 'import-tray-store' } + ) +) diff --git a/bun.lock b/bun.lock index 42b1506b65..562fdc1411 100644 --- a/bun.lock +++ b/bun.lock @@ -168,6 +168,7 @@ "better-auth-harmony": "1.3.1", "binary-extensions": "3.1.0", "browser-image-compression": "^2.0.2", + "busboy": "1.6.0", "cheerio": "1.1.2", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", @@ -266,6 +267,7 @@ "@tailwindcss/typography": "0.5.19", "@testing-library/jest-dom": "^6.6.3", "@trigger.dev/build": "4.4.3", + "@types/busboy": "1.5.4", "@types/fluent-ffmpeg": "2.1.28", "@types/html-to-text": "9.0.4", "@types/js-yaml": "4.0.9", @@ -1662,6 +1664,8 @@ "@types/braces": ["@types/braces@3.0.5", "", {}, "sha512-SQFof9H+LXeWNz8wDe7oN5zu7ket0qwMu5vZubW4GCJ8Kkeh6nBWUz87+KTz/G3Kqsrp0j/W253XJb3KMEeg3w=="], + "@types/busboy": ["@types/busboy@1.5.4", "", { "dependencies": { "@types/node": "*" } }, "sha512-kG7WrUuAKK0NoyxfQHsVE6j1m01s6kMma64E+OZenQABMQyTJop1DumUWcLwAQ2JzpefU7PDYoRDKl8uZosFjw=="], + "@types/chai": ["@types/chai@5.2.3", "", { "dependencies": { "@types/deep-eql": "*", "assertion-error": "^2.0.1" } }, "sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA=="], "@types/cookie": ["@types/cookie@0.4.1", "", {}, "sha512-XW/Aa8APYr6jSVVA1y/DEIZX0/GMKLEVekNG727R8cs56ahETkRAy/3DR7+fJyh7oUgGwNQaRfXCun0+KbWY7Q=="], @@ -1984,6 +1988,8 @@ "buildcheck": ["buildcheck@0.0.7", "", {}, "sha512-lHblz4ahamxpTmnsk+MNTRWsjYKv965MwOrSJyeD588rR3Jcu7swE+0wN5F+PbL5cjgu/9ObkhfzEPuofEMwLA=="], + "busboy": ["busboy@1.6.0", "", { "dependencies": { "streamsearch": "^1.1.0" } }, "sha512-8SFQbg/0hQ9xy3UNTB0YEnsNBbWfhf7RtnzpL7TkBiTBRfrQ9Fxcnz7VJsleJpyp6rVLvXiuORqjlHi5q+PYuA=="], + "bytes": ["bytes@3.1.2", "", {}, "sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg=="], "c12": ["c12@3.1.0", "", { "dependencies": { "chokidar": "^4.0.3", "confbox": "^0.2.2", "defu": "^6.1.4", "dotenv": "^16.6.1", "exsolve": "^1.0.7", "giget": "^2.0.0", "jiti": "^2.4.2", "ohash": "^2.0.11", "pathe": "^2.0.3", "perfect-debounce": "^1.0.0", "pkg-types": "^2.2.0", "rc9": "^2.1.2" }, "peerDependencies": { "magicast": "^0.3.5" }, "optionalPeers": ["magicast"] }, "sha512-uWoS8OU1MEIsOv8p/5a82c3H31LsWVR5qiyXVfBNOzfffjUWtPnhAb4BYI2uG2HfGmZmFjCtui5XNWaps+iFuw=="], @@ -3640,6 +3646,8 @@ "streamdown": ["streamdown@2.5.0", "", { "dependencies": { "clsx": "^2.1.1", "hast-util-to-jsx-runtime": "^2.3.6", "html-url-attributes": "^3.0.1", "marked": "^17.0.1", "mermaid": "^11.12.2", "rehype-harden": "^1.1.8", "rehype-raw": "^7.0.0", "rehype-sanitize": "^6.0.0", "remark-gfm": "^4.0.1", "remark-parse": "^11.0.0", "remark-rehype": "^11.1.2", "remend": "1.3.0", "tailwind-merge": "^3.4.0", "unified": "^11.0.5", "unist-util-visit": "^5.0.0", "unist-util-visit-parents": "^6.0.0" }, "peerDependencies": { "react": "^18.0.0 || ^19.0.0", "react-dom": "^18.0.0 || ^19.0.0" } }, "sha512-/tTnURfIOxZK/pqJAxsfCvETG/XCJHoWnk3jq9xLcuz6CSpnjjuxSRBTTL4PKGhxiZQf0lqPxGhImdpwcZ2XwA=="], + "streamsearch": ["streamsearch@1.1.0", "", {}, "sha512-Mcc5wHehp9aXz1ax6bZUyY5afg9u2rv5cqQI3mRrYkGC8rW2hM02jWuwjtL++LS5qinSyhj2QfLyNsuc+VsExg=="], + "streamx": ["streamx@2.25.0", "", { "dependencies": { "events-universal": "^1.0.0", "fast-fifo": "^1.3.2", "text-decoder": "^1.1.0" } }, "sha512-0nQuG6jf1w+wddNEEXCF4nTg3LtufWINB5eFEN+5TNZW7KWJp6x87+JFL43vaAUPyCfH1wID+mNVyW6OHtFamg=="], "string-argv": ["string-argv@0.3.2", "", {}, "sha512-aqD2Q0144Z+/RqG52NeHEkZauTAUWJO8c6yTftGJKO3Tja5tUgIfmIl6kExvhtxSDP7fXB6DvzkfMpCd/F3G+Q=="], @@ -4132,6 +4140,8 @@ "@trigger.dev/sdk/uuid": ["uuid@9.0.1", "", { "bin": { "uuid": "dist/bin/uuid" } }, "sha512-b+1eJOlsR9K8HJpow9Ok3fiWOWSIcIzXodvv0rQjVoOVNpWMpxf1wZNpt4y9h10odCNrqnYp1OBzRktckBe3sA=="], + "@types/busboy/@types/node": ["@types/node@25.6.2", "", { "dependencies": { "undici-types": "~7.19.0" } }, "sha512-sokuT28dxf9JT5Kady1fsXOvI4HVpjZa95NKT5y9PNTIrs2AsobR4GFAA90ZG8M+nxVRLysCXsVj6eGC7Vbrlw=="], + "@types/cors/@types/node": ["@types/node@25.6.2", "", { "dependencies": { "undici-types": "~7.19.0" } }, "sha512-sokuT28dxf9JT5Kady1fsXOvI4HVpjZa95NKT5y9PNTIrs2AsobR4GFAA90ZG8M+nxVRLysCXsVj6eGC7Vbrlw=="], "@types/fluent-ffmpeg/@types/node": ["@types/node@24.2.1", "", { "dependencies": { "undici-types": "~7.10.0" } }, "sha512-DRh5K+ka5eJic8CjH7td8QpYEV6Zo10gfRkjHCO3weqZHWDtAaSTFtl4+VMqOJ4N5jcuhZ9/l+yy8rVgw7BQeQ=="], @@ -4592,6 +4602,8 @@ "@trigger.dev/core/socket.io-client/engine.io-client": ["engine.io-client@6.5.4", "", { "dependencies": { "@socket.io/component-emitter": "~3.1.0", "debug": "~4.3.1", "engine.io-parser": "~5.2.1", "ws": "~8.17.1", "xmlhttprequest-ssl": "~2.0.0" } }, "sha512-GeZeeRjpD2qf49cZQ0Wvh/8NJNfeXkXXcoGh+F77oEAgo9gUHwT1fCRxSNU+YEEaysOJTnsFHmM5oAcPy4ntvQ=="], + "@types/busboy/@types/node/undici-types": ["undici-types@7.19.2", "", {}, "sha512-qYVnV5OEm2AW8cJMCpdV20CDyaN3g0AjDlOGf1OW4iaDEx8MwdtChUp4zu4H0VP3nDRF/8RKWH+IPp9uW0YGZg=="], + "@types/cors/@types/node/undici-types": ["undici-types@7.19.2", "", {}, "sha512-qYVnV5OEm2AW8cJMCpdV20CDyaN3g0AjDlOGf1OW4iaDEx8MwdtChUp4zu4H0VP3nDRF/8RKWH+IPp9uW0YGZg=="], "@types/fluent-ffmpeg/@types/node/undici-types": ["undici-types@7.10.0", "", {}, "sha512-t5Fy/nfn+14LuOc2KNYg75vZqClpAiqscVvMygNnlsHBFpSXdJaYtXMcdNLpl/Qvc3P2cB3s6lOV51nqsFq4ag=="], diff --git a/packages/db/migrations/0224_table_import_columns.sql b/packages/db/migrations/0224_table_import_columns.sql new file mode 100644 index 0000000000..a0796056d8 --- /dev/null +++ b/packages/db/migrations/0224_table_import_columns.sql @@ -0,0 +1,92 @@ +ALTER TABLE "user_table_definitions" ADD COLUMN "import_status" text;--> statement-breakpoint +ALTER TABLE "user_table_definitions" ADD COLUMN "import_id" text;--> statement-breakpoint +ALTER TABLE "user_table_definitions" ADD COLUMN "import_error" text;--> statement-breakpoint +ALTER TABLE "user_table_definitions" ADD COLUMN "import_rows_processed" integer DEFAULT 0 NOT NULL;--> statement-breakpoint +ALTER TABLE "user_table_definitions" ADD COLUMN "import_started_at" timestamp;--> statement-breakpoint + +-- ============================================================ +-- Statement-level row-count maintenance for user_table_rows. +-- +-- Replaces the per-row BEFORE INSERT / AFTER DELETE triggers from migration 0158 +-- (whose increment function body was rewritten race-free in 0198, but which still +-- fired FOR EACH ROW). Per-row firing serialized a row-level lock on the single +-- user_table_definitions row once per inserted/deleted row -- the dominant cost and +-- contention point for bulk operations (e.g. a 1M-row import = 1M lock cycles). +-- +-- The statement-level versions use transition tables to bump row_count by the +-- per-table count of affected rows in ONE UPDATE per statement, preserving the +-- atomic cap check. Transition tables require AFTER triggers, so the insert trigger +-- moves BEFORE -> AFTER: rows are inserted, then the count is bumped with the cap +-- check; an over-cap batch RAISEs and rolls back the whole statement. +-- ============================================================ + +CREATE OR REPLACE FUNCTION increment_user_table_row_count_stmt() +RETURNS TRIGGER AS $$ +DECLARE + over_cap text; +BEGIN + -- Per-table counts within this statement; one capped UPDATE per affected table. + -- A table_id present in the inserted rows always exists (FK), so any table the + -- UPDATE did not touch was rejected by the `row_count + n <= max_rows` guard. + WITH counts AS ( + SELECT table_id, count(*)::int AS n + FROM new_rows + GROUP BY table_id + ), + updated AS ( + UPDATE user_table_definitions d + SET row_count = d.row_count + c.n, + updated_at = now() + FROM counts c + WHERE d.id = c.table_id + AND d.row_count + c.n <= d.max_rows + RETURNING d.id + ) + SELECT string_agg(c.table_id, ', ') + INTO over_cap + FROM counts c + WHERE c.table_id NOT IN (SELECT id FROM updated); + + IF over_cap IS NOT NULL THEN + RAISE EXCEPTION 'Maximum row limit reached for table(s) %', over_cap + USING ERRCODE = 'check_violation'; + END IF; + + RETURN NULL; +END; +$$ LANGUAGE plpgsql; +--> statement-breakpoint + +CREATE OR REPLACE FUNCTION decrement_user_table_row_count_stmt() +RETURNS TRIGGER AS $$ +BEGIN + UPDATE user_table_definitions d + SET row_count = GREATEST(d.row_count - c.n, 0), + updated_at = now() + FROM ( + SELECT table_id, count(*)::int AS n + FROM old_rows + GROUP BY table_id + ) c + WHERE d.id = c.table_id; + + RETURN NULL; +END; +$$ LANGUAGE plpgsql; +--> statement-breakpoint + +DROP TRIGGER IF EXISTS user_table_rows_insert_trigger ON user_table_rows;--> statement-breakpoint +DROP TRIGGER IF EXISTS user_table_rows_delete_trigger ON user_table_rows;--> statement-breakpoint + +CREATE TRIGGER user_table_rows_insert_stmt_trigger + AFTER INSERT ON user_table_rows + REFERENCING NEW TABLE AS new_rows + FOR EACH STATEMENT + EXECUTE FUNCTION increment_user_table_row_count_stmt(); +--> statement-breakpoint + +CREATE TRIGGER user_table_rows_delete_stmt_trigger + AFTER DELETE ON user_table_rows + REFERENCING OLD TABLE AS old_rows + FOR EACH STATEMENT + EXECUTE FUNCTION decrement_user_table_row_count_stmt(); diff --git a/packages/db/migrations/meta/0224_snapshot.json b/packages/db/migrations/meta/0224_snapshot.json new file mode 100644 index 0000000000..f400ae3d2f --- /dev/null +++ b/packages/db/migrations/meta/0224_snapshot.json @@ -0,0 +1,17223 @@ +{ + "id": "7d34002f-2e0d-4b4b-9d06-c06334853a07", + "prevId": "09c4bd24-1cac-447d-a9bf-355ebb058cea", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.a2a_agent": { + "name": "a2a_agent", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "version": { + "name": "version", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'1.0.0'" + }, + "capabilities": { + "name": "capabilities", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "skills": { + "name": "skills", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'" + }, + "authentication": { + "name": "authentication", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "signatures": { + "name": "signatures", + "type": "jsonb", + "primaryKey": false, + "notNull": false, + "default": "'[]'" + }, + "is_published": { + "name": "is_published", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "published_at": { + "name": "published_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "a2a_agent_workflow_id_idx": { + "name": "a2a_agent_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "a2a_agent_created_by_idx": { + "name": "a2a_agent_created_by_idx", + "columns": [ + { + "expression": "created_by", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "a2a_agent_workspace_workflow_unique": { + "name": "a2a_agent_workspace_workflow_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"a2a_agent\".\"archived_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "a2a_agent_archived_at_idx": { + "name": "a2a_agent_archived_at_idx", + "columns": [ + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "a2a_agent_workspace_archived_partial_idx": { + "name": "a2a_agent_workspace_archived_partial_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"a2a_agent\".\"archived_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "a2a_agent_workspace_id_workspace_id_fk": { + "name": "a2a_agent_workspace_id_workspace_id_fk", + "tableFrom": "a2a_agent", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "a2a_agent_workflow_id_workflow_id_fk": { + "name": "a2a_agent_workflow_id_workflow_id_fk", + "tableFrom": "a2a_agent", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "a2a_agent_created_by_user_id_fk": { + "name": "a2a_agent_created_by_user_id_fk", + "tableFrom": "a2a_agent", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.a2a_push_notification_config": { + "name": "a2a_push_notification_config", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "task_id": { + "name": "task_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "url": { + "name": "url", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "token": { + "name": "token", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "auth_schemes": { + "name": "auth_schemes", + "type": "jsonb", + "primaryKey": false, + "notNull": false, + "default": "'[]'" + }, + "auth_credentials": { + "name": "auth_credentials", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "is_active": { + "name": "is_active", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "a2a_push_notification_config_task_unique": { + "name": "a2a_push_notification_config_task_unique", + "columns": [ + { + "expression": "task_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "a2a_push_notification_config_task_id_a2a_task_id_fk": { + "name": "a2a_push_notification_config_task_id_a2a_task_id_fk", + "tableFrom": "a2a_push_notification_config", + "tableTo": "a2a_task", + "columnsFrom": ["task_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.a2a_task": { + "name": "a2a_task", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "agent_id": { + "name": "agent_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "session_id": { + "name": "session_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "a2a_task_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'submitted'" + }, + "messages": { + "name": "messages", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'" + }, + "artifacts": { + "name": "artifacts", + "type": "jsonb", + "primaryKey": false, + "notNull": false, + "default": "'[]'" + }, + "execution_id": { + "name": "execution_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": false, + "default": "'{}'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "a2a_task_agent_id_idx": { + "name": "a2a_task_agent_id_idx", + "columns": [ + { + "expression": "agent_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "a2a_task_session_id_idx": { + "name": "a2a_task_session_id_idx", + "columns": [ + { + "expression": "session_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "a2a_task_status_idx": { + "name": "a2a_task_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "a2a_task_execution_id_idx": { + "name": "a2a_task_execution_id_idx", + "columns": [ + { + "expression": "execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "a2a_task_created_at_idx": { + "name": "a2a_task_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "a2a_task_agent_id_a2a_agent_id_fk": { + "name": "a2a_task_agent_id_a2a_agent_id_fk", + "tableFrom": "a2a_task", + "tableTo": "a2a_agent", + "columnsFrom": ["agent_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.academy_certificate": { + "name": "academy_certificate", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "course_id": { + "name": "course_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "academy_cert_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "issued_at": { + "name": "issued_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "certificate_number": { + "name": "certificate_number", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "academy_certificate_user_id_idx": { + "name": "academy_certificate_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "academy_certificate_course_id_idx": { + "name": "academy_certificate_course_id_idx", + "columns": [ + { + "expression": "course_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "academy_certificate_user_course_unique": { + "name": "academy_certificate_user_course_unique", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "course_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "academy_certificate_number_idx": { + "name": "academy_certificate_number_idx", + "columns": [ + { + "expression": "certificate_number", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "academy_certificate_status_idx": { + "name": "academy_certificate_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "academy_certificate_user_id_user_id_fk": { + "name": "academy_certificate_user_id_user_id_fk", + "tableFrom": "academy_certificate", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "academy_certificate_certificate_number_unique": { + "name": "academy_certificate_certificate_number_unique", + "nullsNotDistinct": false, + "columns": ["certificate_number"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.account": { + "name": "account", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "account_id": { + "name": "account_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider_id": { + "name": "provider_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "access_token": { + "name": "access_token", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "refresh_token": { + "name": "refresh_token", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "id_token": { + "name": "id_token", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "access_token_expires_at": { + "name": "access_token_expires_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "refresh_token_expires_at": { + "name": "refresh_token_expires_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "scope": { + "name": "scope", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "password": { + "name": "password", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "account_user_id_idx": { + "name": "account_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_account_on_account_id_provider_id": { + "name": "idx_account_on_account_id_provider_id", + "columns": [ + { + "expression": "account_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "provider_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "account_user_id_user_id_fk": { + "name": "account_user_id_user_id_fk", + "tableFrom": "account", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.api_key": { + "name": "api_key", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "key": { + "name": "key", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "key_hash": { + "name": "key_hash", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "type": { + "name": "type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'personal'" + }, + "last_used": { + "name": "last_used", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "api_key_workspace_type_idx": { + "name": "api_key_workspace_type_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "api_key_user_type_idx": { + "name": "api_key_user_type_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "api_key_key_hash_idx": { + "name": "api_key_key_hash_idx", + "columns": [ + { + "expression": "key_hash", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "api_key_user_id_user_id_fk": { + "name": "api_key_user_id_user_id_fk", + "tableFrom": "api_key", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "api_key_workspace_id_workspace_id_fk": { + "name": "api_key_workspace_id_workspace_id_fk", + "tableFrom": "api_key", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "api_key_created_by_user_id_fk": { + "name": "api_key_created_by_user_id_fk", + "tableFrom": "api_key", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "api_key_key_unique": { + "name": "api_key_key_unique", + "nullsNotDistinct": false, + "columns": ["key"] + } + }, + "policies": {}, + "checkConstraints": { + "workspace_type_check": { + "name": "workspace_type_check", + "value": "(type = 'workspace' AND workspace_id IS NOT NULL) OR (type = 'personal' AND workspace_id IS NULL)" + } + }, + "isRLSEnabled": false + }, + "public.async_jobs": { + "name": "async_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "type": { + "name": "type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "payload": { + "name": "payload", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "started_at": { + "name": "started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "run_at": { + "name": "run_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "attempts": { + "name": "attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "max_attempts": { + "name": "max_attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 3 + }, + "error": { + "name": "error", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "output": { + "name": "output", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "async_jobs_status_started_at_idx": { + "name": "async_jobs_status_started_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "async_jobs_status_completed_at_idx": { + "name": "async_jobs_status_completed_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "completed_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "async_jobs_schedule_pending_run_at_idx": { + "name": "async_jobs_schedule_pending_run_at_idx", + "columns": [ + { + "expression": "run_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"async_jobs\".\"type\" = 'schedule-execution' AND \"async_jobs\".\"status\" = 'pending'", + "concurrently": false, + "method": "btree", + "with": {} + }, + "async_jobs_schedule_processing_started_at_idx": { + "name": "async_jobs_schedule_processing_started_at_idx", + "columns": [ + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"async_jobs\".\"type\" = 'schedule-execution' AND \"async_jobs\".\"status\" = 'processing'", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_log": { + "name": "audit_log", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "actor_id": { + "name": "actor_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "resource_type": { + "name": "resource_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "resource_id": { + "name": "resource_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "actor_name": { + "name": "actor_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "actor_email": { + "name": "actor_email", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "resource_name": { + "name": "resource_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": false, + "default": "'{}'" + }, + "ip_address": { + "name": "ip_address", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "user_agent": { + "name": "user_agent", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "audit_log_workspace_created_idx": { + "name": "audit_log_workspace_created_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "audit_log_workspace_created_at_id_idx": { + "name": "audit_log_workspace_created_at_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "date_trunc('milliseconds', \"created_at\")", + "asc": true, + "isExpression": true, + "nulls": "last" + }, + { + "expression": "id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "audit_log_actor_created_idx": { + "name": "audit_log_actor_created_idx", + "columns": [ + { + "expression": "actor_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "audit_log_resource_idx": { + "name": "audit_log_resource_idx", + "columns": [ + { + "expression": "resource_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "resource_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "audit_log_action_idx": { + "name": "audit_log_action_idx", + "columns": [ + { + "expression": "action", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "audit_log_workspace_id_workspace_id_fk": { + "name": "audit_log_workspace_id_workspace_id_fk", + "tableFrom": "audit_log", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + }, + "audit_log_actor_id_user_id_fk": { + "name": "audit_log_actor_id_user_id_fk", + "tableFrom": "audit_log", + "tableTo": "user", + "columnsFrom": ["actor_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.chat": { + "name": "chat", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "identifier": { + "name": "identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "title": { + "name": "title", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "is_active": { + "name": "is_active", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "customizations": { + "name": "customizations", + "type": "json", + "primaryKey": false, + "notNull": false, + "default": "'{}'" + }, + "auth_type": { + "name": "auth_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'public'" + }, + "password": { + "name": "password", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "allowed_emails": { + "name": "allowed_emails", + "type": "json", + "primaryKey": false, + "notNull": false, + "default": "'[]'" + }, + "output_configs": { + "name": "output_configs", + "type": "json", + "primaryKey": false, + "notNull": false, + "default": "'[]'" + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "identifier_idx": { + "name": "identifier_idx", + "columns": [ + { + "expression": "identifier", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"chat\".\"archived_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "chat_archived_at_partial_idx": { + "name": "chat_archived_at_partial_idx", + "columns": [ + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"chat\".\"archived_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_chat_on_workflow_id_archived_at": { + "name": "idx_chat_on_workflow_id_archived_at", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "chat_workflow_id_workflow_id_fk": { + "name": "chat_workflow_id_workflow_id_fk", + "tableFrom": "chat", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "chat_user_id_user_id_fk": { + "name": "chat_user_id_user_id_fk", + "tableFrom": "chat", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.copilot_async_tool_calls": { + "name": "copilot_async_tool_calls", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "run_id": { + "name": "run_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "checkpoint_id": { + "name": "checkpoint_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "tool_call_id": { + "name": "tool_call_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "tool_name": { + "name": "tool_name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "args": { + "name": "args", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "status": { + "name": "status", + "type": "copilot_async_tool_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "result": { + "name": "result", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "error": { + "name": "error", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "claimed_at": { + "name": "claimed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "claimed_by": { + "name": "claimed_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "copilot_async_tool_calls_run_id_idx": { + "name": "copilot_async_tool_calls_run_id_idx", + "columns": [ + { + "expression": "run_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_async_tool_calls_checkpoint_id_idx": { + "name": "copilot_async_tool_calls_checkpoint_id_idx", + "columns": [ + { + "expression": "checkpoint_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_async_tool_calls_tool_call_id_idx": { + "name": "copilot_async_tool_calls_tool_call_id_idx", + "columns": [ + { + "expression": "tool_call_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_async_tool_calls_status_idx": { + "name": "copilot_async_tool_calls_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_async_tool_calls_run_status_idx": { + "name": "copilot_async_tool_calls_run_status_idx", + "columns": [ + { + "expression": "run_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_async_tool_calls_tool_call_id_unique": { + "name": "copilot_async_tool_calls_tool_call_id_unique", + "columns": [ + { + "expression": "tool_call_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "copilot_async_tool_calls_run_id_copilot_runs_id_fk": { + "name": "copilot_async_tool_calls_run_id_copilot_runs_id_fk", + "tableFrom": "copilot_async_tool_calls", + "tableTo": "copilot_runs", + "columnsFrom": ["run_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "copilot_async_tool_calls_checkpoint_id_copilot_run_checkpoints_id_fk": { + "name": "copilot_async_tool_calls_checkpoint_id_copilot_run_checkpoints_id_fk", + "tableFrom": "copilot_async_tool_calls", + "tableTo": "copilot_run_checkpoints", + "columnsFrom": ["checkpoint_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.copilot_chats": { + "name": "copilot_chats", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "type": { + "name": "type", + "type": "chat_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'copilot'" + }, + "title": { + "name": "title", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "messages": { + "name": "messages", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'" + }, + "model": { + "name": "model", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'claude-3-7-sonnet-latest'" + }, + "conversation_id": { + "name": "conversation_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "preview_yaml": { + "name": "preview_yaml", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "plan_artifact": { + "name": "plan_artifact", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "config": { + "name": "config", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "resources": { + "name": "resources", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'" + }, + "last_seen_at": { + "name": "last_seen_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "pinned": { + "name": "pinned", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "copilot_chats_user_id_idx": { + "name": "copilot_chats_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_chats_workflow_id_idx": { + "name": "copilot_chats_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_chats_user_workflow_idx": { + "name": "copilot_chats_user_workflow_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_chats_user_workspace_idx": { + "name": "copilot_chats_user_workspace_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_chats_created_at_idx": { + "name": "copilot_chats_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_chats_updated_at_idx": { + "name": "copilot_chats_updated_at_idx", + "columns": [ + { + "expression": "updated_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_chats_workspace_created_at_id_idx": { + "name": "copilot_chats_workspace_created_at_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "date_trunc('milliseconds', \"created_at\")", + "asc": true, + "isExpression": true, + "nulls": "last" + }, + { + "expression": "id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "copilot_chats_user_id_user_id_fk": { + "name": "copilot_chats_user_id_user_id_fk", + "tableFrom": "copilot_chats", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "copilot_chats_workflow_id_workflow_id_fk": { + "name": "copilot_chats_workflow_id_workflow_id_fk", + "tableFrom": "copilot_chats", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "copilot_chats_workspace_id_workspace_id_fk": { + "name": "copilot_chats_workspace_id_workspace_id_fk", + "tableFrom": "copilot_chats", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.copilot_feedback": { + "name": "copilot_feedback", + "schema": "", + "columns": { + "feedback_id": { + "name": "feedback_id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "chat_id": { + "name": "chat_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "user_query": { + "name": "user_query", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "agent_response": { + "name": "agent_response", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "is_positive": { + "name": "is_positive", + "type": "boolean", + "primaryKey": false, + "notNull": true + }, + "feedback": { + "name": "feedback", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "workflow_yaml": { + "name": "workflow_yaml", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "copilot_feedback_user_id_idx": { + "name": "copilot_feedback_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_feedback_chat_id_idx": { + "name": "copilot_feedback_chat_id_idx", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_feedback_user_chat_idx": { + "name": "copilot_feedback_user_chat_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_feedback_is_positive_idx": { + "name": "copilot_feedback_is_positive_idx", + "columns": [ + { + "expression": "is_positive", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_feedback_created_at_idx": { + "name": "copilot_feedback_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "copilot_feedback_user_id_user_id_fk": { + "name": "copilot_feedback_user_id_user_id_fk", + "tableFrom": "copilot_feedback", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "copilot_feedback_chat_id_copilot_chats_id_fk": { + "name": "copilot_feedback_chat_id_copilot_chats_id_fk", + "tableFrom": "copilot_feedback", + "tableTo": "copilot_chats", + "columnsFrom": ["chat_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.copilot_messages": { + "name": "copilot_messages", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "chat_id": { + "name": "chat_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "message_id": { + "name": "message_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "role": { + "name": "role", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "content": { + "name": "content", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "stream_id": { + "name": "stream_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "parent_message_id": { + "name": "parent_message_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "model": { + "name": "model", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tokens_in": { + "name": "tokens_in", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "tokens_out": { + "name": "tokens_out", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "seq": { + "name": "seq", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "copilot_messages_chat_message_unique": { + "name": "copilot_messages_chat_message_unique", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "message_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_messages_chat_created_at_idx": { + "name": "copilot_messages_chat_created_at_idx", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"copilot_messages\".\"deleted_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_messages_chat_seq_idx": { + "name": "copilot_messages_chat_seq_idx", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "seq", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"copilot_messages\".\"deleted_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_messages_chat_stream_idx": { + "name": "copilot_messages_chat_stream_idx", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "stream_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"copilot_messages\".\"stream_id\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "copilot_messages_chat_id_copilot_chats_id_fk": { + "name": "copilot_messages_chat_id_copilot_chats_id_fk", + "tableFrom": "copilot_messages", + "tableTo": "copilot_chats", + "columnsFrom": ["chat_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.copilot_run_checkpoints": { + "name": "copilot_run_checkpoints", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "run_id": { + "name": "run_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "pending_tool_call_id": { + "name": "pending_tool_call_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "conversation_snapshot": { + "name": "conversation_snapshot", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "agent_state": { + "name": "agent_state", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "provider_request": { + "name": "provider_request", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "copilot_run_checkpoints_run_id_idx": { + "name": "copilot_run_checkpoints_run_id_idx", + "columns": [ + { + "expression": "run_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_run_checkpoints_pending_tool_call_id_idx": { + "name": "copilot_run_checkpoints_pending_tool_call_id_idx", + "columns": [ + { + "expression": "pending_tool_call_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_run_checkpoints_run_pending_tool_unique": { + "name": "copilot_run_checkpoints_run_pending_tool_unique", + "columns": [ + { + "expression": "run_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "pending_tool_call_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "copilot_run_checkpoints_run_id_copilot_runs_id_fk": { + "name": "copilot_run_checkpoints_run_id_copilot_runs_id_fk", + "tableFrom": "copilot_run_checkpoints", + "tableTo": "copilot_runs", + "columnsFrom": ["run_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.copilot_runs": { + "name": "copilot_runs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "execution_id": { + "name": "execution_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "parent_run_id": { + "name": "parent_run_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "chat_id": { + "name": "chat_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "stream_id": { + "name": "stream_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "agent": { + "name": "agent", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "model": { + "name": "model", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "provider": { + "name": "provider", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "copilot_run_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "request_context": { + "name": "request_context", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "started_at": { + "name": "started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "error": { + "name": "error", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "copilot_runs_execution_id_idx": { + "name": "copilot_runs_execution_id_idx", + "columns": [ + { + "expression": "execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_runs_parent_run_id_idx": { + "name": "copilot_runs_parent_run_id_idx", + "columns": [ + { + "expression": "parent_run_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_runs_chat_id_idx": { + "name": "copilot_runs_chat_id_idx", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_runs_user_id_idx": { + "name": "copilot_runs_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_runs_workflow_id_idx": { + "name": "copilot_runs_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_runs_workspace_id_idx": { + "name": "copilot_runs_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_runs_status_idx": { + "name": "copilot_runs_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_runs_chat_execution_idx": { + "name": "copilot_runs_chat_execution_idx", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_runs_execution_started_at_idx": { + "name": "copilot_runs_execution_started_at_idx", + "columns": [ + { + "expression": "execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_runs_workspace_completed_at_id_idx": { + "name": "copilot_runs_workspace_completed_at_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "date_trunc('milliseconds', \"completed_at\")", + "asc": true, + "isExpression": true, + "nulls": "last" + }, + { + "expression": "id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_runs_stream_id_unique": { + "name": "copilot_runs_stream_id_unique", + "columns": [ + { + "expression": "stream_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "copilot_runs_chat_id_copilot_chats_id_fk": { + "name": "copilot_runs_chat_id_copilot_chats_id_fk", + "tableFrom": "copilot_runs", + "tableTo": "copilot_chats", + "columnsFrom": ["chat_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "copilot_runs_user_id_user_id_fk": { + "name": "copilot_runs_user_id_user_id_fk", + "tableFrom": "copilot_runs", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "copilot_runs_workflow_id_workflow_id_fk": { + "name": "copilot_runs_workflow_id_workflow_id_fk", + "tableFrom": "copilot_runs", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "copilot_runs_workspace_id_workspace_id_fk": { + "name": "copilot_runs_workspace_id_workspace_id_fk", + "tableFrom": "copilot_runs", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.copilot_workflow_read_hashes": { + "name": "copilot_workflow_read_hashes", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "chat_id": { + "name": "chat_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "hash": { + "name": "hash", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "copilot_workflow_read_hashes_chat_id_idx": { + "name": "copilot_workflow_read_hashes_chat_id_idx", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_workflow_read_hashes_workflow_id_idx": { + "name": "copilot_workflow_read_hashes_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "copilot_workflow_read_hashes_chat_workflow_unique": { + "name": "copilot_workflow_read_hashes_chat_workflow_unique", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "copilot_workflow_read_hashes_chat_id_copilot_chats_id_fk": { + "name": "copilot_workflow_read_hashes_chat_id_copilot_chats_id_fk", + "tableFrom": "copilot_workflow_read_hashes", + "tableTo": "copilot_chats", + "columnsFrom": ["chat_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "copilot_workflow_read_hashes_workflow_id_workflow_id_fk": { + "name": "copilot_workflow_read_hashes_workflow_id_workflow_id_fk", + "tableFrom": "copilot_workflow_read_hashes", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.credential": { + "name": "credential", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "type": { + "name": "type", + "type": "credential_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "provider_id": { + "name": "provider_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "account_id": { + "name": "account_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "env_key": { + "name": "env_key", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "env_owner_user_id": { + "name": "env_owner_user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "encrypted_service_account_key": { + "name": "encrypted_service_account_key", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "credential_workspace_id_idx": { + "name": "credential_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_type_idx": { + "name": "credential_type_idx", + "columns": [ + { + "expression": "type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_provider_id_idx": { + "name": "credential_provider_id_idx", + "columns": [ + { + "expression": "provider_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_account_id_idx": { + "name": "credential_account_id_idx", + "columns": [ + { + "expression": "account_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_env_owner_user_id_idx": { + "name": "credential_env_owner_user_id_idx", + "columns": [ + { + "expression": "env_owner_user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_workspace_account_unique": { + "name": "credential_workspace_account_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "account_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "account_id IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_workspace_env_unique": { + "name": "credential_workspace_env_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "env_key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "type = 'env_workspace'", + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_workspace_personal_env_unique": { + "name": "credential_workspace_personal_env_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "env_key", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "env_owner_user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "type = 'env_personal'", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "credential_workspace_id_workspace_id_fk": { + "name": "credential_workspace_id_workspace_id_fk", + "tableFrom": "credential", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "credential_account_id_account_id_fk": { + "name": "credential_account_id_account_id_fk", + "tableFrom": "credential", + "tableTo": "account", + "columnsFrom": ["account_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "credential_env_owner_user_id_user_id_fk": { + "name": "credential_env_owner_user_id_user_id_fk", + "tableFrom": "credential", + "tableTo": "user", + "columnsFrom": ["env_owner_user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "credential_created_by_user_id_fk": { + "name": "credential_created_by_user_id_fk", + "tableFrom": "credential", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "credential_oauth_source_check": { + "name": "credential_oauth_source_check", + "value": "(type <> 'oauth') OR (account_id IS NOT NULL AND provider_id IS NOT NULL)" + }, + "credential_workspace_env_source_check": { + "name": "credential_workspace_env_source_check", + "value": "(type <> 'env_workspace') OR (env_key IS NOT NULL AND env_owner_user_id IS NULL)" + }, + "credential_personal_env_source_check": { + "name": "credential_personal_env_source_check", + "value": "(type <> 'env_personal') OR (env_key IS NOT NULL AND env_owner_user_id IS NOT NULL)" + } + }, + "isRLSEnabled": false + }, + "public.credential_member": { + "name": "credential_member", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "credential_id": { + "name": "credential_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "role": { + "name": "role", + "type": "credential_member_role", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'member'" + }, + "status": { + "name": "status", + "type": "credential_member_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "joined_at": { + "name": "joined_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "invited_by": { + "name": "invited_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "credential_member_user_id_idx": { + "name": "credential_member_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_member_role_idx": { + "name": "credential_member_role_idx", + "columns": [ + { + "expression": "role", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_member_status_idx": { + "name": "credential_member_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_member_unique": { + "name": "credential_member_unique", + "columns": [ + { + "expression": "credential_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "credential_member_credential_id_credential_id_fk": { + "name": "credential_member_credential_id_credential_id_fk", + "tableFrom": "credential_member", + "tableTo": "credential", + "columnsFrom": ["credential_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "credential_member_user_id_user_id_fk": { + "name": "credential_member_user_id_user_id_fk", + "tableFrom": "credential_member", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "credential_member_invited_by_user_id_fk": { + "name": "credential_member_invited_by_user_id_fk", + "tableFrom": "credential_member", + "tableTo": "user", + "columnsFrom": ["invited_by"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.credential_set": { + "name": "credential_set", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "organization_id": { + "name": "organization_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "provider_id": { + "name": "provider_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "credential_set_created_by_idx": { + "name": "credential_set_created_by_idx", + "columns": [ + { + "expression": "created_by", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_set_org_name_unique": { + "name": "credential_set_org_name_unique", + "columns": [ + { + "expression": "organization_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_set_provider_id_idx": { + "name": "credential_set_provider_id_idx", + "columns": [ + { + "expression": "provider_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "credential_set_organization_id_organization_id_fk": { + "name": "credential_set_organization_id_organization_id_fk", + "tableFrom": "credential_set", + "tableTo": "organization", + "columnsFrom": ["organization_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "credential_set_created_by_user_id_fk": { + "name": "credential_set_created_by_user_id_fk", + "tableFrom": "credential_set", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.credential_set_invitation": { + "name": "credential_set_invitation", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "credential_set_id": { + "name": "credential_set_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "token": { + "name": "token", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "invited_by": { + "name": "invited_by", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "credential_set_invitation_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "accepted_at": { + "name": "accepted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "accepted_by_user_id": { + "name": "accepted_by_user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "credential_set_invitation_set_id_idx": { + "name": "credential_set_invitation_set_id_idx", + "columns": [ + { + "expression": "credential_set_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_set_invitation_token_idx": { + "name": "credential_set_invitation_token_idx", + "columns": [ + { + "expression": "token", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_set_invitation_status_idx": { + "name": "credential_set_invitation_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_set_invitation_expires_at_idx": { + "name": "credential_set_invitation_expires_at_idx", + "columns": [ + { + "expression": "expires_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "credential_set_invitation_credential_set_id_credential_set_id_fk": { + "name": "credential_set_invitation_credential_set_id_credential_set_id_fk", + "tableFrom": "credential_set_invitation", + "tableTo": "credential_set", + "columnsFrom": ["credential_set_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "credential_set_invitation_invited_by_user_id_fk": { + "name": "credential_set_invitation_invited_by_user_id_fk", + "tableFrom": "credential_set_invitation", + "tableTo": "user", + "columnsFrom": ["invited_by"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "credential_set_invitation_accepted_by_user_id_user_id_fk": { + "name": "credential_set_invitation_accepted_by_user_id_user_id_fk", + "tableFrom": "credential_set_invitation", + "tableTo": "user", + "columnsFrom": ["accepted_by_user_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "credential_set_invitation_token_unique": { + "name": "credential_set_invitation_token_unique", + "nullsNotDistinct": false, + "columns": ["token"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.credential_set_member": { + "name": "credential_set_member", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "credential_set_id": { + "name": "credential_set_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "credential_set_member_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "joined_at": { + "name": "joined_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "invited_by": { + "name": "invited_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "credential_set_member_user_id_idx": { + "name": "credential_set_member_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_set_member_unique": { + "name": "credential_set_member_unique", + "columns": [ + { + "expression": "credential_set_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "credential_set_member_status_idx": { + "name": "credential_set_member_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "credential_set_member_credential_set_id_credential_set_id_fk": { + "name": "credential_set_member_credential_set_id_credential_set_id_fk", + "tableFrom": "credential_set_member", + "tableTo": "credential_set", + "columnsFrom": ["credential_set_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "credential_set_member_user_id_user_id_fk": { + "name": "credential_set_member_user_id_user_id_fk", + "tableFrom": "credential_set_member", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "credential_set_member_invited_by_user_id_fk": { + "name": "credential_set_member_invited_by_user_id_fk", + "tableFrom": "credential_set_member", + "tableTo": "user", + "columnsFrom": ["invited_by"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custom_tools": { + "name": "custom_tools", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "title": { + "name": "title", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "schema": { + "name": "schema", + "type": "json", + "primaryKey": false, + "notNull": true + }, + "code": { + "name": "code", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "custom_tools_workspace_id_idx": { + "name": "custom_tools_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "custom_tools_workspace_title_unique": { + "name": "custom_tools_workspace_title_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "title", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "custom_tools_workspace_id_workspace_id_fk": { + "name": "custom_tools_workspace_id_workspace_id_fk", + "tableFrom": "custom_tools", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "custom_tools_user_id_user_id_fk": { + "name": "custom_tools_user_id_user_id_fk", + "tableFrom": "custom_tools", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.data_drain_runs": { + "name": "data_drain_runs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "drain_id": { + "name": "drain_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "data_drain_run_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "trigger": { + "name": "trigger", + "type": "data_drain_run_trigger", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "started_at": { + "name": "started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "finished_at": { + "name": "finished_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "rows_exported": { + "name": "rows_exported", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "bytes_written": { + "name": "bytes_written", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "cursor_before": { + "name": "cursor_before", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "cursor_after": { + "name": "cursor_after", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "error": { + "name": "error", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "locators": { + "name": "locators", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + } + }, + "indexes": { + "data_drain_runs_drain_started_idx": { + "name": "data_drain_runs_drain_started_idx", + "columns": [ + { + "expression": "drain_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "data_drain_runs_drain_id_data_drains_id_fk": { + "name": "data_drain_runs_drain_id_data_drains_id_fk", + "tableFrom": "data_drain_runs", + "tableTo": "data_drains", + "columnsFrom": ["drain_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.data_drains": { + "name": "data_drains", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "organization_id": { + "name": "organization_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source": { + "name": "source", + "type": "data_drain_source", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "destination_type": { + "name": "destination_type", + "type": "data_drain_destination", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "destination_config": { + "name": "destination_config", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "destination_credentials": { + "name": "destination_credentials", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "schedule_cadence": { + "name": "schedule_cadence", + "type": "data_drain_cadence", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "enabled": { + "name": "enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "cursor": { + "name": "cursor", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "last_run_at": { + "name": "last_run_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "last_success_at": { + "name": "last_success_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "data_drains_org_idx": { + "name": "data_drains_org_idx", + "columns": [ + { + "expression": "organization_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "data_drains_due_idx": { + "name": "data_drains_due_idx", + "columns": [ + { + "expression": "enabled", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "last_run_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "data_drains_org_name_unique": { + "name": "data_drains_org_name_unique", + "columns": [ + { + "expression": "organization_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "data_drains_organization_id_organization_id_fk": { + "name": "data_drains_organization_id_organization_id_fk", + "tableFrom": "data_drains", + "tableTo": "organization", + "columnsFrom": ["organization_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "data_drains_created_by_user_id_fk": { + "name": "data_drains_created_by_user_id_fk", + "tableFrom": "data_drains", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.docs_embeddings": { + "name": "docs_embeddings", + "schema": "", + "columns": { + "chunk_id": { + "name": "chunk_id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "chunk_text": { + "name": "chunk_text", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source_document": { + "name": "source_document", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source_link": { + "name": "source_link", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "header_text": { + "name": "header_text", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "header_level": { + "name": "header_level", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "token_count": { + "name": "token_count", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "embedding": { + "name": "embedding", + "type": "vector(1536)", + "primaryKey": false, + "notNull": true + }, + "embedding_model": { + "name": "embedding_model", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'text-embedding-3-small'" + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "chunk_text_tsv": { + "name": "chunk_text_tsv", + "type": "tsvector", + "primaryKey": false, + "notNull": false, + "generated": { + "as": "to_tsvector('english', \"docs_embeddings\".\"chunk_text\")", + "type": "stored" + } + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "docs_emb_source_document_idx": { + "name": "docs_emb_source_document_idx", + "columns": [ + { + "expression": "source_document", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "docs_emb_header_level_idx": { + "name": "docs_emb_header_level_idx", + "columns": [ + { + "expression": "header_level", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "docs_emb_source_header_idx": { + "name": "docs_emb_source_header_idx", + "columns": [ + { + "expression": "source_document", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "header_level", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "docs_emb_model_idx": { + "name": "docs_emb_model_idx", + "columns": [ + { + "expression": "embedding_model", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "docs_emb_created_at_idx": { + "name": "docs_emb_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "docs_embedding_vector_hnsw_idx": { + "name": "docs_embedding_vector_hnsw_idx", + "columns": [ + { + "expression": "embedding", + "isExpression": false, + "asc": true, + "nulls": "last", + "opclass": "vector_cosine_ops" + } + ], + "isUnique": false, + "concurrently": false, + "method": "hnsw", + "with": { + "m": 16, + "ef_construction": 64 + } + }, + "docs_emb_metadata_gin_idx": { + "name": "docs_emb_metadata_gin_idx", + "columns": [ + { + "expression": "metadata", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "gin", + "with": {} + }, + "docs_emb_chunk_text_fts_idx": { + "name": "docs_emb_chunk_text_fts_idx", + "columns": [ + { + "expression": "chunk_text_tsv", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "gin", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "docs_embedding_not_null_check": { + "name": "docs_embedding_not_null_check", + "value": "\"embedding\" IS NOT NULL" + }, + "docs_header_level_check": { + "name": "docs_header_level_check", + "value": "\"header_level\" >= 1 AND \"header_level\" <= 6" + } + }, + "isRLSEnabled": false + }, + "public.document": { + "name": "document", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "knowledge_base_id": { + "name": "knowledge_base_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "file_url": { + "name": "file_url", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_key": { + "name": "storage_key", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "file_size": { + "name": "file_size", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "chunk_count": { + "name": "chunk_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "token_count": { + "name": "token_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "character_count": { + "name": "character_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "processing_status": { + "name": "processing_status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "processing_started_at": { + "name": "processing_started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "processing_completed_at": { + "name": "processing_completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "processing_error": { + "name": "processing_error", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "enabled": { + "name": "enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "user_excluded": { + "name": "user_excluded", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "tag1": { + "name": "tag1", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag2": { + "name": "tag2", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag3": { + "name": "tag3", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag4": { + "name": "tag4", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag5": { + "name": "tag5", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag6": { + "name": "tag6", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag7": { + "name": "tag7", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "number1": { + "name": "number1", + "type": "double precision", + "primaryKey": false, + "notNull": false + }, + "number2": { + "name": "number2", + "type": "double precision", + "primaryKey": false, + "notNull": false + }, + "number3": { + "name": "number3", + "type": "double precision", + "primaryKey": false, + "notNull": false + }, + "number4": { + "name": "number4", + "type": "double precision", + "primaryKey": false, + "notNull": false + }, + "number5": { + "name": "number5", + "type": "double precision", + "primaryKey": false, + "notNull": false + }, + "date1": { + "name": "date1", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "date2": { + "name": "date2", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "boolean1": { + "name": "boolean1", + "type": "boolean", + "primaryKey": false, + "notNull": false + }, + "boolean2": { + "name": "boolean2", + "type": "boolean", + "primaryKey": false, + "notNull": false + }, + "boolean3": { + "name": "boolean3", + "type": "boolean", + "primaryKey": false, + "notNull": false + }, + "connector_id": { + "name": "connector_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "external_id": { + "name": "external_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "content_hash": { + "name": "content_hash", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_url": { + "name": "source_url", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "uploaded_at": { + "name": "uploaded_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "doc_kb_id_idx": { + "name": "doc_kb_id_idx", + "columns": [ + { + "expression": "knowledge_base_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_filename_idx": { + "name": "doc_filename_idx", + "columns": [ + { + "expression": "filename", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_processing_status_idx": { + "name": "doc_processing_status_idx", + "columns": [ + { + "expression": "knowledge_base_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "processing_status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_connector_external_id_idx": { + "name": "doc_connector_external_id_idx", + "columns": [ + { + "expression": "connector_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "external_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"document\".\"deleted_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_connector_id_idx": { + "name": "doc_connector_id_idx", + "columns": [ + { + "expression": "connector_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_storage_key_idx": { + "name": "doc_storage_key_idx", + "columns": [ + { + "expression": "storage_key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"document\".\"storage_key\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_archived_at_partial_idx": { + "name": "doc_archived_at_partial_idx", + "columns": [ + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"document\".\"archived_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_deleted_at_partial_idx": { + "name": "doc_deleted_at_partial_idx", + "columns": [ + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"document\".\"deleted_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_tag1_idx": { + "name": "doc_tag1_idx", + "columns": [ + { + "expression": "tag1", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_tag2_idx": { + "name": "doc_tag2_idx", + "columns": [ + { + "expression": "tag2", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_tag3_idx": { + "name": "doc_tag3_idx", + "columns": [ + { + "expression": "tag3", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_tag4_idx": { + "name": "doc_tag4_idx", + "columns": [ + { + "expression": "tag4", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_tag5_idx": { + "name": "doc_tag5_idx", + "columns": [ + { + "expression": "tag5", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_tag6_idx": { + "name": "doc_tag6_idx", + "columns": [ + { + "expression": "tag6", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_tag7_idx": { + "name": "doc_tag7_idx", + "columns": [ + { + "expression": "tag7", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_number1_idx": { + "name": "doc_number1_idx", + "columns": [ + { + "expression": "number1", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_number2_idx": { + "name": "doc_number2_idx", + "columns": [ + { + "expression": "number2", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_number3_idx": { + "name": "doc_number3_idx", + "columns": [ + { + "expression": "number3", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_number4_idx": { + "name": "doc_number4_idx", + "columns": [ + { + "expression": "number4", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_number5_idx": { + "name": "doc_number5_idx", + "columns": [ + { + "expression": "number5", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_date1_idx": { + "name": "doc_date1_idx", + "columns": [ + { + "expression": "date1", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_date2_idx": { + "name": "doc_date2_idx", + "columns": [ + { + "expression": "date2", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_boolean1_idx": { + "name": "doc_boolean1_idx", + "columns": [ + { + "expression": "boolean1", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_boolean2_idx": { + "name": "doc_boolean2_idx", + "columns": [ + { + "expression": "boolean2", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "doc_boolean3_idx": { + "name": "doc_boolean3_idx", + "columns": [ + { + "expression": "boolean3", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "document_knowledge_base_id_knowledge_base_id_fk": { + "name": "document_knowledge_base_id_knowledge_base_id_fk", + "tableFrom": "document", + "tableTo": "knowledge_base", + "columnsFrom": ["knowledge_base_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "document_connector_id_knowledge_connector_id_fk": { + "name": "document_connector_id_knowledge_connector_id_fk", + "tableFrom": "document", + "tableTo": "knowledge_connector", + "columnsFrom": ["connector_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.embedding": { + "name": "embedding", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "knowledge_base_id": { + "name": "knowledge_base_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "document_id": { + "name": "document_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "chunk_index": { + "name": "chunk_index", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "chunk_hash": { + "name": "chunk_hash", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "content": { + "name": "content", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "content_length": { + "name": "content_length", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "token_count": { + "name": "token_count", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "embedding": { + "name": "embedding", + "type": "vector(1536)", + "primaryKey": false, + "notNull": false + }, + "embedding_model": { + "name": "embedding_model", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'text-embedding-3-small'" + }, + "start_offset": { + "name": "start_offset", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "end_offset": { + "name": "end_offset", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "tag1": { + "name": "tag1", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag2": { + "name": "tag2", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag3": { + "name": "tag3", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag4": { + "name": "tag4", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag5": { + "name": "tag5", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag6": { + "name": "tag6", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tag7": { + "name": "tag7", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "number1": { + "name": "number1", + "type": "double precision", + "primaryKey": false, + "notNull": false + }, + "number2": { + "name": "number2", + "type": "double precision", + "primaryKey": false, + "notNull": false + }, + "number3": { + "name": "number3", + "type": "double precision", + "primaryKey": false, + "notNull": false + }, + "number4": { + "name": "number4", + "type": "double precision", + "primaryKey": false, + "notNull": false + }, + "number5": { + "name": "number5", + "type": "double precision", + "primaryKey": false, + "notNull": false + }, + "date1": { + "name": "date1", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "date2": { + "name": "date2", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "boolean1": { + "name": "boolean1", + "type": "boolean", + "primaryKey": false, + "notNull": false + }, + "boolean2": { + "name": "boolean2", + "type": "boolean", + "primaryKey": false, + "notNull": false + }, + "boolean3": { + "name": "boolean3", + "type": "boolean", + "primaryKey": false, + "notNull": false + }, + "enabled": { + "name": "enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "content_tsv": { + "name": "content_tsv", + "type": "tsvector", + "primaryKey": false, + "notNull": false, + "generated": { + "as": "to_tsvector('english', \"embedding\".\"content\")", + "type": "stored" + } + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "emb_kb_id_idx": { + "name": "emb_kb_id_idx", + "columns": [ + { + "expression": "knowledge_base_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_doc_id_idx": { + "name": "emb_doc_id_idx", + "columns": [ + { + "expression": "document_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_doc_chunk_idx": { + "name": "emb_doc_chunk_idx", + "columns": [ + { + "expression": "document_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "chunk_index", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_kb_model_idx": { + "name": "emb_kb_model_idx", + "columns": [ + { + "expression": "knowledge_base_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "embedding_model", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_kb_enabled_idx": { + "name": "emb_kb_enabled_idx", + "columns": [ + { + "expression": "knowledge_base_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "enabled", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_doc_enabled_idx": { + "name": "emb_doc_enabled_idx", + "columns": [ + { + "expression": "document_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "enabled", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "embedding_vector_hnsw_idx": { + "name": "embedding_vector_hnsw_idx", + "columns": [ + { + "expression": "embedding", + "isExpression": false, + "asc": true, + "nulls": "last", + "opclass": "vector_cosine_ops" + } + ], + "isUnique": false, + "concurrently": false, + "method": "hnsw", + "with": { + "m": 16, + "ef_construction": 64 + } + }, + "emb_tag1_idx": { + "name": "emb_tag1_idx", + "columns": [ + { + "expression": "tag1", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_tag2_idx": { + "name": "emb_tag2_idx", + "columns": [ + { + "expression": "tag2", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_tag3_idx": { + "name": "emb_tag3_idx", + "columns": [ + { + "expression": "tag3", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_tag4_idx": { + "name": "emb_tag4_idx", + "columns": [ + { + "expression": "tag4", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_tag5_idx": { + "name": "emb_tag5_idx", + "columns": [ + { + "expression": "tag5", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_tag6_idx": { + "name": "emb_tag6_idx", + "columns": [ + { + "expression": "tag6", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_tag7_idx": { + "name": "emb_tag7_idx", + "columns": [ + { + "expression": "tag7", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_number1_idx": { + "name": "emb_number1_idx", + "columns": [ + { + "expression": "number1", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_number2_idx": { + "name": "emb_number2_idx", + "columns": [ + { + "expression": "number2", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_number3_idx": { + "name": "emb_number3_idx", + "columns": [ + { + "expression": "number3", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_number4_idx": { + "name": "emb_number4_idx", + "columns": [ + { + "expression": "number4", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_number5_idx": { + "name": "emb_number5_idx", + "columns": [ + { + "expression": "number5", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_date1_idx": { + "name": "emb_date1_idx", + "columns": [ + { + "expression": "date1", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_date2_idx": { + "name": "emb_date2_idx", + "columns": [ + { + "expression": "date2", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_boolean1_idx": { + "name": "emb_boolean1_idx", + "columns": [ + { + "expression": "boolean1", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_boolean2_idx": { + "name": "emb_boolean2_idx", + "columns": [ + { + "expression": "boolean2", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_boolean3_idx": { + "name": "emb_boolean3_idx", + "columns": [ + { + "expression": "boolean3", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "emb_content_fts_idx": { + "name": "emb_content_fts_idx", + "columns": [ + { + "expression": "content_tsv", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "gin", + "with": {} + } + }, + "foreignKeys": { + "embedding_knowledge_base_id_knowledge_base_id_fk": { + "name": "embedding_knowledge_base_id_knowledge_base_id_fk", + "tableFrom": "embedding", + "tableTo": "knowledge_base", + "columnsFrom": ["knowledge_base_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "embedding_document_id_document_id_fk": { + "name": "embedding_document_id_document_id_fk", + "tableFrom": "embedding", + "tableTo": "document", + "columnsFrom": ["document_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "embedding_not_null_check": { + "name": "embedding_not_null_check", + "value": "\"embedding\" IS NOT NULL" + } + }, + "isRLSEnabled": false + }, + "public.environment": { + "name": "environment", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "variables": { + "name": "variables", + "type": "json", + "primaryKey": false, + "notNull": true + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "environment_user_id_user_id_fk": { + "name": "environment_user_id_user_id_fk", + "tableFrom": "environment", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "environment_user_id_unique": { + "name": "environment_user_id_unique", + "nullsNotDistinct": false, + "columns": ["user_id"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.execution_large_value_dependencies": { + "name": "execution_large_value_dependencies", + "schema": "", + "columns": { + "parent_key": { + "name": "parent_key", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "child_key": { + "name": "child_key", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "execution_large_value_dependencies_workspace_parent_key_idx": { + "name": "execution_large_value_dependencies_workspace_parent_key_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "parent_key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "execution_large_value_dependencies_workspace_child_key_idx": { + "name": "execution_large_value_dependencies_workspace_child_key_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "child_key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "execution_large_value_dependencies_workspace_id_workspace_id_fk": { + "name": "execution_large_value_dependencies_workspace_id_workspace_id_fk", + "tableFrom": "execution_large_value_dependencies", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "execution_large_value_dependencies_parent_key_child_key_pk": { + "name": "execution_large_value_dependencies_parent_key_child_key_pk", + "columns": ["parent_key", "child_key"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.execution_large_value_references": { + "name": "execution_large_value_references", + "schema": "", + "columns": { + "key": { + "name": "key", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "execution_id": { + "name": "execution_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source": { + "name": "source", + "type": "execution_large_value_reference_source", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "execution_large_value_references_workspace_execution_source_idx": { + "name": "execution_large_value_references_workspace_execution_source_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "source", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "execution_large_value_references_workspace_id_workspace_id_fk": { + "name": "execution_large_value_references_workspace_id_workspace_id_fk", + "tableFrom": "execution_large_value_references", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "execution_large_value_references_workflow_id_workflow_id_fk": { + "name": "execution_large_value_references_workflow_id_workflow_id_fk", + "tableFrom": "execution_large_value_references", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "execution_large_value_references_key_execution_id_source_pk": { + "name": "execution_large_value_references_key_execution_id_source_pk", + "columns": ["key", "execution_id", "source"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.execution_large_values": { + "name": "execution_large_values", + "schema": "", + "columns": { + "key": { + "name": "key", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "owner_execution_id": { + "name": "owner_execution_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size": { + "name": "size", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "execution_large_values_owner_execution_id_idx": { + "name": "execution_large_values_owner_execution_id_idx", + "columns": [ + { + "expression": "owner_execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "execution_large_values_cleanup_idx": { + "name": "execution_large_values_cleanup_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"execution_large_values\".\"deleted_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "execution_large_values_tombstone_cleanup_idx": { + "name": "execution_large_values_tombstone_cleanup_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"execution_large_values\".\"deleted_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "execution_large_values_workspace_id_workspace_id_fk": { + "name": "execution_large_values_workspace_id_workspace_id_fk", + "tableFrom": "execution_large_values", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "execution_large_values_workflow_id_workflow_id_fk": { + "name": "execution_large_values_workflow_id_workflow_id_fk", + "tableFrom": "execution_large_values", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.form": { + "name": "form", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "identifier": { + "name": "identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "title": { + "name": "title", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "is_active": { + "name": "is_active", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "customizations": { + "name": "customizations", + "type": "json", + "primaryKey": false, + "notNull": false, + "default": "'{}'" + }, + "auth_type": { + "name": "auth_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'public'" + }, + "password": { + "name": "password", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "allowed_emails": { + "name": "allowed_emails", + "type": "json", + "primaryKey": false, + "notNull": false, + "default": "'[]'" + }, + "show_branding": { + "name": "show_branding", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "form_identifier_idx": { + "name": "form_identifier_idx", + "columns": [ + { + "expression": "identifier", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"form\".\"archived_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "form_workflow_id_idx": { + "name": "form_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "form_user_id_idx": { + "name": "form_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "form_archived_at_partial_idx": { + "name": "form_archived_at_partial_idx", + "columns": [ + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"form\".\"archived_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "form_workflow_id_workflow_id_fk": { + "name": "form_workflow_id_workflow_id_fk", + "tableFrom": "form", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "form_user_id_user_id_fk": { + "name": "form_user_id_user_id_fk", + "tableFrom": "form", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.idempotency_key": { + "name": "idempotency_key", + "schema": "", + "columns": { + "key": { + "name": "key", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "result": { + "name": "result", + "type": "json", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "idempotency_key_created_at_idx": { + "name": "idempotency_key_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.invitation": { + "name": "invitation", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "kind": { + "name": "kind", + "type": "invitation_kind", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'organization'" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "inviter_id": { + "name": "inviter_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "organization_id": { + "name": "organization_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "membership_intent": { + "name": "membership_intent", + "type": "invitation_membership_intent", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'internal'" + }, + "role": { + "name": "role", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "invitation_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "token": { + "name": "token", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "invitation_email_idx": { + "name": "invitation_email_idx", + "columns": [ + { + "expression": "email", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "invitation_organization_id_idx": { + "name": "invitation_organization_id_idx", + "columns": [ + { + "expression": "organization_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "invitation_status_idx": { + "name": "invitation_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "invitation_pending_email_org_unique": { + "name": "invitation_pending_email_org_unique", + "columns": [ + { + "expression": "email", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "organization_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"invitation\".\"status\" = 'pending' AND \"invitation\".\"organization_id\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "invitation_inviter_id_user_id_fk": { + "name": "invitation_inviter_id_user_id_fk", + "tableFrom": "invitation", + "tableTo": "user", + "columnsFrom": ["inviter_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "invitation_organization_id_organization_id_fk": { + "name": "invitation_organization_id_organization_id_fk", + "tableFrom": "invitation", + "tableTo": "organization", + "columnsFrom": ["organization_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "invitation_token_unique": { + "name": "invitation_token_unique", + "nullsNotDistinct": false, + "columns": ["token"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.invitation_workspace_grant": { + "name": "invitation_workspace_grant", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "invitation_id": { + "name": "invitation_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "permission": { + "name": "permission", + "type": "permission_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "invitation_workspace_grant_unique": { + "name": "invitation_workspace_grant_unique", + "columns": [ + { + "expression": "invitation_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "invitation_workspace_grant_workspace_id_idx": { + "name": "invitation_workspace_grant_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "invitation_workspace_grant_invitation_id_invitation_id_fk": { + "name": "invitation_workspace_grant_invitation_id_invitation_id_fk", + "tableFrom": "invitation_workspace_grant", + "tableTo": "invitation", + "columnsFrom": ["invitation_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "invitation_workspace_grant_workspace_id_workspace_id_fk": { + "name": "invitation_workspace_grant_workspace_id_workspace_id_fk", + "tableFrom": "invitation_workspace_grant", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.job_execution_logs": { + "name": "job_execution_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "schedule_id": { + "name": "schedule_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "execution_id": { + "name": "execution_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "level": { + "name": "level", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'running'" + }, + "trigger": { + "name": "trigger", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "started_at": { + "name": "started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "ended_at": { + "name": "ended_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "total_duration_ms": { + "name": "total_duration_ms", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "execution_data": { + "name": "execution_data", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "cost": { + "name": "cost", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "job_execution_logs_schedule_id_idx": { + "name": "job_execution_logs_schedule_id_idx", + "columns": [ + { + "expression": "schedule_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "job_execution_logs_workspace_started_at_idx": { + "name": "job_execution_logs_workspace_started_at_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "job_execution_logs_workspace_ended_at_id_idx": { + "name": "job_execution_logs_workspace_ended_at_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "date_trunc('milliseconds', \"ended_at\")", + "asc": true, + "isExpression": true, + "nulls": "last" + }, + { + "expression": "id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "job_execution_logs_execution_id_unique": { + "name": "job_execution_logs_execution_id_unique", + "columns": [ + { + "expression": "execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "job_execution_logs_trigger_idx": { + "name": "job_execution_logs_trigger_idx", + "columns": [ + { + "expression": "trigger", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "job_execution_logs_schedule_id_workflow_schedule_id_fk": { + "name": "job_execution_logs_schedule_id_workflow_schedule_id_fk", + "tableFrom": "job_execution_logs", + "tableTo": "workflow_schedule", + "columnsFrom": ["schedule_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + }, + "job_execution_logs_workspace_id_workspace_id_fk": { + "name": "job_execution_logs_workspace_id_workspace_id_fk", + "tableFrom": "job_execution_logs", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.knowledge_base": { + "name": "knowledge_base", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "token_count": { + "name": "token_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "embedding_model": { + "name": "embedding_model", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'text-embedding-3-small'" + }, + "embedding_dimension": { + "name": "embedding_dimension", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1536 + }, + "chunking_config": { + "name": "chunking_config", + "type": "json", + "primaryKey": false, + "notNull": true, + "default": "'{\"maxSize\": 1024, \"minSize\": 1, \"overlap\": 200}'" + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "kb_user_id_idx": { + "name": "kb_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "kb_workspace_id_idx": { + "name": "kb_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "kb_user_workspace_idx": { + "name": "kb_user_workspace_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "kb_deleted_at_idx": { + "name": "kb_deleted_at_idx", + "columns": [ + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "kb_workspace_deleted_partial_idx": { + "name": "kb_workspace_deleted_partial_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"knowledge_base\".\"deleted_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "kb_workspace_name_active_unique": { + "name": "kb_workspace_name_active_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"knowledge_base\".\"deleted_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "knowledge_base_user_id_user_id_fk": { + "name": "knowledge_base_user_id_user_id_fk", + "tableFrom": "knowledge_base", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "knowledge_base_workspace_id_workspace_id_fk": { + "name": "knowledge_base_workspace_id_workspace_id_fk", + "tableFrom": "knowledge_base", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.knowledge_base_tag_definitions": { + "name": "knowledge_base_tag_definitions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "knowledge_base_id": { + "name": "knowledge_base_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "tag_slot": { + "name": "tag_slot", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "field_type": { + "name": "field_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'text'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "kb_tag_definitions_kb_slot_idx": { + "name": "kb_tag_definitions_kb_slot_idx", + "columns": [ + { + "expression": "knowledge_base_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "tag_slot", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "kb_tag_definitions_kb_display_name_idx": { + "name": "kb_tag_definitions_kb_display_name_idx", + "columns": [ + { + "expression": "knowledge_base_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "display_name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "kb_tag_definitions_kb_id_idx": { + "name": "kb_tag_definitions_kb_id_idx", + "columns": [ + { + "expression": "knowledge_base_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "knowledge_base_tag_definitions_knowledge_base_id_knowledge_base_id_fk": { + "name": "knowledge_base_tag_definitions_knowledge_base_id_knowledge_base_id_fk", + "tableFrom": "knowledge_base_tag_definitions", + "tableTo": "knowledge_base", + "columnsFrom": ["knowledge_base_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.knowledge_connector": { + "name": "knowledge_connector", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "knowledge_base_id": { + "name": "knowledge_base_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "connector_type": { + "name": "connector_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "credential_id": { + "name": "credential_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "encrypted_api_key": { + "name": "encrypted_api_key", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_config": { + "name": "source_config", + "type": "json", + "primaryKey": false, + "notNull": true + }, + "sync_mode": { + "name": "sync_mode", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'full'" + }, + "sync_interval_minutes": { + "name": "sync_interval_minutes", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1440 + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "last_sync_at": { + "name": "last_sync_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "last_sync_error": { + "name": "last_sync_error", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "last_sync_doc_count": { + "name": "last_sync_doc_count", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "next_sync_at": { + "name": "next_sync_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "consecutive_failures": { + "name": "consecutive_failures", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "kc_knowledge_base_id_idx": { + "name": "kc_knowledge_base_id_idx", + "columns": [ + { + "expression": "knowledge_base_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "kc_status_next_sync_idx": { + "name": "kc_status_next_sync_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "next_sync_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "kc_archived_at_partial_idx": { + "name": "kc_archived_at_partial_idx", + "columns": [ + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"knowledge_connector\".\"archived_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "kc_deleted_at_partial_idx": { + "name": "kc_deleted_at_partial_idx", + "columns": [ + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"knowledge_connector\".\"deleted_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "knowledge_connector_knowledge_base_id_knowledge_base_id_fk": { + "name": "knowledge_connector_knowledge_base_id_knowledge_base_id_fk", + "tableFrom": "knowledge_connector", + "tableTo": "knowledge_base", + "columnsFrom": ["knowledge_base_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.knowledge_connector_sync_log": { + "name": "knowledge_connector_sync_log", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "connector_id": { + "name": "connector_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "started_at": { + "name": "started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "docs_added": { + "name": "docs_added", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "docs_updated": { + "name": "docs_updated", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "docs_deleted": { + "name": "docs_deleted", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "docs_unchanged": { + "name": "docs_unchanged", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "docs_failed": { + "name": "docs_failed", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "error_message": { + "name": "error_message", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "kcsl_connector_id_idx": { + "name": "kcsl_connector_id_idx", + "columns": [ + { + "expression": "connector_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "knowledge_connector_sync_log_connector_id_knowledge_connector_id_fk": { + "name": "knowledge_connector_sync_log_connector_id_knowledge_connector_id_fk", + "tableFrom": "knowledge_connector_sync_log", + "tableTo": "knowledge_connector", + "columnsFrom": ["connector_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.mcp_server_oauth": { + "name": "mcp_server_oauth", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "mcp_server_id": { + "name": "mcp_server_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "client_information": { + "name": "client_information", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tokens": { + "name": "tokens", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "code_verifier": { + "name": "code_verifier", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "state": { + "name": "state", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "state_created_at": { + "name": "state_created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "last_refreshed_at": { + "name": "last_refreshed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "mcp_server_oauth_server_unique": { + "name": "mcp_server_oauth_server_unique", + "columns": [ + { + "expression": "mcp_server_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "mcp_server_oauth_state_idx": { + "name": "mcp_server_oauth_state_idx", + "columns": [ + { + "expression": "state", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "mcp_server_oauth_mcp_server_id_mcp_servers_id_fk": { + "name": "mcp_server_oauth_mcp_server_id_mcp_servers_id_fk", + "tableFrom": "mcp_server_oauth", + "tableTo": "mcp_servers", + "columnsFrom": ["mcp_server_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "mcp_server_oauth_user_id_user_id_fk": { + "name": "mcp_server_oauth_user_id_user_id_fk", + "tableFrom": "mcp_server_oauth", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + }, + "mcp_server_oauth_workspace_id_workspace_id_fk": { + "name": "mcp_server_oauth_workspace_id_workspace_id_fk", + "tableFrom": "mcp_server_oauth", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.mcp_servers": { + "name": "mcp_servers", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "transport": { + "name": "transport", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "url": { + "name": "url", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "auth_type": { + "name": "auth_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'headers'" + }, + "oauth_client_id": { + "name": "oauth_client_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "oauth_client_secret": { + "name": "oauth_client_secret", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "headers": { + "name": "headers", + "type": "json", + "primaryKey": false, + "notNull": false, + "default": "'{}'" + }, + "timeout": { + "name": "timeout", + "type": "integer", + "primaryKey": false, + "notNull": false, + "default": 30000 + }, + "retries": { + "name": "retries", + "type": "integer", + "primaryKey": false, + "notNull": false, + "default": 3 + }, + "enabled": { + "name": "enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "last_connected": { + "name": "last_connected", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "connection_status": { + "name": "connection_status", + "type": "text", + "primaryKey": false, + "notNull": false, + "default": "'disconnected'" + }, + "last_error": { + "name": "last_error", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status_config": { + "name": "status_config", + "type": "jsonb", + "primaryKey": false, + "notNull": false, + "default": "'{}'" + }, + "tool_count": { + "name": "tool_count", + "type": "integer", + "primaryKey": false, + "notNull": false, + "default": 0 + }, + "last_tools_refresh": { + "name": "last_tools_refresh", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "total_requests": { + "name": "total_requests", + "type": "integer", + "primaryKey": false, + "notNull": false, + "default": 0 + }, + "last_used": { + "name": "last_used", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "mcp_servers_workspace_enabled_idx": { + "name": "mcp_servers_workspace_enabled_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "enabled", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "mcp_servers_workspace_deleted_partial_idx": { + "name": "mcp_servers_workspace_deleted_partial_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"mcp_servers\".\"deleted_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "mcp_servers_workspace_id_workspace_id_fk": { + "name": "mcp_servers_workspace_id_workspace_id_fk", + "tableFrom": "mcp_servers", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "mcp_servers_created_by_user_id_fk": { + "name": "mcp_servers_created_by_user_id_fk", + "tableFrom": "mcp_servers", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.member": { + "name": "member", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "organization_id": { + "name": "organization_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "role": { + "name": "role", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "member_user_id_unique": { + "name": "member_user_id_unique", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "member_organization_id_idx": { + "name": "member_organization_id_idx", + "columns": [ + { + "expression": "organization_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "member_user_id_user_id_fk": { + "name": "member_user_id_user_id_fk", + "tableFrom": "member", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "member_organization_id_organization_id_fk": { + "name": "member_organization_id_organization_id_fk", + "tableFrom": "member", + "tableTo": "organization", + "columnsFrom": ["organization_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.memory": { + "name": "memory", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "key": { + "name": "key", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "data": { + "name": "data", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "memory_key_idx": { + "name": "memory_key_idx", + "columns": [ + { + "expression": "key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "memory_workspace_idx": { + "name": "memory_workspace_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "memory_workspace_key_idx": { + "name": "memory_workspace_key_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "memory_workspace_deleted_partial_idx": { + "name": "memory_workspace_deleted_partial_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"memory\".\"deleted_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "memory_workspace_id_workspace_id_fk": { + "name": "memory_workspace_id_workspace_id_fk", + "tableFrom": "memory", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.mothership_inbox_allowed_sender": { + "name": "mothership_inbox_allowed_sender", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "label": { + "name": "label", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "added_by": { + "name": "added_by", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "inbox_sender_ws_email_idx": { + "name": "inbox_sender_ws_email_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "email", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "mothership_inbox_allowed_sender_workspace_id_workspace_id_fk": { + "name": "mothership_inbox_allowed_sender_workspace_id_workspace_id_fk", + "tableFrom": "mothership_inbox_allowed_sender", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "mothership_inbox_allowed_sender_added_by_user_id_fk": { + "name": "mothership_inbox_allowed_sender_added_by_user_id_fk", + "tableFrom": "mothership_inbox_allowed_sender", + "tableTo": "user", + "columnsFrom": ["added_by"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.mothership_inbox_task": { + "name": "mothership_inbox_task", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "from_email": { + "name": "from_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "from_name": { + "name": "from_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "body_preview": { + "name": "body_preview", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "body_text": { + "name": "body_text", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "body_html": { + "name": "body_html", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "email_message_id": { + "name": "email_message_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "in_reply_to": { + "name": "in_reply_to", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "response_message_id": { + "name": "response_message_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "agentmail_message_id": { + "name": "agentmail_message_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'received'" + }, + "chat_id": { + "name": "chat_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "trigger_job_id": { + "name": "trigger_job_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "result_summary": { + "name": "result_summary", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "error_message": { + "name": "error_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "rejection_reason": { + "name": "rejection_reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "cc_recipients": { + "name": "cc_recipients", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "processing_started_at": { + "name": "processing_started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "inbox_task_ws_created_at_idx": { + "name": "inbox_task_ws_created_at_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "inbox_task_ws_status_idx": { + "name": "inbox_task_ws_status_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "inbox_task_response_msg_id_idx": { + "name": "inbox_task_response_msg_id_idx", + "columns": [ + { + "expression": "response_message_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "inbox_task_email_msg_id_idx": { + "name": "inbox_task_email_msg_id_idx", + "columns": [ + { + "expression": "email_message_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "mothership_inbox_task_workspace_id_workspace_id_fk": { + "name": "mothership_inbox_task_workspace_id_workspace_id_fk", + "tableFrom": "mothership_inbox_task", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "mothership_inbox_task_chat_id_copilot_chats_id_fk": { + "name": "mothership_inbox_task_chat_id_copilot_chats_id_fk", + "tableFrom": "mothership_inbox_task", + "tableTo": "copilot_chats", + "columnsFrom": ["chat_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.mothership_inbox_webhook": { + "name": "mothership_inbox_webhook", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "webhook_id": { + "name": "webhook_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "secret": { + "name": "secret", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "mothership_inbox_webhook_workspace_id_workspace_id_fk": { + "name": "mothership_inbox_webhook_workspace_id_workspace_id_fk", + "tableFrom": "mothership_inbox_webhook", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "mothership_inbox_webhook_workspace_id_unique": { + "name": "mothership_inbox_webhook_workspace_id_unique", + "nullsNotDistinct": false, + "columns": ["workspace_id"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.mothership_settings": { + "name": "mothership_settings", + "schema": "", + "columns": { + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "mcp_tool_refs": { + "name": "mcp_tool_refs", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "custom_tool_refs": { + "name": "custom_tool_refs", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "skill_refs": { + "name": "skill_refs", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "mothership_settings_workspace_id_idx": { + "name": "mothership_settings_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "mothership_settings_workspace_id_workspace_id_fk": { + "name": "mothership_settings_workspace_id_workspace_id_fk", + "tableFrom": "mothership_settings", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.organization": { + "name": "organization", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "slug": { + "name": "slug", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "logo": { + "name": "logo", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "metadata": { + "name": "metadata", + "type": "json", + "primaryKey": false, + "notNull": false + }, + "whitelabel_settings": { + "name": "whitelabel_settings", + "type": "json", + "primaryKey": false, + "notNull": false + }, + "data_retention_settings": { + "name": "data_retention_settings", + "type": "json", + "primaryKey": false, + "notNull": false + }, + "org_usage_limit": { + "name": "org_usage_limit", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "storage_used_bytes": { + "name": "storage_used_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "departed_member_usage": { + "name": "departed_member_usage", + "type": "numeric", + "primaryKey": false, + "notNull": true, + "default": "'0'" + }, + "credit_balance": { + "name": "credit_balance", + "type": "numeric", + "primaryKey": false, + "notNull": true, + "default": "'0'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.outbox_event": { + "name": "outbox_event", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "event_type": { + "name": "event_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "payload": { + "name": "payload", + "type": "json", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "attempts": { + "name": "attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "max_attempts": { + "name": "max_attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 10 + }, + "available_at": { + "name": "available_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "locked_at": { + "name": "locked_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "last_error": { + "name": "last_error", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "processed_at": { + "name": "processed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "outbox_event_status_available_idx": { + "name": "outbox_event_status_available_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "available_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "outbox_event_locked_at_idx": { + "name": "outbox_event_locked_at_idx", + "columns": [ + { + "expression": "locked_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.paused_executions": { + "name": "paused_executions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "execution_id": { + "name": "execution_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "execution_snapshot": { + "name": "execution_snapshot", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "pause_points": { + "name": "pause_points", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "total_pause_count": { + "name": "total_pause_count", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "resumed_count": { + "name": "resumed_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'paused'" + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "paused_at": { + "name": "paused_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "next_resume_at": { + "name": "next_resume_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "paused_executions_workflow_id_idx": { + "name": "paused_executions_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "paused_executions_status_idx": { + "name": "paused_executions_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "paused_executions_execution_id_unique": { + "name": "paused_executions_execution_id_unique", + "columns": [ + { + "expression": "execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "paused_executions_next_resume_at_idx": { + "name": "paused_executions_next_resume_at_idx", + "columns": [ + { + "expression": "next_resume_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "status = 'paused' AND next_resume_at IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "paused_executions_workflow_id_workflow_id_fk": { + "name": "paused_executions_workflow_id_workflow_id_fk", + "tableFrom": "paused_executions", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.pending_credential_draft": { + "name": "pending_credential_draft", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider_id": { + "name": "provider_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "credential_id": { + "name": "credential_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "pending_draft_user_provider_ws": { + "name": "pending_draft_user_provider_ws", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "provider_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "pending_credential_draft_user_id_user_id_fk": { + "name": "pending_credential_draft_user_id_user_id_fk", + "tableFrom": "pending_credential_draft", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "pending_credential_draft_workspace_id_workspace_id_fk": { + "name": "pending_credential_draft_workspace_id_workspace_id_fk", + "tableFrom": "pending_credential_draft", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "pending_credential_draft_credential_id_credential_id_fk": { + "name": "pending_credential_draft_credential_id_credential_id_fk", + "tableFrom": "pending_credential_draft", + "tableTo": "credential", + "columnsFrom": ["credential_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.permission_group": { + "name": "permission_group", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "config": { + "name": "config", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "auto_add_new_members": { + "name": "auto_add_new_members", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + } + }, + "indexes": { + "permission_group_created_by_idx": { + "name": "permission_group_created_by_idx", + "columns": [ + { + "expression": "created_by", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "permission_group_workspace_name_unique": { + "name": "permission_group_workspace_name_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "permission_group_workspace_auto_add_unique": { + "name": "permission_group_workspace_auto_add_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "auto_add_new_members = true", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "permission_group_workspace_id_workspace_id_fk": { + "name": "permission_group_workspace_id_workspace_id_fk", + "tableFrom": "permission_group", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "permission_group_created_by_user_id_fk": { + "name": "permission_group_created_by_user_id_fk", + "tableFrom": "permission_group", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.permission_group_member": { + "name": "permission_group_member", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "permission_group_id": { + "name": "permission_group_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "assigned_by": { + "name": "assigned_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "assigned_at": { + "name": "assigned_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "permission_group_member_group_id_idx": { + "name": "permission_group_member_group_id_idx", + "columns": [ + { + "expression": "permission_group_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "permission_group_member_group_user_unique": { + "name": "permission_group_member_group_user_unique", + "columns": [ + { + "expression": "permission_group_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "permission_group_member_workspace_user_unique": { + "name": "permission_group_member_workspace_user_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "permission_group_member_permission_group_id_permission_group_id_fk": { + "name": "permission_group_member_permission_group_id_permission_group_id_fk", + "tableFrom": "permission_group_member", + "tableTo": "permission_group", + "columnsFrom": ["permission_group_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "permission_group_member_workspace_id_workspace_id_fk": { + "name": "permission_group_member_workspace_id_workspace_id_fk", + "tableFrom": "permission_group_member", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "permission_group_member_user_id_user_id_fk": { + "name": "permission_group_member_user_id_user_id_fk", + "tableFrom": "permission_group_member", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "permission_group_member_assigned_by_user_id_fk": { + "name": "permission_group_member_assigned_by_user_id_fk", + "tableFrom": "permission_group_member", + "tableTo": "user", + "columnsFrom": ["assigned_by"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.permissions": { + "name": "permissions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "entity_type": { + "name": "entity_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "entity_id": { + "name": "entity_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "permission_type": { + "name": "permission_type", + "type": "permission_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "permissions_user_id_idx": { + "name": "permissions_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "permissions_entity_idx": { + "name": "permissions_entity_idx", + "columns": [ + { + "expression": "entity_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "entity_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "permissions_user_entity_type_idx": { + "name": "permissions_user_entity_type_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "entity_type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "permissions_user_entity_permission_idx": { + "name": "permissions_user_entity_permission_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "entity_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "permission_type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "permissions_user_entity_idx": { + "name": "permissions_user_entity_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "entity_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "entity_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "permissions_unique_constraint": { + "name": "permissions_unique_constraint", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "entity_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "entity_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "permissions_user_id_user_id_fk": { + "name": "permissions_user_id_user_id_fk", + "tableFrom": "permissions", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.rate_limit_bucket": { + "name": "rate_limit_bucket", + "schema": "", + "columns": { + "key": { + "name": "key", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "tokens": { + "name": "tokens", + "type": "numeric", + "primaryKey": false, + "notNull": true + }, + "last_refill_at": { + "name": "last_refill_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.resume_queue": { + "name": "resume_queue", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "paused_execution_id": { + "name": "paused_execution_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "parent_execution_id": { + "name": "parent_execution_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "new_execution_id": { + "name": "new_execution_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "context_id": { + "name": "context_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "resume_input": { + "name": "resume_input", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "queued_at": { + "name": "queued_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "claimed_at": { + "name": "claimed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "failure_reason": { + "name": "failure_reason", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "resume_queue_parent_status_idx": { + "name": "resume_queue_parent_status_idx", + "columns": [ + { + "expression": "parent_execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "queued_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "resume_queue_new_execution_idx": { + "name": "resume_queue_new_execution_idx", + "columns": [ + { + "expression": "new_execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "resume_queue_paused_execution_id_paused_executions_id_fk": { + "name": "resume_queue_paused_execution_id_paused_executions_id_fk", + "tableFrom": "resume_queue", + "tableTo": "paused_executions", + "columnsFrom": ["paused_execution_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.session": { + "name": "session", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "token": { + "name": "token", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "ip_address": { + "name": "ip_address", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "user_agent": { + "name": "user_agent", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "active_organization_id": { + "name": "active_organization_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "impersonated_by": { + "name": "impersonated_by", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "session_user_id_idx": { + "name": "session_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "session_token_idx": { + "name": "session_token_idx", + "columns": [ + { + "expression": "token", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "session_user_id_user_id_fk": { + "name": "session_user_id_user_id_fk", + "tableFrom": "session", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "session_active_organization_id_organization_id_fk": { + "name": "session_active_organization_id_organization_id_fk", + "tableFrom": "session", + "tableTo": "organization", + "columnsFrom": ["active_organization_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "session_token_unique": { + "name": "session_token_unique", + "nullsNotDistinct": false, + "columns": ["token"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.settings": { + "name": "settings", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "theme": { + "name": "theme", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'system'" + }, + "auto_connect": { + "name": "auto_connect", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "telemetry_enabled": { + "name": "telemetry_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "email_preferences": { + "name": "email_preferences", + "type": "json", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "billing_usage_notifications_enabled": { + "name": "billing_usage_notifications_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "show_training_controls": { + "name": "show_training_controls", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "super_user_mode_enabled": { + "name": "super_user_mode_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "mothership_environment": { + "name": "mothership_environment", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'default'" + }, + "error_notifications_enabled": { + "name": "error_notifications_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "snap_to_grid_size": { + "name": "snap_to_grid_size", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "show_action_bar": { + "name": "show_action_bar", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "copilot_enabled_models": { + "name": "copilot_enabled_models", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "copilot_auto_allowed_tools": { + "name": "copilot_auto_allowed_tools", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'" + }, + "last_active_workspace_id": { + "name": "last_active_workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "settings_user_id_user_id_fk": { + "name": "settings_user_id_user_id_fk", + "tableFrom": "settings", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "settings_user_id_unique": { + "name": "settings_user_id_unique", + "nullsNotDistinct": false, + "columns": ["user_id"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.skill": { + "name": "skill", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "content": { + "name": "content", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "skill_workspace_name_unique": { + "name": "skill_workspace_name_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "skill_workspace_id_workspace_id_fk": { + "name": "skill_workspace_id_workspace_id_fk", + "tableFrom": "skill", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "skill_user_id_user_id_fk": { + "name": "skill_user_id_user_id_fk", + "tableFrom": "skill", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.sso_provider": { + "name": "sso_provider", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "issuer": { + "name": "issuer", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "domain": { + "name": "domain", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "oidc_config": { + "name": "oidc_config", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "saml_config": { + "name": "saml_config", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider_id": { + "name": "provider_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "organization_id": { + "name": "organization_id", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "sso_provider_provider_id_idx": { + "name": "sso_provider_provider_id_idx", + "columns": [ + { + "expression": "provider_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "sso_provider_domain_idx": { + "name": "sso_provider_domain_idx", + "columns": [ + { + "expression": "domain", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "sso_provider_user_id_idx": { + "name": "sso_provider_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "sso_provider_organization_id_idx": { + "name": "sso_provider_organization_id_idx", + "columns": [ + { + "expression": "organization_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "sso_provider_user_id_user_id_fk": { + "name": "sso_provider_user_id_user_id_fk", + "tableFrom": "sso_provider", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "sso_provider_organization_id_organization_id_fk": { + "name": "sso_provider_organization_id_organization_id_fk", + "tableFrom": "sso_provider", + "tableTo": "organization", + "columnsFrom": ["organization_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.subscription": { + "name": "subscription", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "plan": { + "name": "plan", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "reference_id": { + "name": "reference_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "stripe_customer_id": { + "name": "stripe_customer_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "stripe_subscription_id": { + "name": "stripe_subscription_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "period_start": { + "name": "period_start", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "period_end": { + "name": "period_end", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "cancel_at_period_end": { + "name": "cancel_at_period_end", + "type": "boolean", + "primaryKey": false, + "notNull": false + }, + "cancel_at": { + "name": "cancel_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "canceled_at": { + "name": "canceled_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "ended_at": { + "name": "ended_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "seats": { + "name": "seats", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "trial_start": { + "name": "trial_start", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "trial_end": { + "name": "trial_end", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "billing_interval": { + "name": "billing_interval", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "stripe_schedule_id": { + "name": "stripe_schedule_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "metadata": { + "name": "metadata", + "type": "json", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "subscription_reference_status_idx": { + "name": "subscription_reference_status_idx", + "columns": [ + { + "expression": "reference_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "check_enterprise_metadata": { + "name": "check_enterprise_metadata", + "value": "plan != 'enterprise' OR metadata IS NOT NULL" + } + }, + "isRLSEnabled": false + }, + "public.table_row_executions": { + "name": "table_row_executions", + "schema": "", + "columns": { + "table_id": { + "name": "table_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "row_id": { + "name": "row_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "group_id": { + "name": "group_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "execution_id": { + "name": "execution_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "job_id": { + "name": "job_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "error": { + "name": "error", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "running_block_ids": { + "name": "running_block_ids", + "type": "text[]", + "primaryKey": false, + "notNull": true, + "default": "'{}'::text[]" + }, + "block_errors": { + "name": "block_errors", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "cancelled_at": { + "name": "cancelled_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "table_row_executions_table_status_idx": { + "name": "table_row_executions_table_status_idx", + "columns": [ + { + "expression": "table_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"table_row_executions\".\"status\" IN ('queued', 'running', 'pending')", + "concurrently": false, + "method": "btree", + "with": {} + }, + "table_row_executions_execution_id_idx": { + "name": "table_row_executions_execution_id_idx", + "columns": [ + { + "expression": "execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"table_row_executions\".\"execution_id\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "table_row_executions_table_group_idx": { + "name": "table_row_executions_table_group_idx", + "columns": [ + { + "expression": "table_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "group_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "table_row_executions_table_id_user_table_definitions_id_fk": { + "name": "table_row_executions_table_id_user_table_definitions_id_fk", + "tableFrom": "table_row_executions", + "tableTo": "user_table_definitions", + "columnsFrom": ["table_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "table_row_executions_row_id_user_table_rows_id_fk": { + "name": "table_row_executions_row_id_user_table_rows_id_fk", + "tableFrom": "table_row_executions", + "tableTo": "user_table_rows", + "columnsFrom": ["row_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "table_row_executions_row_id_group_id_pk": { + "name": "table_row_executions_row_id_group_id_pk", + "columns": ["row_id", "group_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.table_run_dispatches": { + "name": "table_run_dispatches", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "table_id": { + "name": "table_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "request_id": { + "name": "request_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mode": { + "name": "mode", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "scope": { + "name": "scope", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "cursor": { + "name": "cursor", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "limit": { + "name": "limit", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "processed_count": { + "name": "processed_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "is_manual_run": { + "name": "is_manual_run", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "requested_at": { + "name": "requested_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "cancelled_at": { + "name": "cancelled_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "table_run_dispatches_active_idx": { + "name": "table_run_dispatches_active_idx", + "columns": [ + { + "expression": "table_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "table_run_dispatches_watchdog_idx": { + "name": "table_run_dispatches_watchdog_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "requested_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "table_run_dispatches_table_id_user_table_definitions_id_fk": { + "name": "table_run_dispatches_table_id_user_table_definitions_id_fk", + "tableFrom": "table_run_dispatches", + "tableTo": "user_table_definitions", + "columnsFrom": ["table_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "table_run_dispatches_workspace_id_workspace_id_fk": { + "name": "table_run_dispatches_workspace_id_workspace_id_fk", + "tableFrom": "table_run_dispatches", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.template_creators": { + "name": "template_creators", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "reference_type": { + "name": "reference_type", + "type": "template_creator_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "reference_id": { + "name": "reference_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "profile_image_url": { + "name": "profile_image_url", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "verified": { + "name": "verified", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "template_creators_reference_idx": { + "name": "template_creators_reference_idx", + "columns": [ + { + "expression": "reference_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "reference_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "template_creators_reference_id_idx": { + "name": "template_creators_reference_id_idx", + "columns": [ + { + "expression": "reference_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "template_creators_created_by_idx": { + "name": "template_creators_created_by_idx", + "columns": [ + { + "expression": "created_by", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "template_creators_created_by_user_id_fk": { + "name": "template_creators_created_by_user_id_fk", + "tableFrom": "template_creators", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.template_stars": { + "name": "template_stars", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "template_id": { + "name": "template_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "starred_at": { + "name": "starred_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "template_stars_user_id_idx": { + "name": "template_stars_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "template_stars_template_id_idx": { + "name": "template_stars_template_id_idx", + "columns": [ + { + "expression": "template_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "template_stars_user_template_idx": { + "name": "template_stars_user_template_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "template_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "template_stars_template_user_idx": { + "name": "template_stars_template_user_idx", + "columns": [ + { + "expression": "template_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "template_stars_starred_at_idx": { + "name": "template_stars_starred_at_idx", + "columns": [ + { + "expression": "starred_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "template_stars_template_starred_at_idx": { + "name": "template_stars_template_starred_at_idx", + "columns": [ + { + "expression": "template_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "starred_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "template_stars_user_template_unique": { + "name": "template_stars_user_template_unique", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "template_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "template_stars_user_id_user_id_fk": { + "name": "template_stars_user_id_user_id_fk", + "tableFrom": "template_stars", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "template_stars_template_id_templates_id_fk": { + "name": "template_stars_template_id_templates_id_fk", + "tableFrom": "template_stars", + "tableTo": "templates", + "columnsFrom": ["template_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.templates": { + "name": "templates", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "creator_id": { + "name": "creator_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "views": { + "name": "views", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "stars": { + "name": "stars", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "status": { + "name": "status", + "type": "template_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "tags": { + "name": "tags", + "type": "text[]", + "primaryKey": false, + "notNull": true, + "default": "'{}'::text[]" + }, + "required_credentials": { + "name": "required_credentials", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'" + }, + "state": { + "name": "state", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "og_image_url": { + "name": "og_image_url", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "templates_status_idx": { + "name": "templates_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "templates_creator_id_idx": { + "name": "templates_creator_id_idx", + "columns": [ + { + "expression": "creator_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "templates_views_idx": { + "name": "templates_views_idx", + "columns": [ + { + "expression": "views", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "templates_stars_idx": { + "name": "templates_stars_idx", + "columns": [ + { + "expression": "stars", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "templates_status_views_idx": { + "name": "templates_status_views_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "views", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "templates_status_stars_idx": { + "name": "templates_status_stars_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "stars", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "templates_created_at_idx": { + "name": "templates_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "templates_updated_at_idx": { + "name": "templates_updated_at_idx", + "columns": [ + { + "expression": "updated_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "templates_workflow_id_workflow_id_fk": { + "name": "templates_workflow_id_workflow_id_fk", + "tableFrom": "templates", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + }, + "templates_creator_id_template_creators_id_fk": { + "name": "templates_creator_id_template_creators_id_fk", + "tableFrom": "templates", + "tableTo": "template_creators", + "columnsFrom": ["creator_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.usage_log": { + "name": "usage_log", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "category": { + "name": "category", + "type": "usage_log_category", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "source": { + "name": "source", + "type": "usage_log_source", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "cost": { + "name": "cost", + "type": "numeric", + "primaryKey": false, + "notNull": true + }, + "event_key": { + "name": "event_key", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "billing_entity_type": { + "name": "billing_entity_type", + "type": "billing_entity_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": false + }, + "billing_entity_id": { + "name": "billing_entity_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "billing_period_start": { + "name": "billing_period_start", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "billing_period_end": { + "name": "billing_period_end", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "execution_id": { + "name": "execution_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "usage_log_user_created_at_idx": { + "name": "usage_log_user_created_at_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "usage_log_source_idx": { + "name": "usage_log_source_idx", + "columns": [ + { + "expression": "source", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "usage_log_workspace_id_idx": { + "name": "usage_log_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "usage_log_workflow_id_idx": { + "name": "usage_log_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "usage_log_event_key_unique": { + "name": "usage_log_event_key_unique", + "columns": [ + { + "expression": "event_key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"usage_log\".\"event_key\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "usage_log_billing_entity_period_idx": { + "name": "usage_log_billing_entity_period_idx", + "columns": [ + { + "expression": "billing_entity_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "billing_entity_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "billing_period_start", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "billing_period_end", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"usage_log\".\"billing_entity_type\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "usage_log_workspace_created_at_idx": { + "name": "usage_log_workspace_created_at_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "usage_log_execution_id_idx": { + "name": "usage_log_execution_id_idx", + "columns": [ + { + "expression": "execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "usage_log_user_id_user_id_fk": { + "name": "usage_log_user_id_user_id_fk", + "tableFrom": "usage_log", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "usage_log_workspace_id_workspace_id_fk": { + "name": "usage_log_workspace_id_workspace_id_fk", + "tableFrom": "usage_log", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + }, + "usage_log_workflow_id_workflow_id_fk": { + "name": "usage_log_workflow_id_workflow_id_fk", + "tableFrom": "usage_log", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "usage_log_billing_scope_all_or_none": { + "name": "usage_log_billing_scope_all_or_none", + "value": "(\n (\"usage_log\".\"billing_entity_type\" IS NULL AND \"usage_log\".\"billing_entity_id\" IS NULL AND \"usage_log\".\"billing_period_start\" IS NULL AND \"usage_log\".\"billing_period_end\" IS NULL)\n OR\n (\"usage_log\".\"billing_entity_type\" IS NOT NULL AND \"usage_log\".\"billing_entity_id\" IS NOT NULL AND \"usage_log\".\"billing_period_start\" IS NOT NULL AND \"usage_log\".\"billing_period_end\" IS NOT NULL AND \"usage_log\".\"billing_period_start\" < \"usage_log\".\"billing_period_end\")\n )" + } + }, + "isRLSEnabled": false + }, + "public.user": { + "name": "user", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "normalized_email": { + "name": "normalized_email", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "email_verified": { + "name": "email_verified", + "type": "boolean", + "primaryKey": false, + "notNull": true + }, + "image": { + "name": "image", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "stripe_customer_id": { + "name": "stripe_customer_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "role": { + "name": "role", + "type": "text", + "primaryKey": false, + "notNull": false, + "default": "'user'" + }, + "banned": { + "name": "banned", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + }, + "ban_reason": { + "name": "ban_reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "ban_expires": { + "name": "ban_expires", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "user_email_unique": { + "name": "user_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + }, + "user_normalized_email_unique": { + "name": "user_normalized_email_unique", + "nullsNotDistinct": false, + "columns": ["normalized_email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.user_stats": { + "name": "user_stats", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "total_manual_executions": { + "name": "total_manual_executions", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "total_api_calls": { + "name": "total_api_calls", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "total_webhook_triggers": { + "name": "total_webhook_triggers", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "total_scheduled_executions": { + "name": "total_scheduled_executions", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "total_chat_executions": { + "name": "total_chat_executions", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "total_mcp_executions": { + "name": "total_mcp_executions", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "total_a2a_executions": { + "name": "total_a2a_executions", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "total_tokens_used": { + "name": "total_tokens_used", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "total_cost": { + "name": "total_cost", + "type": "numeric", + "primaryKey": false, + "notNull": true, + "default": "'0'" + }, + "current_usage_limit": { + "name": "current_usage_limit", + "type": "numeric", + "primaryKey": false, + "notNull": false, + "default": "'5'" + }, + "usage_limit_updated_at": { + "name": "usage_limit_updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false, + "default": "now()" + }, + "current_period_cost": { + "name": "current_period_cost", + "type": "numeric", + "primaryKey": false, + "notNull": true, + "default": "'0'" + }, + "last_period_cost": { + "name": "last_period_cost", + "type": "numeric", + "primaryKey": false, + "notNull": false, + "default": "'0'" + }, + "billed_overage_this_period": { + "name": "billed_overage_this_period", + "type": "numeric", + "primaryKey": false, + "notNull": true, + "default": "'0'" + }, + "pro_period_cost_snapshot": { + "name": "pro_period_cost_snapshot", + "type": "numeric", + "primaryKey": false, + "notNull": false, + "default": "'0'" + }, + "pro_period_cost_snapshot_at": { + "name": "pro_period_cost_snapshot_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "credit_balance": { + "name": "credit_balance", + "type": "numeric", + "primaryKey": false, + "notNull": true, + "default": "'0'" + }, + "total_copilot_cost": { + "name": "total_copilot_cost", + "type": "numeric", + "primaryKey": false, + "notNull": true, + "default": "'0'" + }, + "current_period_copilot_cost": { + "name": "current_period_copilot_cost", + "type": "numeric", + "primaryKey": false, + "notNull": true, + "default": "'0'" + }, + "last_period_copilot_cost": { + "name": "last_period_copilot_cost", + "type": "numeric", + "primaryKey": false, + "notNull": false, + "default": "'0'" + }, + "total_copilot_tokens": { + "name": "total_copilot_tokens", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "total_copilot_calls": { + "name": "total_copilot_calls", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "total_mcp_copilot_calls": { + "name": "total_mcp_copilot_calls", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "total_mcp_copilot_cost": { + "name": "total_mcp_copilot_cost", + "type": "numeric", + "primaryKey": false, + "notNull": true, + "default": "'0'" + }, + "current_period_mcp_copilot_cost": { + "name": "current_period_mcp_copilot_cost", + "type": "numeric", + "primaryKey": false, + "notNull": true, + "default": "'0'" + }, + "storage_used_bytes": { + "name": "storage_used_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "last_active": { + "name": "last_active", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "billing_blocked": { + "name": "billing_blocked", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "billing_blocked_reason": { + "name": "billing_blocked_reason", + "type": "billing_blocked_reason", + "typeSchema": "public", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "user_stats_user_id_user_id_fk": { + "name": "user_stats_user_id_user_id_fk", + "tableFrom": "user_stats", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "user_stats_user_id_unique": { + "name": "user_stats_user_id_unique", + "nullsNotDistinct": false, + "columns": ["user_id"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.user_table_definitions": { + "name": "user_table_definitions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "schema": { + "name": "schema", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "max_rows": { + "name": "max_rows", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 10000 + }, + "row_count": { + "name": "row_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "import_status": { + "name": "import_status", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "import_id": { + "name": "import_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "import_error": { + "name": "import_error", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "import_rows_processed": { + "name": "import_rows_processed", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "import_started_at": { + "name": "import_started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "user_table_def_workspace_id_idx": { + "name": "user_table_def_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "user_table_def_workspace_name_unique": { + "name": "user_table_def_workspace_name_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"user_table_definitions\".\"archived_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "user_table_def_archived_at_idx": { + "name": "user_table_def_archived_at_idx", + "columns": [ + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "user_table_def_workspace_archived_partial_idx": { + "name": "user_table_def_workspace_archived_partial_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"user_table_definitions\".\"archived_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "user_table_definitions_workspace_id_workspace_id_fk": { + "name": "user_table_definitions_workspace_id_workspace_id_fk", + "tableFrom": "user_table_definitions", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "user_table_definitions_created_by_user_id_fk": { + "name": "user_table_definitions_created_by_user_id_fk", + "tableFrom": "user_table_definitions", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.user_table_rows": { + "name": "user_table_rows", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "table_id": { + "name": "table_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "data": { + "name": "data", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "position": { + "name": "position", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "user_table_rows_table_id_idx": { + "name": "user_table_rows_table_id_idx", + "columns": [ + { + "expression": "table_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "user_table_rows_data_gin_idx": { + "name": "user_table_rows_data_gin_idx", + "columns": [ + { + "expression": "data", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "gin", + "with": {} + }, + "user_table_rows_workspace_table_idx": { + "name": "user_table_rows_workspace_table_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "table_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "user_table_rows_table_position_idx": { + "name": "user_table_rows_table_position_idx", + "columns": [ + { + "expression": "table_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "position", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "user_table_rows_table_id_user_table_definitions_id_fk": { + "name": "user_table_rows_table_id_user_table_definitions_id_fk", + "tableFrom": "user_table_rows", + "tableTo": "user_table_definitions", + "columnsFrom": ["table_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "user_table_rows_workspace_id_workspace_id_fk": { + "name": "user_table_rows_workspace_id_workspace_id_fk", + "tableFrom": "user_table_rows", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "user_table_rows_created_by_user_id_fk": { + "name": "user_table_rows_created_by_user_id_fk", + "tableFrom": "user_table_rows", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.verification": { + "name": "verification", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "identifier": { + "name": "identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "value": { + "name": "value", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "verification_identifier_idx": { + "name": "verification_identifier_idx", + "columns": [ + { + "expression": "identifier", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "verification_expires_at_idx": { + "name": "verification_expires_at_idx", + "columns": [ + { + "expression": "expires_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.waitlist": { + "name": "waitlist", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "waitlist_email_unique": { + "name": "waitlist_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.webhook": { + "name": "webhook", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "deployment_version_id": { + "name": "deployment_version_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "block_id": { + "name": "block_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "path": { + "name": "path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "provider_config": { + "name": "provider_config", + "type": "json", + "primaryKey": false, + "notNull": false + }, + "is_active": { + "name": "is_active", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "failed_count": { + "name": "failed_count", + "type": "integer", + "primaryKey": false, + "notNull": false, + "default": 0 + }, + "last_failed_at": { + "name": "last_failed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "credential_set_id": { + "name": "credential_set_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "path_deployment_unique": { + "name": "path_deployment_unique", + "columns": [ + { + "expression": "path", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deployment_version_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"webhook\".\"archived_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "webhook_workflow_deployment_idx": { + "name": "webhook_workflow_deployment_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deployment_version_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "webhook_credential_set_id_idx": { + "name": "webhook_credential_set_id_idx", + "columns": [ + { + "expression": "credential_set_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "webhook_archived_at_partial_idx": { + "name": "webhook_archived_at_partial_idx", + "columns": [ + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"webhook\".\"archived_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_webhook_on_provider_is_active_workflow_id_deploym_bdeed5468": { + "name": "idx_webhook_on_provider_is_active_workflow_id_deploym_bdeed5468", + "columns": [ + { + "expression": "provider", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "is_active", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deployment_version_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_webhook_on_workflow_id_block_id_updated_at_desc": { + "name": "idx_webhook_on_workflow_id_block_id_updated_at_desc", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "block_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "updated_at", + "isExpression": false, + "asc": false, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "webhook_workflow_id_workflow_id_fk": { + "name": "webhook_workflow_id_workflow_id_fk", + "tableFrom": "webhook", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "webhook_deployment_version_id_workflow_deployment_version_id_fk": { + "name": "webhook_deployment_version_id_workflow_deployment_version_id_fk", + "tableFrom": "webhook", + "tableTo": "workflow_deployment_version", + "columnsFrom": ["deployment_version_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "webhook_credential_set_id_credential_set_id_fk": { + "name": "webhook_credential_set_id_credential_set_id_fk", + "tableFrom": "webhook", + "tableTo": "credential_set", + "columnsFrom": ["credential_set_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow": { + "name": "workflow", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "folder_id": { + "name": "folder_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sort_order": { + "name": "sort_order", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "color": { + "name": "color", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'#3972F6'" + }, + "last_synced": { + "name": "last_synced", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "is_deployed": { + "name": "is_deployed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "deployed_at": { + "name": "deployed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "is_public_api": { + "name": "is_public_api", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "locked": { + "name": "locked", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "run_count": { + "name": "run_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "last_run_at": { + "name": "last_run_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "variables": { + "name": "variables", + "type": "json", + "primaryKey": false, + "notNull": false, + "default": "'{}'" + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "workflow_user_id_idx": { + "name": "workflow_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_workspace_id_idx": { + "name": "workflow_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_user_workspace_idx": { + "name": "workflow_user_workspace_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_workspace_folder_name_active_unique": { + "name": "workflow_workspace_folder_name_active_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "coalesce(\"folder_id\", '')", + "asc": true, + "isExpression": true, + "nulls": "last" + }, + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"workflow\".\"archived_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_folder_sort_idx": { + "name": "workflow_folder_sort_idx", + "columns": [ + { + "expression": "folder_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "sort_order", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_archived_at_idx": { + "name": "workflow_archived_at_idx", + "columns": [ + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_workspace_archived_partial_idx": { + "name": "workflow_workspace_archived_partial_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"workflow\".\"archived_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_user_id_user_id_fk": { + "name": "workflow_user_id_user_id_fk", + "tableFrom": "workflow", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_workspace_id_workspace_id_fk": { + "name": "workflow_workspace_id_workspace_id_fk", + "tableFrom": "workflow", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_folder_id_workflow_folder_id_fk": { + "name": "workflow_folder_id_workflow_folder_id_fk", + "tableFrom": "workflow", + "tableTo": "workflow_folder", + "columnsFrom": ["folder_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow_blocks": { + "name": "workflow_blocks", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "type": { + "name": "type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "position_x": { + "name": "position_x", + "type": "numeric", + "primaryKey": false, + "notNull": true + }, + "position_y": { + "name": "position_y", + "type": "numeric", + "primaryKey": false, + "notNull": true + }, + "enabled": { + "name": "enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "horizontal_handles": { + "name": "horizontal_handles", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "is_wide": { + "name": "is_wide", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "advanced_mode": { + "name": "advanced_mode", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "trigger_mode": { + "name": "trigger_mode", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "locked": { + "name": "locked", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "height": { + "name": "height", + "type": "numeric", + "primaryKey": false, + "notNull": true, + "default": "'0'" + }, + "sub_blocks": { + "name": "sub_blocks", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "outputs": { + "name": "outputs", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "data": { + "name": "data", + "type": "jsonb", + "primaryKey": false, + "notNull": false, + "default": "'{}'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workflow_blocks_workflow_id_idx": { + "name": "workflow_blocks_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_blocks_type_idx": { + "name": "workflow_blocks_type_idx", + "columns": [ + { + "expression": "type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_blocks_workflow_id_workflow_id_fk": { + "name": "workflow_blocks_workflow_id_workflow_id_fk", + "tableFrom": "workflow_blocks", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow_checkpoints": { + "name": "workflow_checkpoints", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "chat_id": { + "name": "chat_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "message_id": { + "name": "message_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "workflow_state": { + "name": "workflow_state", + "type": "json", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workflow_checkpoints_user_id_idx": { + "name": "workflow_checkpoints_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_checkpoints_workflow_id_idx": { + "name": "workflow_checkpoints_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_checkpoints_chat_id_idx": { + "name": "workflow_checkpoints_chat_id_idx", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_checkpoints_message_id_idx": { + "name": "workflow_checkpoints_message_id_idx", + "columns": [ + { + "expression": "message_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_checkpoints_user_workflow_idx": { + "name": "workflow_checkpoints_user_workflow_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_checkpoints_workflow_chat_idx": { + "name": "workflow_checkpoints_workflow_chat_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_checkpoints_created_at_idx": { + "name": "workflow_checkpoints_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_checkpoints_chat_created_at_idx": { + "name": "workflow_checkpoints_chat_created_at_idx", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_checkpoints_user_id_user_id_fk": { + "name": "workflow_checkpoints_user_id_user_id_fk", + "tableFrom": "workflow_checkpoints", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_checkpoints_workflow_id_workflow_id_fk": { + "name": "workflow_checkpoints_workflow_id_workflow_id_fk", + "tableFrom": "workflow_checkpoints", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_checkpoints_chat_id_copilot_chats_id_fk": { + "name": "workflow_checkpoints_chat_id_copilot_chats_id_fk", + "tableFrom": "workflow_checkpoints", + "tableTo": "copilot_chats", + "columnsFrom": ["chat_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow_deployment_version": { + "name": "workflow_deployment_version", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "state": { + "name": "state", + "type": "json", + "primaryKey": false, + "notNull": true + }, + "is_active": { + "name": "is_active", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "workflow_deployment_version_workflow_version_unique": { + "name": "workflow_deployment_version_workflow_version_unique", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "version", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_deployment_version_workflow_active_idx": { + "name": "workflow_deployment_version_workflow_active_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "is_active", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_deployment_version_created_at_idx": { + "name": "workflow_deployment_version_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_deployment_version_workflow_id_workflow_id_fk": { + "name": "workflow_deployment_version_workflow_id_workflow_id_fk", + "tableFrom": "workflow_deployment_version", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow_edges": { + "name": "workflow_edges", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source_block_id": { + "name": "source_block_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_block_id": { + "name": "target_block_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source_handle": { + "name": "source_handle", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_handle": { + "name": "target_handle", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workflow_edges_workflow_id_idx": { + "name": "workflow_edges_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_edges_workflow_source_idx": { + "name": "workflow_edges_workflow_source_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "source_block_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_edges_workflow_target_idx": { + "name": "workflow_edges_workflow_target_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "target_block_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_edges_workflow_id_workflow_id_fk": { + "name": "workflow_edges_workflow_id_workflow_id_fk", + "tableFrom": "workflow_edges", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_edges_source_block_id_workflow_blocks_id_fk": { + "name": "workflow_edges_source_block_id_workflow_blocks_id_fk", + "tableFrom": "workflow_edges", + "tableTo": "workflow_blocks", + "columnsFrom": ["source_block_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_edges_target_block_id_workflow_blocks_id_fk": { + "name": "workflow_edges_target_block_id_workflow_blocks_id_fk", + "tableFrom": "workflow_edges", + "tableTo": "workflow_blocks", + "columnsFrom": ["target_block_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow_execution_logs": { + "name": "workflow_execution_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "execution_id": { + "name": "execution_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "state_snapshot_id": { + "name": "state_snapshot_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "deployment_version_id": { + "name": "deployment_version_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "level": { + "name": "level", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'running'" + }, + "trigger": { + "name": "trigger", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "started_at": { + "name": "started_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "ended_at": { + "name": "ended_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "total_duration_ms": { + "name": "total_duration_ms", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "execution_data": { + "name": "execution_data", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "cost": { + "name": "cost", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "cost_total": { + "name": "cost_total", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "models_used": { + "name": "models_used", + "type": "text[]", + "primaryKey": false, + "notNull": false + }, + "files": { + "name": "files", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workflow_execution_logs_workflow_id_idx": { + "name": "workflow_execution_logs_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_execution_logs_state_snapshot_id_idx": { + "name": "workflow_execution_logs_state_snapshot_id_idx", + "columns": [ + { + "expression": "state_snapshot_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_execution_logs_deployment_version_id_idx": { + "name": "workflow_execution_logs_deployment_version_id_idx", + "columns": [ + { + "expression": "deployment_version_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_execution_logs_trigger_idx": { + "name": "workflow_execution_logs_trigger_idx", + "columns": [ + { + "expression": "trigger", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_execution_logs_level_idx": { + "name": "workflow_execution_logs_level_idx", + "columns": [ + { + "expression": "level", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_execution_logs_started_at_idx": { + "name": "workflow_execution_logs_started_at_idx", + "columns": [ + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_execution_logs_execution_id_unique": { + "name": "workflow_execution_logs_execution_id_unique", + "columns": [ + { + "expression": "execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_execution_logs_workflow_started_at_idx": { + "name": "workflow_execution_logs_workflow_started_at_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_execution_logs_workspace_started_at_idx": { + "name": "workflow_execution_logs_workspace_started_at_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_execution_logs_workspace_cost_total_idx": { + "name": "workflow_execution_logs_workspace_cost_total_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "cost_total", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_execution_logs_models_used_idx": { + "name": "workflow_execution_logs_models_used_idx", + "columns": [ + { + "expression": "models_used", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "gin", + "with": {} + }, + "workflow_execution_logs_workspace_ended_at_id_idx": { + "name": "workflow_execution_logs_workspace_ended_at_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "date_trunc('milliseconds', \"ended_at\")", + "asc": true, + "isExpression": true, + "nulls": "last" + }, + { + "expression": "id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_execution_logs_running_started_at_idx": { + "name": "workflow_execution_logs_running_started_at_idx", + "columns": [ + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "status = 'running'", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_execution_logs_workflow_id_workflow_id_fk": { + "name": "workflow_execution_logs_workflow_id_workflow_id_fk", + "tableFrom": "workflow_execution_logs", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + }, + "workflow_execution_logs_workspace_id_workspace_id_fk": { + "name": "workflow_execution_logs_workspace_id_workspace_id_fk", + "tableFrom": "workflow_execution_logs", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_execution_logs_state_snapshot_id_workflow_execution_snapshots_id_fk": { + "name": "workflow_execution_logs_state_snapshot_id_workflow_execution_snapshots_id_fk", + "tableFrom": "workflow_execution_logs", + "tableTo": "workflow_execution_snapshots", + "columnsFrom": ["state_snapshot_id"], + "columnsTo": ["id"], + "onDelete": "no action", + "onUpdate": "no action" + }, + "workflow_execution_logs_deployment_version_id_workflow_deployment_version_id_fk": { + "name": "workflow_execution_logs_deployment_version_id_workflow_deployment_version_id_fk", + "tableFrom": "workflow_execution_logs", + "tableTo": "workflow_deployment_version", + "columnsFrom": ["deployment_version_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow_execution_snapshots": { + "name": "workflow_execution_snapshots", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "state_hash": { + "name": "state_hash", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "state_data": { + "name": "state_data", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workflow_snapshots_workflow_id_idx": { + "name": "workflow_snapshots_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_snapshots_hash_idx": { + "name": "workflow_snapshots_hash_idx", + "columns": [ + { + "expression": "state_hash", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_snapshots_workflow_hash_idx": { + "name": "workflow_snapshots_workflow_hash_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "state_hash", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_snapshots_created_at_idx": { + "name": "workflow_snapshots_created_at_idx", + "columns": [ + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_execution_snapshots_workflow_id_workflow_id_fk": { + "name": "workflow_execution_snapshots_workflow_id_workflow_id_fk", + "tableFrom": "workflow_execution_snapshots", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow_folder": { + "name": "workflow_folder", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "parent_id": { + "name": "parent_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "color": { + "name": "color", + "type": "text", + "primaryKey": false, + "notNull": false, + "default": "'#6B7280'" + }, + "is_expanded": { + "name": "is_expanded", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "locked": { + "name": "locked", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "sort_order": { + "name": "sort_order", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "workflow_folder_user_idx": { + "name": "workflow_folder_user_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_folder_workspace_parent_idx": { + "name": "workflow_folder_workspace_parent_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "parent_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_folder_parent_sort_idx": { + "name": "workflow_folder_parent_sort_idx", + "columns": [ + { + "expression": "parent_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "sort_order", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_folder_archived_at_idx": { + "name": "workflow_folder_archived_at_idx", + "columns": [ + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_folder_workspace_archived_partial_idx": { + "name": "workflow_folder_workspace_archived_partial_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"workflow_folder\".\"archived_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_folder_user_id_user_id_fk": { + "name": "workflow_folder_user_id_user_id_fk", + "tableFrom": "workflow_folder", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_folder_workspace_id_workspace_id_fk": { + "name": "workflow_folder_workspace_id_workspace_id_fk", + "tableFrom": "workflow_folder", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow_mcp_server": { + "name": "workflow_mcp_server", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "is_public": { + "name": "is_public", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workflow_mcp_server_workspace_id_idx": { + "name": "workflow_mcp_server_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_mcp_server_created_by_idx": { + "name": "workflow_mcp_server_created_by_idx", + "columns": [ + { + "expression": "created_by", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_mcp_server_deleted_at_idx": { + "name": "workflow_mcp_server_deleted_at_idx", + "columns": [ + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_mcp_server_workspace_deleted_partial_idx": { + "name": "workflow_mcp_server_workspace_deleted_partial_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"workflow_mcp_server\".\"deleted_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_mcp_server_workspace_id_workspace_id_fk": { + "name": "workflow_mcp_server_workspace_id_workspace_id_fk", + "tableFrom": "workflow_mcp_server", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_mcp_server_created_by_user_id_fk": { + "name": "workflow_mcp_server_created_by_user_id_fk", + "tableFrom": "workflow_mcp_server", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow_mcp_tool": { + "name": "workflow_mcp_tool", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "server_id": { + "name": "server_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "tool_name": { + "name": "tool_name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "tool_description": { + "name": "tool_description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "parameter_schema": { + "name": "parameter_schema", + "type": "json", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workflow_mcp_tool_server_id_idx": { + "name": "workflow_mcp_tool_server_id_idx", + "columns": [ + { + "expression": "server_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_mcp_tool_workflow_id_idx": { + "name": "workflow_mcp_tool_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_mcp_tool_server_workflow_unique": { + "name": "workflow_mcp_tool_server_workflow_unique", + "columns": [ + { + "expression": "server_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"workflow_mcp_tool\".\"archived_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_mcp_tool_archived_at_partial_idx": { + "name": "workflow_mcp_tool_archived_at_partial_idx", + "columns": [ + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"workflow_mcp_tool\".\"archived_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_mcp_tool_server_id_workflow_mcp_server_id_fk": { + "name": "workflow_mcp_tool_server_id_workflow_mcp_server_id_fk", + "tableFrom": "workflow_mcp_tool", + "tableTo": "workflow_mcp_server", + "columnsFrom": ["server_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_mcp_tool_workflow_id_workflow_id_fk": { + "name": "workflow_mcp_tool_workflow_id_workflow_id_fk", + "tableFrom": "workflow_mcp_tool", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow_schedule": { + "name": "workflow_schedule", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "deployment_version_id": { + "name": "deployment_version_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "block_id": { + "name": "block_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "cron_expression": { + "name": "cron_expression", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "next_run_at": { + "name": "next_run_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "last_ran_at": { + "name": "last_ran_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "last_queued_at": { + "name": "last_queued_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "trigger_type": { + "name": "trigger_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "timezone": { + "name": "timezone", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'UTC'" + }, + "failed_count": { + "name": "failed_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "infra_retry_count": { + "name": "infra_retry_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "last_failed_at": { + "name": "last_failed_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'workflow'" + }, + "job_title": { + "name": "job_title", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "prompt": { + "name": "prompt", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "lifecycle": { + "name": "lifecycle", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'persistent'" + }, + "success_condition": { + "name": "success_condition", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "max_runs": { + "name": "max_runs", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "run_count": { + "name": "run_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "source_chat_id": { + "name": "source_chat_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_task_name": { + "name": "source_task_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_user_id": { + "name": "source_user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_workspace_id": { + "name": "source_workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "job_history": { + "name": "job_history", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workflow_schedule_workflow_block_deployment_unique": { + "name": "workflow_schedule_workflow_block_deployment_unique", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "block_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deployment_version_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"workflow_schedule\".\"archived_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_schedule_workflow_deployment_idx": { + "name": "workflow_schedule_workflow_deployment_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deployment_version_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_schedule_archived_at_partial_idx": { + "name": "workflow_schedule_archived_at_partial_idx", + "columns": [ + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"workflow_schedule\".\"archived_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_workflow_schedule_on_source_workspace_id_source_t_c07f3bba6": { + "name": "idx_workflow_schedule_on_source_workspace_id_source_t_c07f3bba6", + "columns": [ + { + "expression": "source_workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "source_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "archived_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_schedule_due_workflow_idx": { + "name": "workflow_schedule_due_workflow_idx", + "columns": [ + { + "expression": "next_run_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "last_queued_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deployment_version_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"workflow_schedule\".\"archived_at\" IS NULL AND \"workflow_schedule\".\"status\" NOT IN ('disabled', 'completed') AND (\"workflow_schedule\".\"source_type\" = 'workflow' OR \"workflow_schedule\".\"source_type\" IS NULL)", + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_schedule_due_job_idx": { + "name": "workflow_schedule_due_job_idx", + "columns": [ + { + "expression": "next_run_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "last_queued_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"workflow_schedule\".\"archived_at\" IS NULL AND \"workflow_schedule\".\"status\" NOT IN ('disabled', 'completed') AND \"workflow_schedule\".\"source_type\" = 'job'", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_schedule_workflow_id_workflow_id_fk": { + "name": "workflow_schedule_workflow_id_workflow_id_fk", + "tableFrom": "workflow_schedule", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_schedule_deployment_version_id_workflow_deployment_version_id_fk": { + "name": "workflow_schedule_deployment_version_id_workflow_deployment_version_id_fk", + "tableFrom": "workflow_schedule", + "tableTo": "workflow_deployment_version", + "columnsFrom": ["deployment_version_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_schedule_source_user_id_user_id_fk": { + "name": "workflow_schedule_source_user_id_user_id_fk", + "tableFrom": "workflow_schedule", + "tableTo": "user", + "columnsFrom": ["source_user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workflow_schedule_source_workspace_id_workspace_id_fk": { + "name": "workflow_schedule_source_workspace_id_workspace_id_fk", + "tableFrom": "workflow_schedule", + "tableTo": "workspace", + "columnsFrom": ["source_workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workflow_subflows": { + "name": "workflow_subflows", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "type": { + "name": "type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "config": { + "name": "config", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workflow_subflows_workflow_id_idx": { + "name": "workflow_subflows_workflow_id_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workflow_subflows_workflow_type_idx": { + "name": "workflow_subflows_workflow_type_idx", + "columns": [ + { + "expression": "workflow_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workflow_subflows_workflow_id_workflow_id_fk": { + "name": "workflow_subflows_workflow_id_workflow_id_fk", + "tableFrom": "workflow_subflows", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workspace": { + "name": "workspace", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "color": { + "name": "color", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'#33C482'" + }, + "logo_url": { + "name": "logo_url", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "owner_id": { + "name": "owner_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "organization_id": { + "name": "organization_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "workspace_mode": { + "name": "workspace_mode", + "type": "workspace_mode", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'grandfathered_shared'" + }, + "billed_account_user_id": { + "name": "billed_account_user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "allow_personal_api_keys": { + "name": "allow_personal_api_keys", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "inbox_enabled": { + "name": "inbox_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "inbox_address": { + "name": "inbox_address", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "inbox_provider_id": { + "name": "inbox_provider_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workspace_owner_id_idx": { + "name": "workspace_owner_id_idx", + "columns": [ + { + "expression": "owner_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_organization_id_idx": { + "name": "workspace_organization_id_idx", + "columns": [ + { + "expression": "organization_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_mode_idx": { + "name": "workspace_mode_idx", + "columns": [ + { + "expression": "workspace_mode", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workspace_owner_id_user_id_fk": { + "name": "workspace_owner_id_user_id_fk", + "tableFrom": "workspace", + "tableTo": "user", + "columnsFrom": ["owner_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workspace_organization_id_organization_id_fk": { + "name": "workspace_organization_id_organization_id_fk", + "tableFrom": "workspace", + "tableTo": "organization", + "columnsFrom": ["organization_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + }, + "workspace_billed_account_user_id_user_id_fk": { + "name": "workspace_billed_account_user_id_user_id_fk", + "tableFrom": "workspace", + "tableTo": "user", + "columnsFrom": ["billed_account_user_id"], + "columnsTo": ["id"], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workspace_byok_keys": { + "name": "workspace_byok_keys", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider_id": { + "name": "provider_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "encrypted_api_key": { + "name": "encrypted_api_key", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workspace_byok_provider_unique": { + "name": "workspace_byok_provider_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "provider_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_byok_workspace_idx": { + "name": "workspace_byok_workspace_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workspace_byok_keys_workspace_id_workspace_id_fk": { + "name": "workspace_byok_keys_workspace_id_workspace_id_fk", + "tableFrom": "workspace_byok_keys", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workspace_byok_keys_created_by_user_id_fk": { + "name": "workspace_byok_keys_created_by_user_id_fk", + "tableFrom": "workspace_byok_keys", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workspace_environment": { + "name": "workspace_environment", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "variables": { + "name": "variables", + "type": "json", + "primaryKey": false, + "notNull": true, + "default": "'{}'" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workspace_environment_workspace_unique": { + "name": "workspace_environment_workspace_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workspace_environment_workspace_id_workspace_id_fk": { + "name": "workspace_environment_workspace_id_workspace_id_fk", + "tableFrom": "workspace_environment", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workspace_file": { + "name": "workspace_file", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "key": { + "name": "key", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size": { + "name": "size", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "type": { + "name": "type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "uploaded_by": { + "name": "uploaded_by", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "uploaded_at": { + "name": "uploaded_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workspace_file_workspace_id_idx": { + "name": "workspace_file_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_file_key_idx": { + "name": "workspace_file_key_idx", + "columns": [ + { + "expression": "key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_file_deleted_at_idx": { + "name": "workspace_file_deleted_at_idx", + "columns": [ + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_file_workspace_deleted_partial_idx": { + "name": "workspace_file_workspace_deleted_partial_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"workspace_file\".\"deleted_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workspace_file_workspace_id_workspace_id_fk": { + "name": "workspace_file_workspace_id_workspace_id_fk", + "tableFrom": "workspace_file", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workspace_file_uploaded_by_user_id_fk": { + "name": "workspace_file_uploaded_by_user_id_fk", + "tableFrom": "workspace_file", + "tableTo": "user", + "columnsFrom": ["uploaded_by"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "workspace_file_key_unique": { + "name": "workspace_file_key_unique", + "nullsNotDistinct": false, + "columns": ["key"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workspace_file_folders": { + "name": "workspace_file_folders", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "parent_id": { + "name": "parent_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sort_order": { + "name": "sort_order", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workspace_file_folders_workspace_parent_idx": { + "name": "workspace_file_folders_workspace_parent_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "parent_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_file_folders_parent_sort_idx": { + "name": "workspace_file_folders_parent_sort_idx", + "columns": [ + { + "expression": "parent_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "sort_order", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_file_folders_deleted_at_idx": { + "name": "workspace_file_folders_deleted_at_idx", + "columns": [ + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_file_folders_workspace_deleted_partial_idx": { + "name": "workspace_file_folders_workspace_deleted_partial_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"workspace_file_folders\".\"deleted_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_file_folders_workspace_parent_name_active_unique": { + "name": "workspace_file_folders_workspace_parent_name_active_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "coalesce(\"parent_id\", '')", + "asc": true, + "isExpression": true, + "nulls": "last" + }, + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"workspace_file_folders\".\"deleted_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workspace_file_folders_user_id_user_id_fk": { + "name": "workspace_file_folders_user_id_user_id_fk", + "tableFrom": "workspace_file_folders", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workspace_file_folders_workspace_id_workspace_id_fk": { + "name": "workspace_file_folders_workspace_id_workspace_id_fk", + "tableFrom": "workspace_file_folders", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workspace_file_folders_parent_id_workspace_file_folders_id_fk": { + "name": "workspace_file_folders_parent_id_workspace_file_folders_id_fk", + "tableFrom": "workspace_file_folders", + "tableTo": "workspace_file_folders", + "columnsFrom": ["parent_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workspace_files": { + "name": "workspace_files", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "key": { + "name": "key", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "folder_id": { + "name": "folder_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "context": { + "name": "context", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "chat_id": { + "name": "chat_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "original_name": { + "name": "original_name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "content_type": { + "name": "content_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size": { + "name": "size", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "uploaded_at": { + "name": "uploaded_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workspace_files_key_active_unique": { + "name": "workspace_files_key_active_unique", + "columns": [ + { + "expression": "key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"workspace_files\".\"deleted_at\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_files_workspace_folder_name_active_unique": { + "name": "workspace_files_workspace_folder_name_active_unique", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "coalesce(\"folder_id\", '')", + "asc": true, + "isExpression": true, + "nulls": "last" + }, + { + "expression": "original_name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"workspace_files\".\"deleted_at\" IS NULL AND \"workspace_files\".\"context\" = 'workspace' AND \"workspace_files\".\"workspace_id\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_files_chat_display_name_unique": { + "name": "workspace_files_chat_display_name_unique", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "display_name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"workspace_files\".\"context\" = 'mothership' AND \"workspace_files\".\"chat_id\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_files_key_idx": { + "name": "workspace_files_key_idx", + "columns": [ + { + "expression": "key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_files_user_id_idx": { + "name": "workspace_files_user_id_idx", + "columns": [ + { + "expression": "user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_files_workspace_id_idx": { + "name": "workspace_files_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_files_folder_id_idx": { + "name": "workspace_files_folder_id_idx", + "columns": [ + { + "expression": "folder_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_files_context_idx": { + "name": "workspace_files_context_idx", + "columns": [ + { + "expression": "context", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_files_chat_id_idx": { + "name": "workspace_files_chat_id_idx", + "columns": [ + { + "expression": "chat_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_files_deleted_at_idx": { + "name": "workspace_files_deleted_at_idx", + "columns": [ + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_files_workspace_deleted_partial_idx": { + "name": "workspace_files_workspace_deleted_partial_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"workspace_files\".\"deleted_at\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workspace_files_user_id_user_id_fk": { + "name": "workspace_files_user_id_user_id_fk", + "tableFrom": "workspace_files", + "tableTo": "user", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workspace_files_workspace_id_workspace_id_fk": { + "name": "workspace_files_workspace_id_workspace_id_fk", + "tableFrom": "workspace_files", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workspace_files_folder_id_workspace_file_folders_id_fk": { + "name": "workspace_files_folder_id_workspace_file_folders_id_fk", + "tableFrom": "workspace_files", + "tableTo": "workspace_file_folders", + "columnsFrom": ["folder_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + }, + "workspace_files_chat_id_copilot_chats_id_fk": { + "name": "workspace_files_chat_id_copilot_chats_id_fk", + "tableFrom": "workspace_files", + "tableTo": "copilot_chats", + "columnsFrom": ["chat_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workspace_notification_delivery": { + "name": "workspace_notification_delivery", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "subscription_id": { + "name": "subscription_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "workflow_id": { + "name": "workflow_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "execution_id": { + "name": "execution_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "notification_delivery_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "attempts": { + "name": "attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "last_attempt_at": { + "name": "last_attempt_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "next_attempt_at": { + "name": "next_attempt_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "response_status": { + "name": "response_status", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "response_body": { + "name": "response_body", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "error_message": { + "name": "error_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workspace_notification_delivery_subscription_id_idx": { + "name": "workspace_notification_delivery_subscription_id_idx", + "columns": [ + { + "expression": "subscription_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_notification_delivery_execution_id_idx": { + "name": "workspace_notification_delivery_execution_id_idx", + "columns": [ + { + "expression": "execution_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_notification_delivery_status_idx": { + "name": "workspace_notification_delivery_status_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_notification_delivery_next_attempt_idx": { + "name": "workspace_notification_delivery_next_attempt_idx", + "columns": [ + { + "expression": "next_attempt_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workspace_notification_delivery_subscription_id_workspace_notification_subscription_id_fk": { + "name": "workspace_notification_delivery_subscription_id_workspace_notification_subscription_id_fk", + "tableFrom": "workspace_notification_delivery", + "tableTo": "workspace_notification_subscription", + "columnsFrom": ["subscription_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workspace_notification_delivery_workflow_id_workflow_id_fk": { + "name": "workspace_notification_delivery_workflow_id_workflow_id_fk", + "tableFrom": "workspace_notification_delivery", + "tableTo": "workflow", + "columnsFrom": ["workflow_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.workspace_notification_subscription": { + "name": "workspace_notification_subscription", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "workspace_id": { + "name": "workspace_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "notification_type": { + "name": "notification_type", + "type": "notification_type", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "workflow_ids": { + "name": "workflow_ids", + "type": "text[]", + "primaryKey": false, + "notNull": true, + "default": "'{}'::text[]" + }, + "all_workflows": { + "name": "all_workflows", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "level_filter": { + "name": "level_filter", + "type": "text[]", + "primaryKey": false, + "notNull": true, + "default": "ARRAY['info', 'error']::text[]" + }, + "trigger_filter": { + "name": "trigger_filter", + "type": "text[]", + "primaryKey": false, + "notNull": true, + "default": "ARRAY['api', 'webhook', 'schedule', 'manual', 'chat']::text[]" + }, + "include_final_output": { + "name": "include_final_output", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "include_trace_spans": { + "name": "include_trace_spans", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "include_rate_limits": { + "name": "include_rate_limits", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "include_usage_data": { + "name": "include_usage_data", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "webhook_config": { + "name": "webhook_config", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "email_recipients": { + "name": "email_recipients", + "type": "text[]", + "primaryKey": false, + "notNull": false + }, + "slack_config": { + "name": "slack_config", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "alert_config": { + "name": "alert_config", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "last_alert_at": { + "name": "last_alert_at", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "active": { + "name": "active", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "workspace_notification_workspace_id_idx": { + "name": "workspace_notification_workspace_id_idx", + "columns": [ + { + "expression": "workspace_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_notification_active_idx": { + "name": "workspace_notification_active_idx", + "columns": [ + { + "expression": "active", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "workspace_notification_type_idx": { + "name": "workspace_notification_type_idx", + "columns": [ + { + "expression": "notification_type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "workspace_notification_subscription_workspace_id_workspace_id_fk": { + "name": "workspace_notification_subscription_workspace_id_workspace_id_fk", + "tableFrom": "workspace_notification_subscription", + "tableTo": "workspace", + "columnsFrom": ["workspace_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "workspace_notification_subscription_created_by_user_id_fk": { + "name": "workspace_notification_subscription_created_by_user_id_fk", + "tableFrom": "workspace_notification_subscription", + "tableTo": "user", + "columnsFrom": ["created_by"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.a2a_task_status": { + "name": "a2a_task_status", + "schema": "public", + "values": [ + "submitted", + "working", + "input-required", + "completed", + "failed", + "canceled", + "rejected", + "auth-required", + "unknown" + ] + }, + "public.academy_cert_status": { + "name": "academy_cert_status", + "schema": "public", + "values": ["active", "revoked", "expired"] + }, + "public.billing_blocked_reason": { + "name": "billing_blocked_reason", + "schema": "public", + "values": ["payment_failed", "dispute"] + }, + "public.billing_entity_type": { + "name": "billing_entity_type", + "schema": "public", + "values": ["user", "organization"] + }, + "public.chat_type": { + "name": "chat_type", + "schema": "public", + "values": ["mothership", "copilot"] + }, + "public.copilot_async_tool_status": { + "name": "copilot_async_tool_status", + "schema": "public", + "values": ["pending", "running", "completed", "failed", "cancelled", "delivered"] + }, + "public.copilot_run_status": { + "name": "copilot_run_status", + "schema": "public", + "values": ["active", "paused_waiting_for_tool", "resuming", "complete", "error", "cancelled"] + }, + "public.credential_member_role": { + "name": "credential_member_role", + "schema": "public", + "values": ["admin", "member"] + }, + "public.credential_member_status": { + "name": "credential_member_status", + "schema": "public", + "values": ["active", "pending", "revoked"] + }, + "public.credential_set_invitation_status": { + "name": "credential_set_invitation_status", + "schema": "public", + "values": ["pending", "accepted", "expired", "cancelled"] + }, + "public.credential_set_member_status": { + "name": "credential_set_member_status", + "schema": "public", + "values": ["active", "pending", "revoked"] + }, + "public.credential_type": { + "name": "credential_type", + "schema": "public", + "values": ["oauth", "env_workspace", "env_personal", "service_account"] + }, + "public.data_drain_cadence": { + "name": "data_drain_cadence", + "schema": "public", + "values": ["hourly", "daily"] + }, + "public.data_drain_destination": { + "name": "data_drain_destination", + "schema": "public", + "values": ["s3", "gcs", "azure_blob", "datadog", "bigquery", "snowflake", "webhook"] + }, + "public.data_drain_run_status": { + "name": "data_drain_run_status", + "schema": "public", + "values": ["running", "success", "failed"] + }, + "public.data_drain_run_trigger": { + "name": "data_drain_run_trigger", + "schema": "public", + "values": ["cron", "manual"] + }, + "public.data_drain_source": { + "name": "data_drain_source", + "schema": "public", + "values": ["workflow_logs", "job_logs", "audit_logs", "copilot_chats", "copilot_runs"] + }, + "public.execution_large_value_reference_source": { + "name": "execution_large_value_reference_source", + "schema": "public", + "values": ["execution_log", "paused_snapshot"] + }, + "public.invitation_kind": { + "name": "invitation_kind", + "schema": "public", + "values": ["organization", "workspace"] + }, + "public.invitation_membership_intent": { + "name": "invitation_membership_intent", + "schema": "public", + "values": ["internal", "external"] + }, + "public.invitation_status": { + "name": "invitation_status", + "schema": "public", + "values": ["pending", "accepted", "rejected", "cancelled", "expired"] + }, + "public.notification_delivery_status": { + "name": "notification_delivery_status", + "schema": "public", + "values": ["pending", "in_progress", "success", "failed"] + }, + "public.notification_type": { + "name": "notification_type", + "schema": "public", + "values": ["webhook", "email", "slack"] + }, + "public.permission_type": { + "name": "permission_type", + "schema": "public", + "values": ["admin", "write", "read"] + }, + "public.template_creator_type": { + "name": "template_creator_type", + "schema": "public", + "values": ["user", "organization"] + }, + "public.template_status": { + "name": "template_status", + "schema": "public", + "values": ["pending", "approved", "rejected"] + }, + "public.usage_log_category": { + "name": "usage_log_category", + "schema": "public", + "values": ["model", "fixed", "tool"] + }, + "public.usage_log_source": { + "name": "usage_log_source", + "schema": "public", + "values": [ + "workflow", + "wand", + "copilot", + "workspace-chat", + "mcp_copilot", + "mothership_block", + "knowledge-base", + "voice-input", + "enrichment" + ] + }, + "public.workspace_mode": { + "name": "workspace_mode", + "schema": "public", + "values": ["personal", "organization", "grandfathered_shared"] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/db/migrations/meta/_journal.json b/packages/db/migrations/meta/_journal.json index f6867cb67a..92567ac3c3 100644 --- a/packages/db/migrations/meta/_journal.json +++ b/packages/db/migrations/meta/_journal.json @@ -1562,6 +1562,13 @@ "when": 1780420540516, "tag": "0223_lowly_shocker", "breakpoints": true + }, + { + "idx": 224, + "version": "7", + "when": 1780472914844, + "tag": "0224_table_import_columns", + "breakpoints": true } ] } diff --git a/packages/db/schema.ts b/packages/db/schema.ts index 00b84c3049..edc570760a 100644 --- a/packages/db/schema.ts +++ b/packages/db/schema.ts @@ -3270,6 +3270,16 @@ export const userTableDefinitions = pgTable( maxRows: integer('max_rows').notNull().default(10000), rowCount: integer('row_count').notNull().default(0), archivedAt: timestamp('archived_at'), + /** + * Async-import state. NULL = a normal table (never imported in the background). + * `'importing'` hides rows until the load completes; `'ready'` reveals them; + * `'failed'` surfaces a partial import. See `apps/sim/lib/table/import-runner.ts`. + */ + importStatus: text('import_status'), + importId: text('import_id'), + importError: text('import_error'), + importRowsProcessed: integer('import_rows_processed').notNull().default(0), + importStartedAt: timestamp('import_started_at'), createdBy: text('created_by') .notNull() .references(() => user.id, { onDelete: 'cascade' }), diff --git a/scripts/check-api-validation-contracts.ts b/scripts/check-api-validation-contracts.ts index 83cf378eb3..f84af741a7 100644 --- a/scripts/check-api-validation-contracts.ts +++ b/scripts/check-api-validation-contracts.ts @@ -9,8 +9,8 @@ const QUERY_HOOKS_DIR = path.join(ROOT, 'apps/sim/hooks/queries') const SELECTOR_HOOKS_DIR = path.join(ROOT, 'apps/sim/hooks/selectors') const BASELINE = { - totalRoutes: 762, - zodRoutes: 762, + totalRoutes: 765, + zodRoutes: 765, nonZodRoutes: 0, } as const