improvement(external-endpoints): v2 versions with clean signatures + updated docs based on openapi spec (#5273)

* v0.6.29: login improvements, posthog telemetry (#4026)

* feat(posthog): Add tracking on mothership abort (#4023)

Co-authored-by: Theodore Li <theo@sim.ai>

* fix(login): fix captcha headers for manual login  (#4025)

* fix(signup): fix turnstile key loading

* fix(login): fix captcha header passing

* Catch user already exists, remove login form captcha

* improvement(external-endpoints): v2 versions with clean signatures + updated docs

* feat(usage): accept X-API-Key on usage-logs list + export

/api/users/me/usage-logs and /export now use checkHybridAuth — the same
auth /api/users/me/usage-limits already accepts — so external monitors
can read summary.bySourceCredits (the source breakdown of usage-limits'
aggregate currentPeriodCost) instead of estimating Copilot spend by
subtraction. Workspace-scoped keys are pinned to their own workspace's
slice of the ledger: the filter defaults to the key's workspace and an
explicit mismatch 403s. Both endpoints documented in openapi-core.json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* feat(billing): dedicated v2 usage endpoints; keep internal usage routes session-only

Replaces the earlier X-API-Key enablement on /api/users/me/usage-logs
with a dedicated public surface, so the internal Billing-settings
endpoints can evolve with the UI while external monitors get a stable
versioned contract:

- GET /api/v2/billing/usage — current-billing-period summary with
  bySourceCredits (the source breakdown external monitors need to watch
  e.g. Copilot consumption without estimating by subtraction), plus
  limitCredits and plan
- GET /api/v2/billing/usage/logs — cursor-paged credit ledger in the v2
  envelope
- workspace-scoped keys are pinned to their own workspace's slice;
  personal keys read the account ledger

The public wire is credits-only: usage-logs rows now carry a hasCost
boolean instead of dollarCost (the Billing UI only needed the >0
signal), and the rateLimit block is removed from the usage-limits
response and docs (deploy-modal tab relabeled accordingly).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* feat(docs): validate OpenAPI specs against the Zod contracts in CI

The specs in apps/docs are hand-authored because they carry what Zod
never defines — error envelopes, status codes, prose, examples — so
they can't be generated; check:openapi validates them instead:

- spec integrity: $refs resolve, operationIds unique, 2xx documented,
  no orphaned component schemas
- v2 conventions: every /api/v2 operation documents 401 + 429 and every
  4xx/5xx resolves to the canonical { error: { code, message } } envelope
- contract cross-check: contracts are auto-discovered from
  lib/api/contracts/v2 (each carries its method + path); doc<->contract
  coverage both ways, query/body/response field diffs via z.toJSONSchema
- examples: documented request/response examples must parse with the
  matching contract's actual Zod schemas

First run caught real drift, fixed here: 16 stale orphaned schemas in
the core spec, the v2 billing ops referencing v1-shaped error
components, deploy/rollback examples missing the required nullable
lifecycle keys, CreateTableBody missing folderId, a legacy-grammar
delete-rows example, and four knowledge document ops missing their
required workspaceId query param.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* fix(docs): recursive field diff in check:openapi + the deep drift it found

A mutation test showed the doc<->contract field diff only compared
top-level properties, so a typo inside the { data } envelope passed.
The diff now descends through matching object properties and array
items (both sides must expose a property set — passthrough contracts
and prose-only docs end the descent instead of false-positive), with
the Zod JSON-schema root doubling as the $defs context.

Deep drift it immediately caught, fixed here: select-column config
(options/multiple) missing from every tables column schema, AddColumnBody
hand-rolling a third column shape (now composed from ColumnInput, with
position/workflowGroupId as the per-op extensions the contracts actually
admit), chunking strategyOptions undocumented, and the deployment
lifecycle fields (activeDeployment/latestDeploymentAttempt) missing from
DeploymentState.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* fix(security): close the triggerType rate-limit bypass on workflow execute

Caller-supplied triggerType flowed unchecked into preprocessExecution,
whose checkRateLimit default turns OFF for 'manual'/'chat' — so any
API-key caller, and any anonymous public-API caller billed to the
workspace owner, could execute unthrottled by sending
{"triggerType":"manual"} (async runs also skipped the worker-side check
via admissionCompleted). External callers may now only send the
redundant 'api' value; internal JWT callers ('workflow'/'mcp') are
unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* refactor(execution): extract enqueue/status/cancel into shared libs

Prepares the v2 execution surface: handleAsyncExecution's queue logic
moves to lib/workflows/executor/enqueue-execution.ts (slot/claim
semantics encoded in a discriminated outcome, not HTTP statuses), the
execution-status read to execution-status.ts, and the order-sensitive
cancel machinery to lib/execution/cancel-workflow-execution.ts. The v1
routes re-render identically — their suites pass unmodified.

Also: preprocessExecution gains rateLimitCounter ('sync'|'async') and
its 429 now carries code RATE_LIMIT_EXCEEDED + retryAfterMs (previously
indistinguishable from the concurrency 429 and Retry-After was
discarded); and the duplicate cancel contract in contracts/logs.ts is
unified on the full 5-value reason enum — its narrower copy made
requestJson throw a client ZodError when cancelling a paused HITL run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* feat(execution): callable execution service + structured error classifier

executeWorkflowService composes the same libs the v1 route holds inline
(call-chain guard, execution-id claim, LoggingSession, preprocessing,
deployed-state load + file-field processing, timeout-bound
executeWorkflowCore, output hydration/compaction) for the deployed-state
caller class — the seam the v2 execute route and in-process internal
callers share, making the HTTP endpoint syntactic sugar.

classifyExecutionError stops discarding the block context that
buildBlockExecutionError already attaches at throw sites: failed runs
now yield {message, code, blockId, blockName, blockType} with a stable
append-only code enum (TIMEOUT/CANCELLED/USAGE_LIMIT_EXCEEDED/
INVALID_INPUT/BLOCK_EXECUTION_FAILED/CHILD_WORKFLOW_FAILED/
OUTPUT_TOO_LARGE/EXECUTION_FAILED), so callers route on error class
instead of substring-matching messages — the single place raw errors
are interpreted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* feat(api): POST /api/v2/workflows/[id]/execute

Thin route over executeWorkflowService: X-API-Key or anonymous
public-API auth (sync/stream only for anonymous), strict body with
body-flag async (no mode headers on v2), SSE passthrough for stream,
and the execution resource response — executionId always present,
in-band run failures are status:'failed' with the structured
{message, code, blockId, blockName, blockType} error, sync timeout is
status:'failed' + TIMEOUT instead of v1's 408, and a Response block's
payload stays inside output (authors never control response
status/headers on this origin). Async debits the async bucket and the
202 statusUrl points at the v2 executions resource. Adds
CLIENT_CLOSED_REQUEST/SERVICE_UNAVAILABLE to the v2 error codes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* feat(api): v2 executions status + cancel with queued backfill

GET /api/v2/workflows/[id]/executions/[executionId] is the single
status URL for sync and async runs: before the async worker writes the
durable log row, status is backfilled from the job queue (deterministic
job id) as 'queued'/'running' — closing v1's 202-to-pickup 404 window —
and failed runs carry the structured error object. POST .../cancel
renders the shared cancellation lib in the v2 envelope with the
tightened 5-value reason enum. Both authenticate via the shared
resolveV2WorkflowAccess (X-API-Key, authz masked as 404,
allowPersonalApiKeys honored).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* feat(execution): workflow tool + MCP bridge run in-process

workflow_executor (workflow-as-agent-tool) short-circuits in executeTool
through WorkflowBlockHandler — the same invocation boundary canvas child
workflows use — mirroring the deployed_block_executor precedent. The
MCP serve bridge calls executeWorkflowService directly instead of
fetching its own execute endpoint; deployment-version pinning, MCP
response-size rejection, and the actor override become typed options
instead of header sniffing. Both callers drop the double admission slot
and duplicate top-level log row the HTTP hop cost, and failed child
runs now surface the structured error + child executionId so parents
and MCP clients can route on error class and hand providers a
reproducible handle.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* feat(infra): CORS + CSP coverage for the v2 execute path

/api/v2/workflows/:id/execute gets the same wildcard-origin,
credential-free CORS policy as v1 (the default credentialed policy
would block browser API-key calls and open a cookie CSRF surface) with
X-Sim-Stream-Protocol allowed and no X-Execution-Mode (async is
body-selected on v2), plus the COEP/COOP/CSP header block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* feat(ui): deploy modal + copilot advertise the v2 execute surface

All 20 API-tab snippets move to POST /api/v2/workflows/{id}/execute with
the nested {"input": ...} body, async as the "async": true body flag
(X-Execution-Mode gone), status polling against the v2 executions
resource, the third tab renamed Usage and pointed at
/api/v2/billing/usage, and {data} envelope unwraps in the printed
responses. Fixes the latent baseUrl derivation
(endpoint.split('/api/workflows/')) that would have silently built
garbage URLs under a v2 endpoint, and deletes dead code (exampleCommand
across 3 sites, getAsyncExampleTitle). Copilot deploy/manage/serializer
endpoint builders and the api_trigger bestPractices example follow (the
latter also drops its hardcoded staging host).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* docs(api): document the v2 execution surface

Adds execute, execution status, and cancel to openapi-v2-workflows.json
with the structured ExecutionError schema (append-only code enum + block
attribution) and the ExecutionResource contract, documenting the rules
that differ from v1: modes are body-selected, a failed run is HTTP 200
with status 'failed', an executionId always means data (never the error
envelope), queued status is visible immediately, and Response-block
payloads stay inside output. Registers the three pages in the generated
workflows meta.json and bumps the route-count baseline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* feat(api): gate the whole /api/v2 surface behind one flag; UI stays on v1

Every v2 route now runs exactly one check immediately after auth —
v2ApiGateError — and answers 404 when the `v2-api` flag is off, so the
surface is invisible until it is deliberately rolled out. The gate is
keyed on userId only: a workspace/org-keyed check would have to read
membership for a caller-supplied id before authorization runs, and its
404-vs-403 split would leak cohort membership (the trap the per-domain
table gate worked around by running late). The two executions routes
inherit it from the shared access resolver; the tables-specific gate is
removed so no route checks twice.

`tables-v2-api` stays, now gating only the internal predicate-grammar
route /api/table/[tableId]/query — note v2 tables routes move to the
unified flag, so enabling them is a `v2-api` decision now.

Reverts the deploy modal, copilot handlers, and api_trigger example to
the v1 execute endpoint: v1 works unchanged, and the UI must not
advertise a surface most users would get a 404 from.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz

* fix(executor): restore child-cost aggregation dropped by the staging merge

Staging's custom-block rewrite deleted `aggregateChildCost` from
workflow-handler.ts, and git merged that file cleanly — but this branch's
workflow-tool-runner.ts, added for the v2 execute migration, still imports it.
A silent semantic conflict: no marker, broken build.

Taking staging's rewrite is correct, so the helper is defined locally in its
one remaining consumer rather than resurrected in the file staging just
rewrote. Same four lines over the still-exported `calculateCostSummary`, so a
failed child workflow keeps billing the hosted-key spend it consumed instead
of reporting $0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(tables): make lib/table/orchestration the single implementation (#6134)

* refactor(orchestration): move the shared error contract out of lib/workflows

OrchestrationErrorCode and statusForOrchestrationError are the contract every
lib/[resource]/orchestration module returns against, but they lived inside the
workflows module, so resource-neutral code (lib/folders) already had to import
from a workflow path. Moved to lib/core/orchestration/types.

Adds a 'locked' class mapping to 423. Both tables and workflows have a lock
that forbids a mutation, and each caller was translating that to a status
itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(tables): make lib/table/orchestration the single implementation

Column update was implemented four times — the UI route, v1, v2, and the
copilot table tool — each calling the same column services but owning its own
guards, error mapping, and audit. The copies had drifted, and the drift was the
bug: v2 was missing both guards, only the copilot copy minted stable option
ids, and only v1/v2 audited.

performUpdateTableColumn, performDeleteTable, and performDeleteTableRow now own
that logic; all ten call sites reduce to auth, parse, call, render. The guards
are asserted once in lib/table/orchestration rather than four times against
four routes.

Behavior this consolidates, previously true on only some paths:

- The typeChanging guard. updateColumnType early-returns on an unchanged type
  and drops any options sent with it, so restating the current type alongside
  new options silently discarded them. v2 had no guard at all and, since its
  contract shares v1's body schema, accepted options and ignored them.
- The select-unique guard. Each write is its own locked transaction, so a
  rename or type change paired with a constraint write that is going to fail
  commits first and then throws, half-applying the schema change.
- Stable select-option ids. Cells reference the option id, so an edit that
  re-sends an option by name has to reuse it or every cell holding it is
  orphaned. Only the copilot path did this; normalizeSelectOptionsInput moves to
  lib/table/select-options and now covers every caller. It preserves a supplied
  id, so it is a no-op for the fully-formed options the HTTP contracts accept.
- required forwarded into the type and options writes, so a conversion
  validates against the constraint the same request is setting.
- An audit on every successful update. The UI route and the copilot tool
  emitted none.
- Single-row delete through the row service. v2 did a raw db.delete, skipping
  assertRowDelete and deleteOrderedRow, so a delete-locked table returned 200
  and the row-count bookkeeping never ran.
- The delete actor handed to deleteTable, which audits only when a row was
  actually archived. v1 and v2 omitted it and audited themselves outside that
  check, emitting TABLE_DELETED for a no-op delete of an archived table.

Failure classes come back as OrchestrationErrorCode; v2 renders them through a
new v2ErrorForOrchestration, mirroring statusForOrchestrationError on the v1
and UI surfaces, so a given failure maps to the same status everywhere.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(tables): bind the column-update tests to the orchestration function

The base's route tests assert which column service each payload reaches — the
behavior that now lives in performUpdateTableColumn. They mocked the `@/lib/table`
barrel; the orchestration module imports the service directly, so they mock that
too and keep asserting the same thing through the extracted implementation.

The orchestration tests move onto the base's semantics: writes address the
stable column id, a rename rides inside the write it accompanies rather than
running first, and the currency guards replace the non-select options guard the
service now owns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(copilot): drop the column-type import the delegation made dead

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(tables): move the audit log out of the table service

`lib/table/service.ts` wrote its own audit rows, so whether an operation was
audited depended on which function a caller reached for rather than on a user
having performed it. That is what let v1 and v2 audit a no-op delete, and what
made `deleteTable`'s optional `actingUserId` double as an audit opt-out flag.

Worse, most sites fell back to `actingUserId ?? createdBy`, so an unattributed
call was logged against the table's *creator*. The copilot `mv` path passed no
actor at all: renaming someone else's table recorded them as the renamer.

Audit now lives in the orchestration functions — performDeleteTable,
performRenameTable, performMoveTableToFolder, performUpdateTableLocks — and
the services just write. Internal callers (folder cascade, import rollback)
keep calling the service and are silent by construction rather than by
remembering to omit an argument.

Two services now return what the audit needs: `deleteTable` reports whether it
actually archived a row, so a repeat delete logs nothing; `updateTableLocks`
returns the before/after locks, since only the locked write can observe the
transition its description names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(tables): restore audit provenance and conflict status in orchestration

Moving the audits into the orchestration functions dropped three things the
routes had been carrying, and added one the orchestration now owns twice.

- The v1 and v2 column-update routes passed `request` to `recordAudit`, so
  their audit rows recorded the caller's IP and user-agent. The orchestration
  function had no way to receive it. Every table orchestration function now
  takes an optional `OrchestrationRequestContext` and every HTTP route
  forwards it; the copilot and VFS callers, which have no request, omit it.
- `classifyTableMutation` matched `TableConflictError` on "already exists"
  appearing in the message and reported it as `validation`, turning the UI
  route's 409 on a duplicate table rename into a 400. It now matches the type,
  the way `performRestoreTable` already did.
- `captureServerEvent` ran on every delete while the audit was gated on a row
  actually being archived, so a repeat delete of an archived table still
  reported `table_deleted`. Both now hang off the same evidence.
- The copilot delete path kept its own `captureServerEvent` from when the
  service did not emit one, double-counting every copilot table delete.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a

* fix(tables): say which type a no-op column update restated

A copilot `update_column` payload whose only content was the column's current
type used to return success with the live schema, while the v1, v2, and UI
routes rejected the same payload with "No updates specified". Delegating to
`performUpdateTableColumn` unified them onto the routes' rejection — correct,
but the message tells the caller its request was empty when it named a type.

The orchestration function now reports the same thing `updateColumnType` reports
when it loses this race concurrently: the column is already that type, re-issue
without the type change. An empty payload still reads "No updates specified".

Drops the copilot's `outcome.table ?? tableForUpdate` fallback with it — the
comment described the no-op that can no longer reach that line, and a success
always carries a table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a

* refactor(tables): classify failures by type instead of by message text

The table module decided HTTP statuses by searching error messages for
phrases. `VALIDATION_MESSAGE_FRAGMENTS` and `ROW_WRITE_ERROR_PATTERNS` held 32
substrings between them, and fifteen more lists were inlined in routes — 83
matchers over 17 files, each its own copy of the guesswork and already drifted
apart. It made message wording load-bearing: `TableRowLimitError`'s own doc
comment noted that its text had to contain "row limit" for a route to answer
400, and adding "already exists" to a rename message silently demoted a 409 to
a 400 (the bug fixed one commit ago, by adding another special case).

Services now throw `OrchestrationError`, which carries the transport-neutral
`OrchestrationErrorCode` the layers above already speak. Classification is one
`instanceof` in `orchestrationErrorResponse` (UI + v1) and
`v2CaughtOrchestrationError` (v2). Every pattern list is gone. Wording is free
to change; an unclassified error still becomes a generic 500, which is what an
unexpected fault should be.

`asOrchestrationError` walks the `cause` chain rather than testing the caught
value directly: drizzle wraps a throw raised inside a transaction callback in a
`DrizzleQueryError` whose own message is the failed SQL, so a bare `instanceof`
would drop every failure raised inside `withLockedTable`. That is the same
reason `rootErrorMessage` had to dig for a root cause before.

Three throws stay bare `Error` deliberately — `Table ID mismatch`, `Workspace
ID mismatch`, and `Failed to build upsert conflict predicate` are internal
invariants no consumer classified, and they keep falling through to a 500.
`Insufficient capacity` was in the pattern list with no producer anywhere in
the codebase.

Status changes, all deliberate:

- `'forbidden'` joins the code union so the table-row-limit ceiling keeps its
  403; without it this refactor would have flattened it to 400.
- import-async's table-limit rejection: 400 -> 403, matching the two other
  create routes it had drifted from.
- Renaming a table to an invalid name: 500 -> 400. `validateTableName`
  messages don't contain "Invalid", so no matcher ever caught them.
- Restoring a table that isn't archived, or into an archived workspace:
  500 -> 400.
- A duplicate *column* name stays `validation`/400 rather than becoming a 409
  like a duplicate table name. Both v1 and the orchestration have always
  answered 400 for it; changing a published status is not this refactor's job.

The twelve tests that changed were asserting the substring mechanism itself,
constructing plain `Error`s with magic strings. They now assert the real
contract, plus new cases pinning that identical wording carrying no
classification stays internal and keeps its message off the wire.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

* feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials (#6150)

* feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials

* fix(api): correct credential role, skill permission bar, MCP url identity, and custom-tool conflict mapping

* fix(api): align credential mutation gating, provider-outage status, and unique-violation conflicts

* fix(api): close unique-violation, revival, orphan-write, and env-rename gaps

* fix(api): treat every provider-outage code as unavailable on create and update

* fix(credentials): use the shared outage predicate on the session update path

* fix(contracts): anchor the predicate double-cast annotation to the cast

`check:api-validation:strict` counted 9 unannotated double-casts against a
baseline of 8, failing CI. The predicate leaf schema was annotated, but the
annotation sat above the declaration while the checker anchors on the line
carrying the cast — five lines below, at the close of the object literal. The
scanner walks back at most three lines and stops at the first non-comment one,
so it hit `value: z.unknown().optional(),` and never saw the reason.

Splitting the object schema from the cast puts them adjacent, so the existing
reason binds. No behavior change — the cast, the schema, and the reasoning are
unchanged.

Also lowers the rawJsonReads ratchet 6 -> 5 to match the current count, which
had drifted down; leaving it high lets a removed raw read silently come back.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(skills): point the orchestration error contract at its moved module

#6150 branched before #6134, so skill-lifecycle.ts imports
@/lib/workflows/orchestration/types — the module #6134 moved to
@/lib/core/orchestration/types. Git merged a file deletion on one side with a
new file referencing it on the other: no textual conflict, broken build.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(knowledge): make lib/knowledge/orchestration the single implementation (#6154)

* refactor(knowledge): make lib/knowledge/orchestration the single implementation

Knowledge base create was implemented four times — the internal route, v1, v2,
and the copilot tool — and the orchestration around the shared write had
drifted. Extract it the same way lib/table/orchestration was: services write,
orchestration decides which writes run, guards them, audits them, and returns a
transport-neutral failure.

Behavior converged, not preserved:

- One chunking default (DEFAULT_CHUNKING_CONFIG). The agent defaulted minSize to
  1 against the API's 100, so identical input produced differently-chunked
  knowledge bases depending on who created it. The agent path now chunks at 100.
- Every successful mutation is audited inside the orchestration function. The
  copilot tool called recordAudit zero times, so agent-created knowledge bases,
  document uploads, updates and deletes left no audit trail at all.
- Failures classify by class, not by message text. The knowledge service errors
  are OrchestrationError subclasses and storage-quota rejections throw a shared
  StorageLimitExceededError, replacing four separate message greps for
  "already exists" / "does not have permission" / "storage limit".

delete_connector reported the opposite of what happened. It reached the route
through an internal HTTP self-call that sent no query string, so the route's
keep-documents default always applied while the agent told the user the
documents had been removed. The self-call is gone — all four connector
operations run in-process — and the orchestration returns the real counts.

Also:

- OrchestrationErrorCode gains 'payload_too_large' (413 / PAYLOAD_TOO_LARGE).
  Without it, dropping the storage-limit message match would have regressed the
  documented 413 on knowledge base create and document upload to a 500.
- messageForOrchestrationError renders a route's own wording for an unclassified
  fault, so a driver's message no longer reaches the client on a 500.
- v1 and v2 knowledge base update now forward actorUserId, which the service
  requires for a workspace move; both omitted it.
- The connector DELETE route reads deleteDocuments through parseRequest. Its
  contract declared z.boolean(), which would have rejected the string a query
  param actually is.
- Drop the 409 from POST /api/v2/knowledge/{id}/documents in the OpenAPI spec.
  Nothing on the upload path throws a conflict; it was only ever reachable by
  the message match this change removes.

Behavior change worth noting: a v1/v2 PUT carrying only the workspaceId scope
field and no actual updates now returns 400 rather than 200 with the unchanged
knowledge base.

Deliberately deferred: document update remains internal-only. Extracting
performUpdateKnowledgeDocument makes exposing it on v1/v2 a contract and a route
away, but that is a new public surface rather than part of this consolidation.

* fix(knowledge): make connector create atomic and stop flattening failures

Review round 1 on #6154.

- Resolve the billing payer before the connector is committed, not after. A
  malformed attribution header rejected post-commit left a live connector behind
  a 500, and a retry created a duplicate plus duplicate sync work. Manual sync
  resolves before writing its audit for the same reason.
- Let the source-config validator carry its own failure class. Collapsing every
  rejection to `validation` flattened the connector PATCH route's 401 (stale
  stored credential) and 409 (missing workspace context) into a 400.
- Add `unauthorized` to OrchestrationErrorCode. It is the class that 401 was
  already expressing on this route, and the v2 vocabulary already had
  UNAUTHORIZED; only the shared union was missing it.
- Report a knowledge base that exists but failed to archive as failed, with the
  reason, rather than as not found. The copilot delete loop folded every
  non-not-found failure into `notFound`, telling the user it was never there.
- Route copilot failures through the same message helper the HTTP surfaces use,
  so an unclassified fault's raw text (a driver's failed SQL) no longer reaches
  the agent verbatim while the UI and public APIs get the generic wording.

* feat(api): expand the public v2 files surface (#6160)

* feat(api): expand the public v2 files surface

Adds folder support, rename/restore, move, bulk archive, share, and content
replace to /api/v2/files, so managing files by API no longer stops at
upload + download + archive-one.

Routes are thin: auth -> parse -> perform* -> serialize. Share and content
replace get their orchestration extracted first so the session routes and
the public ones cannot diverge on the effective-authType resolution, the
EE public-sharing gate, or the storage-quota classification.

Presigned upload stays session-only: presign does an advisory quota check
and the real debit happens in the separate register step, so a caller that
never registers leaves unaccounted bytes with no reaper. The buffered
multipart path debits inside uploadWorkspaceFile's own transaction.

* fix(files): classify folder and content failures instead of 500ing them

Bugbot round 1. The v2 routes map errorCode straight to a status, so every
manager failure that arrived unclassified became a 500 for what is really a
caller-fixable 400 or 404.

- Folder manager throws OrchestrationError: missing target/folder -> not_found,
  reparent cycle / self-parent / restore-into-archived-workspace -> validation.
- File manager does the same for the in-transaction 'File not found' paths that
  the earlier pass missed.
- updateWorkspaceFileContent's outer catch re-wrapped everything in a bare
  Error, which stripped the class off StorageLimitExceededError and the new
  not_found alike. It now rethrows a classified failure untouched and attaches
  cause to the generic wrap, so asOrchestrationError can still walk the chain.
- Every remaining perform* gained the asOrchestrationError branch.
- renameWorkspaceFile returned the pre-update read, so the v2 PATCH reported a
  stale updatedAt; it now returns the timestamp it actually wrote.

Docs: upload auto-suffixes a duplicate name rather than rejecting it, matching
the in-app uploader. The description claimed 409 and was simply wrong.

* fix(files): surface a failed upload read-back as the real error

getWorkspaceFile swallows a query failure and returns null unless throwOnError
is set, so a transient blip on the post-upload read reported as 'file could not
be read back'. Distinguish the two: a real null after a just-committed write is
an invariant break, a query failure is itself.

* revert(api): drop the dedicated v2 file-folder routes

File folders already live in the shared folder table as resourceType 'file'
(#6045 cut them over, #6051 dropped workspace_file_folders), and the remaining
file-specific folder machinery is being folded into the generic folder engine.
Publishing /api/v2/files/folders/** would pin that transitional split into a
public contract we'd then have to keep or break.

Files stay folder-aware — folderId/folderPath on the projection, folderId on
upload, and the move route — because a folder id is a folder.id and survives
the unification untouched. Folder management belongs on /api/v2/folders once
that surface serves resourceType 'file'; until then there is no v2 way to
enumerate file folders, which is the deliberate gap.

The orchestration classification fixes stay: the internal routes and the
copilot file-folder tools still call those perform* functions.

* fix(files): classify upload failures instead of matching their wording

Bugbot round 2. uploadWorkspaceFile had the same outer-catch rewrap that
updateWorkspaceFileContent did, so a blown storage quota reached the route as a
bare Error and the v2 handler recovered the status by substring-matching the
message. Any rewording silently demoted a 413 to a 500.

- uploadWorkspaceFile rethrows a classified failure untouched and attaches cause
  to the generic wrap.
- FileConflictError is now an OrchestrationError('conflict'), so a duplicate name
  classifies like every other conflict. Its 'FILE_EXISTS' discriminator had no
  readers and is gone; the instanceof checks elsewhere still hold.
- The v2 upload handler uses v2CaughtOrchestrationError, dropping all three
  string matches.

Also documents that bulk-archive is best-effort: unknown or already-archived ids
are skipped rather than failing the call, and deletedItems is what actually
happened. That asymmetry with the single-id DELETE was undocumented.

* feat(api): add search, filtering, and sorting to the v2 list endpoints (#6189)

* feat(api): add search, filtering, and sorting to the v2 list endpoints

One convention across every v2 list, documented on lib/api/contracts/v2/shared.ts:
`search` (case-insensitive substring on the resource's natural name field),
`sortBy` + `sortOrder` (per-resource enum, never a free string), and enumerated
resource-specific filters. Reuses the sortBy/sortOrder pair v2 logs and v2
knowledge-documents already ship rather than inventing a third dialect
alongside the Logs filters and the Tables predicate grammar.

Every filter and sort is pushed into SQL. GET /api/v2/files previously read the
whole scope and sorted/sliced it in JS; it now goes through a new
queryWorkspaceFiles that filters, orders, and bounds the page in one query.

Cursors are stamped with the sort they were minted under, so replaying one
under a different sort is a 400 instead of silently duplicated or skipped rows.

* fix(api): validate v2 cursor key values and compare timestamps at ms precision

Two review findings, fixed at the root by making a keyset key own its cursor
codec instead of hand-writing a decoder per sort.

Cursor key values are caller-controlled, and matching the sort stamp and key
count was not enough: an unparseable timestamp or a non-numeric size reached
the query as an Invalid Date or NaN and surfaced as a 500. Each key now type-
checks its own value and rejects a cursor it cannot hold, which both routes
render as the documented 400.

Timestamp keys now order and compare on date_trunc('milliseconds', col).
Postgres keeps microseconds and defaultNow() populates them, but a cursor value
round-trips through a millisecond-only JS Date — comparing the raw column
against the truncated value re-admitted the page's own last row, duplicating it
and stalling pagination outright at a page size of one. Reachable today via
workspace_files.updated_at, which insertFileMetadata leaves to defaultNow().

* feat(api): complete the v2 workflows resource with versions and CRUD (#6184)

* feat(api): complete the v2 workflows resource with versions and CRUD

Adds version listing/detail plus create, update, and delete to the v2
workflows surface, which previously covered only execution and deployment.

- GET /api/v2/workflows/[id]/versions — cursor-paginated, newest first
- GET /api/v2/workflows/[id]/versions/[version] — version + pinned state
- POST /api/v2/workflows, PATCH and DELETE /api/v2/workflows/[id]

All six delegate to the existing orchestration and persistence helpers;
no new domain logic.

* fix(api): check folder containment before lock state; reject malformed version cursors

assertFolderMutable walks a folder's ancestor chain without filtering on
workspace, so inspecting it before containment let a caller tell a locked
folder in someone else's workspace (423) from a nonexistent one (400).
Create and update now assert containment first, matching the ordering
import-workflow.ts already uses.

A version cursor that decodes to JSON without a numeric version filtered
every row out and returned an empty page with nextCursor null, which reads
as a clean end-of-list. Malformed cursors are now a 400.

* refactor(api): page workflow versions in the persistence helper

listWorkflowVersions read every version row and the route filtered and
sliced the result in memory, so the response was bounded but the query
was not. It now takes optional limit/afterVersion, turning the cursor
into a real keyset query; the route asks for limit + 1 and only trims
the has-more probe. Both params are optional, so the internal, v1 admin,
and copilot callers are unchanged.

Also restores the untouched GET handler in [id]/route.ts to its original
formatting — collapsing its signature had re-indented the whole body and
buried the actual additions in whitespace churn.

* feat(api): expand v2 tables with stateless multipart transfers (#6188)

* feat(api): expand the public v2 tables surface

Adds 16 operations so a v2 caller can do what the internal surface can:
rename/move/lock a table, restore it, manage saved views, run enrichment
columns, look up rows, and import/export with observable job control.

Extracts lib/table/orchestration/import.ts (performTableCsvImport,
performCreateTableFromCsv) and lib/table/export-stream.ts from the
first-party routes, then repoints those routes at them, so v1 and v2
cannot drift on what an import or export actually does.

events/stream, metadata and dispatches stay internal — they are editor
state, not public API.

* fix(api): make v2 table PATCH all-or-nothing and name the lock in every 423

Greptile P1: PATCH applied locks, rename and move as three sequential
transactions, so a folder rejected mid-request left the earlier writes
persisted while the response reported failure — and the schema-changed
signal was skipped, leaving open clients on stale state. Every rejectable
condition now runs before the first write, and the signal fires whenever
anything did land.

Cursor: v2TableLockError dropped the lock kind, so async import, column
run, enrichment and table mutations returned a bare LOCKED. A table has
four independent locks, so the caller could not tell which to clear.

* fix(api): report the lock kind on classified 423s too, not just thrown ones

The previous commit named the lock only where the rejection was thrown and
caught at the route boundary. Where it instead arrives as a classified
`errorCode: 'locked'` outcome — delete table, delete row, update column,
and the table mutations — the kind was dropped, so those 423s stayed
unactionable while their neighbours improved.

The orchestration results now carry `lock`, and a shared
`v2TableOrchestrationError` renders both arrival paths into the same
`{ code, message, details: { lock } }` body. `details` is omitted rather
than sent null when the kind is unknown, so a caller branching on it sees
absence instead of a phantom value.

* fix(api): make async table imports observable, not just startable

`POST /import-async` pointed callers at `GET /api/v2/tables/jobs` to track
progress, but that endpoint filters to `type = 'export'` — imports are
derived onto the table itself, one write job at a time, and exports get a
separate list precisely because they are excluded from that derivation.
The public Table shape omitted those derived fields, so an async import
could be started and cancelled but never observed to completion, failure,
or progress. That is the gap the import/export/job-control set was meant
to close.

Table now carries `job` — id, type, status, rowsProcessed, error, or null
when idle — and the import-async docs point at the table rather than the
export list.

* feat(api): make v2 table PATCH state which operations landed on failure

Greptile held the PR at 4/5 on the residual non-atomicity and named two
acceptable resolutions: make PATCH atomic, or have the contract adopt and
expose partial-success explicitly. Atomicity would mean threading one
transaction through renameTable, moveTableToFolder and updateTableLocks —
three shared service functions with four non-test callers including the
first-party route and two copilot tools — and deferring their per-operation
audits to commit time. That is a refactor of shared write paths well
outside this PR.

So the contract states it instead. Every rejectable condition is already
pre-validated, so a failure here is a genuine fault; when one follows a
successful operation the error now carries `details.applied` listing what
is live. Absent when nothing applied, so its presence always means "these
changes took effect despite the error". Documented on the operation.

`v2ErrorForOrchestration` gained the optional `details` this needs.

* fix(api): make table lock flags read-only on the public v2 surface

The new PATCH /api/v2/tables/[tableId] accepted a `locks` object, gated
on workspace admin plus the table-locks feature. That still lets an API
key clear the guard placed there to stop it: `write` is the floor for
the endpoint, and admin keys are ordinary API keys, so a lock is no
longer a boundary the key cannot cross.

Locks stay readable on the table resource and enforcement is unchanged
(a locked verb still returns 423). Changing one is now a first-party
admin action only.

The v2 body is declared here rather than reusing the first-party
updateTableBodySchema, which keeps its `locks` field so the UI can still
toggle them. It is .strict(), so a request carrying `locks` is rejected
with a 400 naming the field instead of silently succeeding without
applying it.

* fix(api): keep reporting applied operations when the PATCH re-read fails

The composite table PATCH promises that `error.details.applied` names the
operations that are live despite an error, but `applied` was scoped
inside the try. A rename or move that committed and was then followed by
a throw in the final re-read — or a re-read finding the table archived —
returned a bare 500/404 with no details, telling the caller nothing had
landed. It would then retry into a duplicate-name conflict or repeat the
move.

`applied` is now function-scoped so every post-write exit carries it: the
404 on a missing re-read, a thrown lock error, a classified orchestration
error, and the generic 500. `v2TableLockError` gains the same
`extraDetails` parameter `v2TableOrchestrationError` already had.

* feat(api): add workflow group writes to the v2 tables surface

v2 exposed GET /groups but none of the writes, so the public API could
run an enrichment or workflow column and read its binding, but never
create one. A caller could add a plain data column and trigger the
machine; wiring the two together still required the UI.

Adds POST/PATCH/DELETE on /api/v2/tables/[tableId]/groups. The group is
the unit that fills columns — one group feeds several — so creating one
creates its output columns in the same call, matching the first-party
shape rather than inverting it onto the column endpoint.

Four departures from the first-party body, all public-surface concerns:
- group.id is optional and server-generated. The UI mints an id to render
  optimistically; a public caller has no such need and a client-chosen id
  is a collision waiting to happen.
- outputColumns[].workflowGroupId is dropped from the body and stamped
  from the resolved group, so it cannot disagree with it.
- autoRun defaults to false. First-party defaults true so a UI add fills
  cells immediately; here it would make one POST fan out a metered run
  across every existing row.
- A group naming neither a workflowId (type manual) nor an enrichmentId
  (type enrichment) is a 400 rather than a half-specified group the route
  has to guess about.

Also rejects an outputColumns entry no group output feeds — the two
arrays are joined by column name, and the first-party client builds both
from one picker so it cannot desync, but a public caller can.

Workspace containment on workflowId is asserted before it is persisted,
on create and on any update that re-points the group; without it a table
becomes a way to invoke workflows the key cannot otherwise reach.

* improvement(api): make v2 table import and export async-only

Drops the three synchronous entry points: POST /tables/[tableId]/import,
POST /tables/import-csv, and GET /tables/[tableId]/export.

Sync import tied a write to the lifetime of an HTTP request. The body
*was* the data, so it carried a 10 MB cap that Next silently truncates
past — a partial import reporting success. It also had no job, so a
timeout mid-write left rows in place with nothing to poll and nothing to
cancel. The async path reads the file from storage instead: upload via
POST /api/v2/files for a key, start with POST /import-async, watch
GET /tables/[tableId] -> job, stop with POST /job/cancel.

Sync export carried no such hazard, but one shape per operation beats
two: with both removed the surface has exactly one way to move a table
in or out, and the CLI wraps the extra calls.

This also removes the last multipart handling in v2 tables. Those were
the only routes bypassing parseRequest — form fields were parsed by hand
against separate form schemas, outside the contract system every other
v2 write goes through.

Create-a-table-from-CSV is now two calls: POST /tables, then
/import-async with createColumns. csvImportModeSchema is append|replace,
so there is no single-call create.

Route baseline 1064 -> 1061.

* docs(api): correct the import-async note about upload size limits

The docstring claimed there is no synchronous upload endpoint and so no
request-body size cliff. Both are wrong: POST /api/v2/files is a
synchronous multipart upload with a 100 MB cap, and it is the only v2
upload path (presigned is deliberately absent).

What async-only actually bought: the cap went 10 MB -> 100 MB, it fails
on an explicit size check and a bounded body read rather than a proxy cap
that silently truncates, authorization completes before any body is
buffered, and the table write is a job that can be watched and cancelled.

* feat(api): unify file and table transfers

* improvement(api): make multipart transfers stateless

* fix(api): make table import completion retries idempotent

* feat(v2-tables): paginate the table list

`GET /api/v2/tables` returned every table in the workspace in one response —
it used the cursor envelope but hardcoded `nextCursor: null`, and had no
`limit`. That was defensible when tables were only created through the UI;
`POST /api/v2/tables` is public now, so a script can create them in bulk and
the list has no way to ask for less.

Adds `queryTables` alongside `listTables` rather than changing it, so the
internal callers that genuinely want the whole scope are untouched — the same
split `queryWorkspaceFiles` / `listWorkspaceFiles` already uses. Filter, order
and slice all run in the query, so a `search` never costs a full-workspace read.

A cursor whose values don't bind raises a validation error instead of being
coerced to "no filter", which would have silently served page 1 under a resumed
cursor. The keyset closes on `id` so a page boundary inside a run of equal names
or timestamps stays stable.

The shared `LimitQuery` doc component said "Maximum rows to return"; it now
serves the table list too, so the wording is resource-neutral.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(api): add multipart knowledge document uploads

* fix(api): keep usage admission at knowledge upload session creation

* feat(knowledge): wire knowledge base uploads to multipart sessions

* fix(knowledge): refuse to abort an upload once a document is bound

* fix(uploads): prevent multipart cleanup races

* Unify file creation and signed upload sessions (#6264)

* feat(uploads): unify signed upload sessions

* fix(uploads): preserve attachment storage semantics

* feat(files): add authored file creation

* fix(uploads): omit hoisted S3 metadata headers

* feat(api): add file metadata endpoint

* improvement(api): scope folders to resource paths (#6284)

* improvement(api): scope folders to resource paths

* fix(files): serialize folder resolution with uploads

* fix(files): release folder lock before upload setup

* fix(api): normalize folder paths and unblock resource mutations

* fix(api): make resource cleanup and metadata consistent

* improvement(uploads): persist multipart sessions in postgres

* fix(db): store table row trigger timestamps in UTC

* improvement(api): default folder deletion to non-recursive

* fix(billing): unify chat usage source

* improvement(logs): expose trace spans on log detail

* fix(logs): parse list trace spans

* improvement(api): replace workflow jobs with execution resources (#6294)

* improvement(api): replace workflow jobs with execution resources

* fix(api): preserve legacy jobs while preferring v2 executions

* fix(api): make execution polling resume-aware

* fix(ui): hide async examples for public workflows

* fix(api): bridge resume queue visibility lag

* feat(api): add v2 workflow resume endpoint

* fix(api): project pending resume attempts

* fix(api): prefer terminal logs over stale resumes

* improvement(api): unify v2 resource query layers (#6319)

* improvement(api): unify v2 resource query layers

* fix(api): address v2 review findings

* fix(api): preserve cancelled queue status

* fix(api): guard cancelled job transitions

* fix(api): close v2 resume and log gaps

* feat(api): rename v2 executions to runs

* feat(api): split credentials and secrets

* feat(api): add workspace metadata and email attribution

* improvement(api): consolidate public v2 route handling

* improvement(files): centralize operations across APIs and Copilot (#6392)

* improvement(files): unify rename authorization

* chore(skills): add file operation migration guide

* improvement(files): consolidate file operation authorization

* improvement(files): extract shared operation foundation

* improvement(api): simplify internal route declarations

* improvement(files): centralize application authorization

* refactor(api): share workspace file name validation

* refactor(files): centralize copilot application calls

* docs(skills): generalize application operation migration

* improvement(api): centralize remaining v2 resource operations (#6412)

* improvement(api): centralize v2 resource operations

* fix(api): preserve custom tool conflict errors

* improvement(api): migrate policy-sensitive v2 reads (#6410)

* improvement(workflows): centralize v2 application operations (#6411)

* refactor(api): migrate v2 knowledge operations (#6413)

* refactor(api): migrate v2 knowledge operations

* fix(knowledge): fail upload completion on dispatch errors

* fix(knowledge): preserve upload retry and VFS errors

* improvement(tables): centralize v2 application operations (#6414)

* improvement(tables): centralize v2 application operations

* fix(tables): preserve run validation and signals

* feat(auth): add scoped internal executor delegation (#6459)

* feat(auth): add scoped internal executor delegation

* fix(auth): derive delegation lifetime from one timestamp

* Include share status in file metadata

* feat(auth): centralize delegated identity policy (#6462)

* improvement(copilot): consolidate application adapters (#6450)

* improvement(api): harden application route boundaries (#6451)

* improvement(api): harden application route boundaries

* fix(folders): reject creates at workspace cap

* fix(knowledge): enforce trusted workspace scope (#6452)

* fix(knowledge): enforce trusted workspace scope

* refactor(knowledge): declare v2 body lifecycle

* finish knowledge application migration

* refactor(knowledge): compose copilot batch commands

* fix(knowledge): parse connector query flags

* fix(knowledge): finalize partial batch effects

* fix(knowledge): align merged application boundaries

* fix(knowledge): close application boundary review gaps

* style(knowledge): satisfy branch biome checks

* fix(knowledge): page connector documents in editor

* refactor: enforce Copilot table application boundary (#6453)

* refactor: enforce copilot table application boundary

* fix(tables): finish application boundary migration

* fix(tables): restore scoped copilot imports

* fix(tables): compose copilot commands atomically

* fix(tables): preserve workflow group scheduling

* fix(tables): complete fixed copilot composition

* fix(tables): reject enrichment output mutation

* fix(tables): complete authorized application boundary

* fix(workflows): migrate Copilot application boundary (#6455)

* fix(workflows): migrate Copilot application boundary

* fix(workflows): finish delegated application migration

* fix(workflows): encode VFS folder aliases

* fix(workflows): close application composition gaps

* fix(workflows): preserve VFS validation errors

* fix(workflows): complete application boundary migration

* test(workflows): format canonical binding coverage

* fix(workflows): scope executor metadata reads

* fix(workflows): bind executor metadata targets

* improvement(skills): align application operation guidance (#6532)

* feat(api): expose v2 resource owners

* fix(api): distinguish visible resource authorization failures (#6537)

* feat(api): generate v2 OpenAPI from contracts (#6509)

* feat(api): generate v2 OpenAPI from contracts

* fix(api): preserve string boolean wire defaults

* fix(api): document file download headers

* fix(docs): use TypeScript CLI with Next.js

* fix(docs): avoid client-rendered theme script

* fix(api): document departed audit default

* feat(api): replace legacy core docs with v2

* feat(api): generate v2 OpenAPI from contracts

* feat(api): refine generated v2 OpenAPI docs

* fix(docs): align localized v2 execution examples

* fix(ci): restore Helm diff and sync audit mock

* fix CI regressions after staging merge

---------

Co-authored-by: Waleed <walif6@gmail.com>
Co-authored-by: Theodore Li <theodoreqili@gmail.com>
Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com>
Co-authored-by: Theodore Li <theo@sim.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Vikhyath Mondreti
2026-08-11 05:45:25 -04:00
committed by GitHub
co-authored by Claude Opus 5 Waleed Theodore Li Siddharth Ganesan Theodore Li
parent 9277e7d2d2
commit 263e3ca67e
1255 changed files with 170265 additions and 49843 deletions
+4 -2
View File
@@ -768,9 +768,11 @@ tools: {
}
```
#### 3. Create Internal API Route
#### 3. Create Special Internal Tool Execution Route
Create `apps/sim/app/api/tools/{service}/{action}/route.ts`. Internal tool routes are HTTP boundaries and follow the same contract policy as public routes — define the request/response shape in `apps/sim/lib/api/contracts/tools/{service}.ts` (or an existing aggregate) and validate with canonical helpers from `@/lib/api/server`. Never write a route-local Zod schema.
Create `apps/sim/app/api/tools/{service}/{action}/route.ts`. This raw route pattern is only for an integration's provider-execution boundary when it needs special file normalization, large-body handling, or protocol behavior. It is not the pattern for CRUD or other operations on protected Sim resources. For those, use the `migrate-application-operation` skill and an authorized application use case with the ordinary internal/v2 route builders.
Internal tool routes are HTTP boundaries and follow the same contract policy as public routes — define the request/response shape in `apps/sim/lib/api/contracts/tools/{service}.ts` (or an existing aggregate) and validate with canonical helpers from `@/lib/api/server`. Never write a route-local Zod schema. Authenticate and perform cheap admission before parsing or downloading files.
```typescript
// apps/sim/lib/api/contracts/tools/{service}.ts
@@ -0,0 +1,304 @@
---
name: migrate-application-operation
description: Create or migrate a protected Sim resource operation in the shared Principal and application-use-case architecture across internal APIs, public or versioned APIs, Copilot, and other trusted tool adapters. Use when adding a protected endpoint, tool command, or CRUD method; removing route- or tool-local authorization and business logic; consolidating resource reads or writes behind semantic operation policies; or adding another surface to an existing application operation while preserving contracts, identity, errors, rate limits, audit, analytics, and compatibility behavior. Treat v1, uploads, streams, large bodies, bulk recursion, and polymorphic tools as explicitly scoped special cases.
---
# Create Or Migrate Application Operation
Create or migrate one bounded semantic operation at a time. Share authorization and business behavior without forcing internal APIs, public APIs, Copilot, and other tools to share authentication, input schemas, or response shapes.
## Enforce the application boundary
Apply this invariant:
> Every real operation on persisted or protected data enters through an authorized application use case.
This includes mutations, content and metadata reads, canonical resource lookup, and reference-to-resource resolution when the lookup is authorization-sensitive.
Surface helpers may:
- Normalize an already-authenticated surface context into a `Principal`.
- Translate aliases or wire arguments into application input.
- Select a code-defined operation and application use case.
- Call the application use case.
- Translate typed results and errors into the surface contract.
Surface helpers must not:
- Query databases or storage.
- Decide workspace or resource authorization.
- Implement business transactions.
- Record semantic audit or shared domain notifications.
- Infer authoritative identity, workspace, audience, or scope from untrusted arguments.
- Substitute billing attribution for identity.
A helper that resolves a path is valid only when the actual protected lookup runs through an authorized application resolver. If a helper begins doing real data work, move that work into an application use case.
## Read the foundation first
Read these files completely before editing:
- `packages/auth/src/principal.ts`
- `apps/sim/lib/core/application/operation.ts`
- `apps/sim/lib/core/application/workspace-operation.ts`
- `apps/sim/lib/core/application/workspace-authorization.ts`
- `apps/sim/lib/core/application/authorized-workspace-use-case.ts`
- `apps/sim/lib/api/server/routes/definition.ts`
- `apps/sim/lib/api/server/routes/internal-json-route.ts`
- `apps/sim/lib/api/server/routes/v2-json-route.ts`
- `apps/sim/lib/auth/internal-delegation.ts`
- `apps/sim/lib/copilot/application/application-adapter.ts`
- `apps/sim/lib/copilot/auth/application-delegation.ts`
Use the file domain only as a representative golden slice:
- `apps/sim/lib/workspace-files/application/operations.ts`
- `apps/sim/lib/workspace-files/application/authorized-workspace-file-use-case.ts`
- `apps/sim/lib/workspace-files/application/rename-workspace-file.ts`
- `apps/sim/lib/copilot/application/execute-file-use-case.ts`
- `apps/sim/lib/copilot/auth/file-delegation.ts`
Then read the target domain's operation registry, application code, repositories, contracts, adapters, aliases, resume paths, and focused tests. Fail immediately if the shared foundation is absent. Do not recreate it inside the domain.
## Bound the migration
Inventory every entry point for the behavior before editing:
- Internal HTTP routes and contracts.
- Public or versioned API routes and contracts.
- Copilot tools, aliases, resume paths, and polymorphic branches.
- Other tool servers, workflow executors, jobs, or service callers.
- Current authentication, authorization, workspace assertions, and concealment.
- Manager or orchestration call chains.
- Audit, notification, analytics, and billing side effects.
- Error/status/result behavior.
- Rate-limit identity, rollout gates, quota, and concurrency admission.
Classify each as `migrate`, `defer`, or `non-goal`. Do not migrate adjacent operations merely because they share a module. Do not modify v1 unless the request explicitly includes it.
Preserve behavior unless the task explicitly changes it. Stop and report a decision when surfaces currently disagree on security or compatibility behavior; do not silently choose one.
## Keep the layers distinct
Use these responsibilities:
1. Authentication adapter: verify the surface credential or trusted execution context and construct a `Principal`.
2. Route or tool adapter: select rate policy, parse its contract, translate input, call the application use case, and render its own result.
3. Application use case: load canonical context, compare asserted scope, authorize the semantic operation, execute business behavior, project semantic audit, and trigger shared domain effects.
4. Manager or repository: perform database and storage reads or writes using canonical identifiers and scope. Never accept credentials or principals.
5. Presenter: return only the surface success body or typed binary descriptor. Never construct auth, rate, or error behavior.
For ordinary public JSON routes, preserve this order:
```text
IP abuse limit
-> authenticate
-> build Principal
-> operation rate limit
-> parse surface contract
-> application use case
-> canonical load
-> asserted-scope concealment
-> current authorization
-> manager read or mutation
-> semantic audit
-> shared domain effects
-> surface presenter
```
Internal routes may omit the IP bucket or operation limit only through an explicit policy with a reason. Usage billing, storage quota, cost admission, and concurrency are separate from request-rate limiting.
Never query API keys or sessions from the application layer. Never add fallback identity or authorization behavior. Propagate infrastructure failures instead of turning them into not-found or forbidden results.
## Define the semantic operation once
Add one stable entry to the target domain's operation registry:
```ts
rename: defineWorkspaceOperation({
id: 'widgets.rename',
minimumRole: 'write',
workspaceApiKey: 'allow',
principalKinds: ['session', 'personal_api_key', 'workspace_api_key', 'delegated'],
delegatedServices: ['copilot'],
})
```
Do not create internal-, public-, or Copilot-specific versions of the same semantic operation. If two callers have materially different business or transactional semantics, define separate semantic operations and use cases and explain the distinction.
Choose principal kinds from actual behavior. Do not accept every principal merely because the use case is shared. Workspace API keys have a write ceiling and cannot satisfy admin operations. The operation definition must fail fast when its role, workspace-key policy, and principal kinds disagree.
Route declarations, tool adapters, and use cases must use the same literal operation. Runtime operation selection is permitted only from a trusted, code-defined registry. Never accept an operation ID or permission tag from an HTTP body, model argument, or other untrusted input.
## Implement the application use case
Use `defineAuthorizedWorkspaceUseCase` directly or a thin domain binding that supplies domain-specific authorization options:
```ts
export const renameWidget = defineAuthorizedWorkspaceUseCase({
operation: widgetOperations.rename,
resolveContext: ({ input }: { input: RenameWidgetInput }) =>
loadCanonicalWidgetContext(input.id, input.assertedWorkspaceId),
authorizationOptions: { delegation: widgetDelegationPolicy },
execute: async ({ input, context }) => renameWidgetRecord({
workspaceId: context.workspaceId,
widgetId: context.resourceId,
name: input.name,
}),
projectAudit: ({ result }) => ({
action: AuditAction.WIDGET_UPDATED,
resourceType: AuditResourceType.WIDGET,
resourceId: result.id,
resourceName: result.name,
}),
afterSuccess: ({ context }) => notifyWidgetsChanged(context.workspaceId),
})
```
Adapt the example to the domain's real authorization options; do not copy invented field names.
The wrapper must own this lifecycle:
1. Reject disallowed principal kinds before protected loading.
2. Load canonical context and conceal asserted-scope mismatches as required.
3. Authorize the operation using current policy state.
4. Execute the manager or repository primitive.
5. Project audit from authoritative results.
6. Await shared post-success effects.
Do not call shared authorization, principal audit attribution, or `recordAudit` manually from an ordinary migrated use-case body. Use `projectAudit` only when the operation has semantic audit. Return no audit entries for authoritative no-ops. Keep product analytics such as `captureServerEvent` surface-specific through the adapter's success hook.
Inspect legacy orchestration before reusing it. If it already authorizes, audits, notifies, or captures analytics, call a lower-level primitive or remove duplicate responsibility for migrated callers.
Application code must remain surface-neutral. It must not import `app/api/**`, `next/server`, internal/v1/v2 contracts or presenters, or Copilot tool handlers. Return domain values and let each surface presenter project its own wire result.
## Adapt internal APIs
Use `defineInternalJsonRoute` for ordinary JSON routes. Explicitly declare the contract, authentication policy, semantic operation, rate policy, error policy, input mapping, use case, and presenter when the wire result differs.
Use `internalSessionAuth` for session-only routes. Use `createInternalSessionOrExecutorAuth` only when the endpoint genuinely supports signed executor delegation; the semantic operation must then allow `delegated` principals from the `executor` service. Never turn an actorless legacy JWT into a fake session, owner, or user principal.
The internal adapter owns authentication and internal response envelopes. It must not implement workspace authorization. Preserve internal-only analytics through `onSuccess` after application success.
Keep the route module declarative. If several internal routes repeat authentication, parsing, error rendering, or response construction, improve the shared internal route builder instead of adding a domain-specific route wrapper.
## Adapt public or versioned APIs
Use the appropriate public/versioned route builder, such as `defineV2JsonRoute`, with API-key authentication, explicit semantic operation and rate policy, external error projection, input mapping, application use case, and an external presenter. V2 rollout admission is centralized by the builder; do not invent a route-local rollout policy.
Authentication and HTTP formatting may differ from internal APIs; authorization and business behavior must not. Rate-limit using the credential or principal subject, never a billed owner. Resolve billing attribution only for billing, quota, or legacy required-user fields.
Keep surface contracts separate when their wire shapes differ. Reuse shared primitive schemas and domain validators for invariants such as IDs, names, bounds, and formats. Do not maintain duplicate internal and external schemas merely because the routes are separate; import the same schema when the wire shape is genuinely identical. Never cast one surface response into another.
Keep v1 middleware and routes unchanged unless explicitly included.
## Adapt Copilot
Copilot is a surface adapter, not a separate application layer. If an HTTP or other surface already uses an application use case, Copilot must call that exact use case rather than reimplementing protected business behavior under `lib/copilot`.
Create one domain-level Copilot application adapter with `createCopilotApplicationAdapter` instead of constructing delegated principals in every tool:
```ts
executeCopilotWidgetUseCase(context, renameWidget, input, { resourceId })
```
That adapter must:
- Require a trusted server-authored Copilot execution marker.
- Require the authenticated subject, canonical workspace, tool-call or execution identity, and required audience or lifecycle scope.
- Construct the shared delegated `Principal` in one place.
- Optionally bind the canonical resource scope after trusted resolution.
- Verify that the use case exposes a registered code-defined operation.
- Use the domain's exact immutable operation registry so operation-object membership and identity are checked centrally.
- Call the application use case directly.
Never construct authoritative delegation from model-provided workspace IDs, user IDs, operation IDs, resource scope, or permission tags. Model arguments are requested targets only and must be checked against trusted execution context and canonical data.
Tool handlers own argument aliases, resumable legacy names, abort checks, tool-call reporting, and tool-specific presentation. They must not query managers directly for protected operations, manually authorize, or implement protected business behavior. If a Copilot-only compound action expresses real domain behavior, define a surface-neutral domain operation and application use case for it.
A Copilot reference helper may translate a path to a resource only by calling an authorized application resolver under the intended semantic operation. Passing a code-defined operation object is acceptable; passing a model-provided operation string is not. An immediate same-request resolver followed by the operation may reuse one trusted principal, though the application operation still performs its own canonical authorization. Fresh authentication and authorization are required across lifecycle boundaries such as resumed tool calls, executor callbacks, queued or background work, upload control legs and finalization, durable completion, and long-running provider operations. When resolution and execution form one business operation, need a consistent snapshot, or appear repeatedly together, prefer a top-level application use case such as `renameWidgetByReference`.
Surface adapters must not compose protected mutations. An atomic compound action requires one top-level semantic domain operation and application use case that owns the transaction and authoritative result. An explicitly best-effort application command may coordinate multiple operations only when it defines hard input and expansion caps, cancellation checkpoints, partial-result semantics, audit behavior, and rate/quota policy. Keep composition exceptional and explicit; ordinary tools should use the shared execution adapter.
Map expected typed errors to safe tool results. Unknown errors must become generic system/retryable messages while retaining full causes in server logs. Never return raw database or storage errors to the model.
## Adapt other internal or external tools
Treat every tool runtime as a surface adapter:
- Normalize its already-authenticated execution context into an existing `Principal` through one shared adapter for that runtime or domain.
- Call the same application use case used by HTTP and Copilot surfaces.
- Preserve the tool protocol's input, output, retry, and cancellation semantics.
- Keep authoritative workspace and subject scope server-authored.
An internal caller is not automatically trusted to bypass authorization. It must supply an explicit principal or use a deliberately designed service/delegation principal. If the current principal model cannot express its authority, stop and extend the identity model intentionally; do not fall back to an owner, uploader, creator, or arbitrary user ID.
External tool endpoints authenticate at their adapter exactly like public APIs. Do not authenticate again inside the application use case.
## Preserve identity and attribution
- Session and personal-key principals authorize through current human workspace permission.
- Personal API keys also respect the workspace's personal-key policy.
- Workspace keys authorize as the workspace under explicit operation policy and the write ceiling, independent of creator membership.
- Delegated principals re-check the current subject and their workspace, audience, expiry, execution, and resource scope.
- Billing owners are attribution for billing or legacy required columns only, never authorization, rate identity, delegated identity, audit actor, or human analytics identity.
- Preserve structured `PrincipalActor` metadata in semantic audit.
If a required legacy user column cannot represent the real actor, label the compatibility attribution explicitly. Never pretend it is the acting human.
## Handle special operations explicitly
Do not force these through an ordinary JSON migration:
- Upload or multipart lifecycles: bind immutable credential identity, reauthorize control legs and finalization, and make durable completion idempotent.
- Large bodies: authenticate and perform cheap admission before bounded buffering.
- Binary or streaming responses: use binary/stream builders and typed descriptors.
- Bulk or recursive operations: deduplicate and cap inputs and expansion, load all resources canonically, and define atomic versus best-effort behavior.
- Polymorphic tools: select the semantic operation only after trusted target-kind resolution; do not route unrelated branches through one domain registry.
- Multi-resource transactions: keep canonical scope predicates and derive audit from authoritative affected rows.
Stop and report a missing design rather than weakening identity, authorization, limits, or errors.
## Test the complete matrix
Add focused tests for every migrated surface and principal kind allowed by the operation:
- Application: allowed and disallowed roles, principal-kind rejection before canonical loading, workspace assertion mismatch, delegated scope, not found, conflict, no-op, and infrastructure propagation.
- Operation registry: role/workspace-key/principal-kind/delegated-service consistency and fail-fast rejection of invalid definitions.
- Repository: canonical active lookup, workspace-predicated writes, archived resources, authoritative affected rows, and database error propagation.
- Internal API: authentication before parsing, exact contract, typed errors, and surface analytics only after success.
- Public API: personal and workspace keys, rate and rollout behavior, concealment, exact external envelope, and rate headers.
- Copilot or tools: trusted context, exact registered operation membership, rejected forged scope, aliases and resume paths, permission re-check, safe errors, and unchanged tool result shapes.
- Side effects: audit derives from authoritative results; shared notifications follow audit; neither occurs for rejection or no-op.
Run at minimum:
```bash
bunx vitest run <focused test files>
bunx biome check <changed source and test files>
bunx turbo run type-check --filter=sim --filter=@sim/auth
bun run check:api-validation:strict
git diff --check
```
Do not claim a check passed unless it completed successfully.
## Work safely in parallel
- Assign non-overlapping route modules and caller sets. Two methods in one route file are one ownership unit.
- Treat operation registries, contract families, route policies, and shared surface adapters as merge hotspots.
- Keep shared core foundations owned by one task; ordinary domain migrations should consume them without modifying them.
- Preserve unrelated working-tree changes. Never stage proposal docs, lockfile drift, or another agent's edits.
- Do not commit, push, or open a PR unless requested.
## Hand off
Report:
1. Semantic operation, role, workspace-key policy, and principal kinds.
2. Migrated, deferred, and non-goal entry points.
3. Behavior preserved per internal, public, Copilot, and other tool surface.
4. Identity construction and authoritative scope source for each surface.
5. Files changed and shared merge hotspots.
6. Tests and checks run with results.
7. Remaining risks or blockers. Fail fast when an invariant could not be implemented.
@@ -0,0 +1,4 @@
interface:
display_name: "Create Or Migrate Application Operation"
short_description: "Share one protected operation across surfaces"
default_prompt: "Use $migrate-application-operation to create or migrate one protected resource operation across internal APIs, public APIs, Copilot, and other tools."
+4 -2
View File
@@ -767,9 +767,11 @@ tools: {
}
```
#### 3. Create Internal API Route
#### 3. Create Special Internal Tool Execution Route
Create `apps/sim/app/api/tools/{service}/{action}/route.ts`. Internal tool routes are HTTP boundaries and follow the same contract policy as public routes — define the request/response shape in `apps/sim/lib/api/contracts/tools/{service}.ts` (or an existing aggregate) and validate with canonical helpers from `@/lib/api/server`. Never write a route-local Zod schema.
Create `apps/sim/app/api/tools/{service}/{action}/route.ts`. This raw route pattern is only for an integration's provider-execution boundary when it needs special file normalization, large-body handling, or protocol behavior. It is not the pattern for CRUD or other operations on protected Sim resources. For those, use the `migrate-application-operation` skill and an authorized application use case with the ordinary internal/v2 route builders.
Internal tool routes are HTTP boundaries and follow the same contract policy as public routes — define the request/response shape in `apps/sim/lib/api/contracts/tools/{service}.ts` (or an existing aggregate) and validate with canonical helpers from `@/lib/api/server`. Never write a route-local Zod schema. Authenticate and perform cheap admission before parsing or downloading files.
```typescript
// apps/sim/lib/api/contracts/tools/{service}.ts
@@ -0,0 +1,303 @@
---
description: Create or migrate a protected Sim resource operation in the shared Principal and application-use-case architecture across internal APIs, public or versioned APIs, Copilot, and other trusted tool adapters. Use when adding a protected endpoint, tool command, or CRUD method; removing route- or tool-local authorization and business logic; consolidating resource reads or writes behind semantic operation policies; or adding another surface to an existing application operation while preserving contracts, identity, errors, rate limits, audit, analytics, and compatibility behavior. Treat v1, uploads, streams, large bodies, bulk recursion, and polymorphic tools as explicitly scoped special cases.
---
# Create Or Migrate Application Operation
Create or migrate one bounded semantic operation at a time. Share authorization and business behavior without forcing internal APIs, public APIs, Copilot, and other tools to share authentication, input schemas, or response shapes.
## Enforce the application boundary
Apply this invariant:
> Every real operation on persisted or protected data enters through an authorized application use case.
This includes mutations, content and metadata reads, canonical resource lookup, and reference-to-resource resolution when the lookup is authorization-sensitive.
Surface helpers may:
- Normalize an already-authenticated surface context into a `Principal`.
- Translate aliases or wire arguments into application input.
- Select a code-defined operation and application use case.
- Call the application use case.
- Translate typed results and errors into the surface contract.
Surface helpers must not:
- Query databases or storage.
- Decide workspace or resource authorization.
- Implement business transactions.
- Record semantic audit or shared domain notifications.
- Infer authoritative identity, workspace, audience, or scope from untrusted arguments.
- Substitute billing attribution for identity.
A helper that resolves a path is valid only when the actual protected lookup runs through an authorized application resolver. If a helper begins doing real data work, move that work into an application use case.
## Read the foundation first
Read these files completely before editing:
- `packages/auth/src/principal.ts`
- `apps/sim/lib/core/application/operation.ts`
- `apps/sim/lib/core/application/workspace-operation.ts`
- `apps/sim/lib/core/application/workspace-authorization.ts`
- `apps/sim/lib/core/application/authorized-workspace-use-case.ts`
- `apps/sim/lib/api/server/routes/definition.ts`
- `apps/sim/lib/api/server/routes/internal-json-route.ts`
- `apps/sim/lib/api/server/routes/v2-json-route.ts`
- `apps/sim/lib/auth/internal-delegation.ts`
- `apps/sim/lib/copilot/application/application-adapter.ts`
- `apps/sim/lib/copilot/auth/application-delegation.ts`
Use the file domain only as a representative golden slice:
- `apps/sim/lib/workspace-files/application/operations.ts`
- `apps/sim/lib/workspace-files/application/authorized-workspace-file-use-case.ts`
- `apps/sim/lib/workspace-files/application/rename-workspace-file.ts`
- `apps/sim/lib/copilot/application/execute-file-use-case.ts`
- `apps/sim/lib/copilot/auth/file-delegation.ts`
Then read the target domain's operation registry, application code, repositories, contracts, adapters, aliases, resume paths, and focused tests. Fail immediately if the shared foundation is absent. Do not recreate it inside the domain.
## Bound the migration
Inventory every entry point for the behavior before editing:
- Internal HTTP routes and contracts.
- Public or versioned API routes and contracts.
- Copilot tools, aliases, resume paths, and polymorphic branches.
- Other tool servers, workflow executors, jobs, or service callers.
- Current authentication, authorization, workspace assertions, and concealment.
- Manager or orchestration call chains.
- Audit, notification, analytics, and billing side effects.
- Error/status/result behavior.
- Rate-limit identity, rollout gates, quota, and concurrency admission.
Classify each as `migrate`, `defer`, or `non-goal`. Do not migrate adjacent operations merely because they share a module. Do not modify v1 unless the request explicitly includes it.
Preserve behavior unless the task explicitly changes it. Stop and report a decision when surfaces currently disagree on security or compatibility behavior; do not silently choose one.
## Keep the layers distinct
Use these responsibilities:
1. Authentication adapter: verify the surface credential or trusted execution context and construct a `Principal`.
2. Route or tool adapter: select rate policy, parse its contract, translate input, call the application use case, and render its own result.
3. Application use case: load canonical context, compare asserted scope, authorize the semantic operation, execute business behavior, project semantic audit, and trigger shared domain effects.
4. Manager or repository: perform database and storage reads or writes using canonical identifiers and scope. Never accept credentials or principals.
5. Presenter: return only the surface success body or typed binary descriptor. Never construct auth, rate, or error behavior.
For ordinary public JSON routes, preserve this order:
```text
IP abuse limit
-> authenticate
-> build Principal
-> operation rate limit
-> parse surface contract
-> application use case
-> canonical load
-> asserted-scope concealment
-> current authorization
-> manager read or mutation
-> semantic audit
-> shared domain effects
-> surface presenter
```
Internal routes may omit the IP bucket or operation limit only through an explicit policy with a reason. Usage billing, storage quota, cost admission, and concurrency are separate from request-rate limiting.
Never query API keys or sessions from the application layer. Never add fallback identity or authorization behavior. Propagate infrastructure failures instead of turning them into not-found or forbidden results.
## Define the semantic operation once
Add one stable entry to the target domain's operation registry:
```ts
rename: defineWorkspaceOperation({
id: 'widgets.rename',
minimumRole: 'write',
workspaceApiKey: 'allow',
principalKinds: ['session', 'personal_api_key', 'workspace_api_key', 'delegated'],
delegatedServices: ['copilot'],
})
```
Do not create internal-, public-, or Copilot-specific versions of the same semantic operation. If two callers have materially different business or transactional semantics, define separate semantic operations and use cases and explain the distinction.
Choose principal kinds from actual behavior. Do not accept every principal merely because the use case is shared. Workspace API keys have a write ceiling and cannot satisfy admin operations. The operation definition must fail fast when its role, workspace-key policy, and principal kinds disagree.
Route declarations, tool adapters, and use cases must use the same literal operation. Runtime operation selection is permitted only from a trusted, code-defined registry. Never accept an operation ID or permission tag from an HTTP body, model argument, or other untrusted input.
## Implement the application use case
Use `defineAuthorizedWorkspaceUseCase` directly or a thin domain binding that supplies domain-specific authorization options:
```ts
export const renameWidget = defineAuthorizedWorkspaceUseCase({
operation: widgetOperations.rename,
resolveContext: ({ input }: { input: RenameWidgetInput }) =>
loadCanonicalWidgetContext(input.id, input.assertedWorkspaceId),
authorizationOptions: { delegation: widgetDelegationPolicy },
execute: async ({ input, context }) => renameWidgetRecord({
workspaceId: context.workspaceId,
widgetId: context.resourceId,
name: input.name,
}),
projectAudit: ({ result }) => ({
action: AuditAction.WIDGET_UPDATED,
resourceType: AuditResourceType.WIDGET,
resourceId: result.id,
resourceName: result.name,
}),
afterSuccess: ({ context }) => notifyWidgetsChanged(context.workspaceId),
})
```
Adapt the example to the domain's real authorization options; do not copy invented field names.
The wrapper must own this lifecycle:
1. Reject disallowed principal kinds before protected loading.
2. Load canonical context and conceal asserted-scope mismatches as required.
3. Authorize the operation using current policy state.
4. Execute the manager or repository primitive.
5. Project audit from authoritative results.
6. Await shared post-success effects.
Do not call shared authorization, principal audit attribution, or `recordAudit` manually from an ordinary migrated use-case body. Use `projectAudit` only when the operation has semantic audit. Return no audit entries for authoritative no-ops. Keep product analytics such as `captureServerEvent` surface-specific through the adapter's success hook.
Inspect legacy orchestration before reusing it. If it already authorizes, audits, notifies, or captures analytics, call a lower-level primitive or remove duplicate responsibility for migrated callers.
Application code must remain surface-neutral. It must not import `app/api/**`, `next/server`, internal/v1/v2 contracts or presenters, or Copilot tool handlers. Return domain values and let each surface presenter project its own wire result.
## Adapt internal APIs
Use `defineInternalJsonRoute` for ordinary JSON routes. Explicitly declare the contract, authentication policy, semantic operation, rate policy, error policy, input mapping, use case, and presenter when the wire result differs.
Use `internalSessionAuth` for session-only routes. Use `createInternalSessionOrExecutorAuth` only when the endpoint genuinely supports signed executor delegation; the semantic operation must then allow `delegated` principals from the `executor` service. Never turn an actorless legacy JWT into a fake session, owner, or user principal.
The internal adapter owns authentication and internal response envelopes. It must not implement workspace authorization. Preserve internal-only analytics through `onSuccess` after application success.
Keep the route module declarative. If several internal routes repeat authentication, parsing, error rendering, or response construction, improve the shared internal route builder instead of adding a domain-specific route wrapper.
## Adapt public or versioned APIs
Use the appropriate public/versioned route builder, such as `defineV2JsonRoute`, with API-key authentication, explicit semantic operation and rate policy, external error projection, input mapping, application use case, and an external presenter. V2 rollout admission is centralized by the builder; do not invent a route-local rollout policy.
Authentication and HTTP formatting may differ from internal APIs; authorization and business behavior must not. Rate-limit using the credential or principal subject, never a billed owner. Resolve billing attribution only for billing, quota, or legacy required-user fields.
Keep surface contracts separate when their wire shapes differ. Reuse shared primitive schemas and domain validators for invariants such as IDs, names, bounds, and formats. Do not maintain duplicate internal and external schemas merely because the routes are separate; import the same schema when the wire shape is genuinely identical. Never cast one surface response into another.
Keep v1 middleware and routes unchanged unless explicitly included.
## Adapt Copilot
Copilot is a surface adapter, not a separate application layer. If an HTTP or other surface already uses an application use case, Copilot must call that exact use case rather than reimplementing protected business behavior under `lib/copilot`.
Create one domain-level Copilot application adapter with `createCopilotApplicationAdapter` instead of constructing delegated principals in every tool:
```ts
executeCopilotWidgetUseCase(context, renameWidget, input, { resourceId })
```
That adapter must:
- Require a trusted server-authored Copilot execution marker.
- Require the authenticated subject, canonical workspace, tool-call or execution identity, and required audience or lifecycle scope.
- Construct the shared delegated `Principal` in one place.
- Optionally bind the canonical resource scope after trusted resolution.
- Verify that the use case exposes a registered code-defined operation.
- Use the domain's exact immutable operation registry so operation-object membership and identity are checked centrally.
- Call the application use case directly.
Never construct authoritative delegation from model-provided workspace IDs, user IDs, operation IDs, resource scope, or permission tags. Model arguments are requested targets only and must be checked against trusted execution context and canonical data.
Tool handlers own argument aliases, resumable legacy names, abort checks, tool-call reporting, and tool-specific presentation. They must not query managers directly for protected operations, manually authorize, or implement protected business behavior. If a Copilot-only compound action expresses real domain behavior, define a surface-neutral domain operation and application use case for it.
A Copilot reference helper may translate a path to a resource only by calling an authorized application resolver under the intended semantic operation. Passing a code-defined operation object is acceptable; passing a model-provided operation string is not. An immediate same-request resolver followed by the operation may reuse one trusted principal, though the application operation still performs its own canonical authorization. Fresh authentication and authorization are required across lifecycle boundaries such as resumed tool calls, executor callbacks, queued or background work, upload control legs and finalization, durable completion, and long-running provider operations. When resolution and execution form one business operation, need a consistent snapshot, or appear repeatedly together, prefer a top-level application use case such as `renameWidgetByReference`.
Surface adapters must not compose protected mutations. An atomic compound action requires one top-level semantic domain operation and application use case that owns the transaction and authoritative result. An explicitly best-effort application command may coordinate multiple operations only when it defines hard input and expansion caps, cancellation checkpoints, partial-result semantics, audit behavior, and rate/quota policy. Keep composition exceptional and explicit; ordinary tools should use the shared execution adapter.
Map expected typed errors to safe tool results. Unknown errors must become generic system/retryable messages while retaining full causes in server logs. Never return raw database or storage errors to the model.
## Adapt other internal or external tools
Treat every tool runtime as a surface adapter:
- Normalize its already-authenticated execution context into an existing `Principal` through one shared adapter for that runtime or domain.
- Call the same application use case used by HTTP and Copilot surfaces.
- Preserve the tool protocol's input, output, retry, and cancellation semantics.
- Keep authoritative workspace and subject scope server-authored.
An internal caller is not automatically trusted to bypass authorization. It must supply an explicit principal or use a deliberately designed service/delegation principal. If the current principal model cannot express its authority, stop and extend the identity model intentionally; do not fall back to an owner, uploader, creator, or arbitrary user ID.
External tool endpoints authenticate at their adapter exactly like public APIs. Do not authenticate again inside the application use case.
## Preserve identity and attribution
- Session and personal-key principals authorize through current human workspace permission.
- Personal API keys also respect the workspace's personal-key policy.
- Workspace keys authorize as the workspace under explicit operation policy and the write ceiling, independent of creator membership.
- Delegated principals re-check the current subject and their workspace, audience, expiry, execution, and resource scope.
- Billing owners are attribution for billing or legacy required columns only, never authorization, rate identity, delegated identity, audit actor, or human analytics identity.
- Preserve structured `PrincipalActor` metadata in semantic audit.
If a required legacy user column cannot represent the real actor, label the compatibility attribution explicitly. Never pretend it is the acting human.
## Handle special operations explicitly
Do not force these through an ordinary JSON migration:
- Upload or multipart lifecycles: bind immutable credential identity, reauthorize control legs and finalization, and make durable completion idempotent.
- Large bodies: authenticate and perform cheap admission before bounded buffering.
- Binary or streaming responses: use binary/stream builders and typed descriptors.
- Bulk or recursive operations: deduplicate and cap inputs and expansion, load all resources canonically, and define atomic versus best-effort behavior.
- Polymorphic tools: select the semantic operation only after trusted target-kind resolution; do not route unrelated branches through one domain registry.
- Multi-resource transactions: keep canonical scope predicates and derive audit from authoritative affected rows.
Stop and report a missing design rather than weakening identity, authorization, limits, or errors.
## Test the complete matrix
Add focused tests for every migrated surface and principal kind allowed by the operation:
- Application: allowed and disallowed roles, principal-kind rejection before canonical loading, workspace assertion mismatch, delegated scope, not found, conflict, no-op, and infrastructure propagation.
- Operation registry: role/workspace-key/principal-kind/delegated-service consistency and fail-fast rejection of invalid definitions.
- Repository: canonical active lookup, workspace-predicated writes, archived resources, authoritative affected rows, and database error propagation.
- Internal API: authentication before parsing, exact contract, typed errors, and surface analytics only after success.
- Public API: personal and workspace keys, rate and rollout behavior, concealment, exact external envelope, and rate headers.
- Copilot or tools: trusted context, exact registered operation membership, rejected forged scope, aliases and resume paths, permission re-check, safe errors, and unchanged tool result shapes.
- Side effects: audit derives from authoritative results; shared notifications follow audit; neither occurs for rejection or no-op.
Run at minimum:
```bash
bunx vitest run <focused test files>
bunx biome check <changed source and test files>
bunx turbo run type-check --filter=sim --filter=@sim/auth
bun run check:api-validation:strict
git diff --check
```
Do not claim a check passed unless it completed successfully.
## Work safely in parallel
- Assign non-overlapping route modules and caller sets. Two methods in one route file are one ownership unit.
- Treat operation registries, contract families, route policies, and shared surface adapters as merge hotspots.
- Keep shared core foundations owned by one task; ordinary domain migrations should consume them without modifying them.
- Preserve unrelated working-tree changes. Never stage proposal docs, lockfile drift, or another agent's edits.
- Do not commit, push, or open a PR unless requested.
## Hand off
Report:
1. Semantic operation, role, workspace-key policy, and principal kinds.
2. Migrated, deferred, and non-goal entry points.
3. Behavior preserved per internal, public, Copilot, and other tool surface.
4. Identity construction and authoritative scope source for each surface.
5. Files changed and shared merge hotspots.
6. Tests and checks run with results.
7. Remaining risks or blockers. Fail fast when an invariant could not be implemented.
+6 -1
View File
@@ -4,7 +4,12 @@
Import `createLogger` from `@sim/logger`. Use `logger.info`, `logger.warn`, `logger.error` instead of `console.log`. Inside API routes wrapped with `withRouteHandler`, loggers automatically include the request ID.
## API Route Handlers
All API route handlers must be wrapped with `withRouteHandler` from `@/lib/core/utils/with-route-handler`. Never export a bare `async function GET/POST/...` — always use `export const METHOD = withRouteHandler(...)`.
All API route handlers must run inside `withRouteHandler`. Ordinary internal and v2 JSON/binary handlers use the shared route builders, which already apply it; never double-wrap them. Use raw `withRouteHandler` only for documented protocol or lifecycle exceptions. Never export a bare `async function GET/POST/...`.
## Application Operation Boundary
Every protected read, write, canonical lookup, or authorization-sensitive reference resolution must enter through an authorized application use case. Surfaces authenticate and build a `Principal`, rate-limit, parse, map input, call the use case, and present their own result. They must not query protected data, decide resource authorization, implement business transactions, or record semantic audit.
Define one stable semantic operation with its role, workspace-key policy, principal kinds, and delegated services. Internal, v2, Copilot, and trusted-tool adapters call the same use case when domain behavior is the same. Copilot uses `createCopilotApplicationAdapter`; it is not a separate protected business layer. Protected compound mutations require one top-level semantic application operation. Never substitute billing attribution, an uploader, creator, or key owner for the acting principal. Use the `migrate-application-operation` skill for new or migrated protected operations.
## Comments
Use TSDoc for documentation. No `====` separators. No non-TSDoc comments.
+15
View File
@@ -38,6 +38,21 @@ packages/ # @sim/* — audit, auth, db, logger, realtime-protocol
- `apps/* → packages/*` only. Packages never import from `apps/*`.
- `apps/realtime` avoids Next.js, React, the block/tool registry, provider SDKs, and the executor; never add `@/lib/webhooks/providers/*`, `@/executor/*`, `@/blocks/*`, or `@/tools/*` imports to any package it consumes. CI enforces this via `scripts/check-monorepo-boundaries.ts` and `scripts/check-realtime-prune-graph.ts`.
## Protected Application Operations
Every real operation on protected or persisted data crosses one authorized application boundary:
1. The surface authenticates its credential or trusted context and constructs a `Principal`.
2. A fixed, code-defined semantic operation declares minimum role, workspace-key policy, allowed principal kinds, and delegated services.
3. The application use case loads canonical context, checks asserted scope, authorizes current access, executes the manager/repository, projects semantic audit, and runs shared domain effects.
4. The surface presents its own internal, v2, Copilot, or tool result.
Routes and tools must not query protected data, authorize resources, implement business transactions, or record semantic audit. Application modules must not import `app/api/**`, `next/server`, route contracts/presenters, or Copilot handlers. Copilot must call the same domain use case through `createCopilotApplicationAdapter`; do not create a second Copilot business implementation. Atomic compound mutations need one top-level semantic application operation rather than sequential surface calls.
Ordinary internal and v2 routes use the shared JSON/binary route builders. Those builders already apply `withRouteHandler`; do not double-wrap them. Use raw `withRouteHandler` only for explicit protocol, streaming, large-body, multipart, or lifecycle exceptions, while keeping protected business work inside application use cases.
Use the `migrate-application-operation` skill before creating or migrating a protected endpoint, tool command, or resource method.
## The `'use client'` server boundary
Every export of a `'use client'` module becomes a *client reference* on the server — server-evaluated code (RSC pages/layouts, `prefetch.ts`, route handlers, block definitions, triggers) can only *render* it as a component or pass it as a prop, never *call* it (doing so throws at runtime, e.g. `tableKeys.list is not a function`; `next build` does not catch it). Keep server-importable query primitives (key factories, fetchers, mappers, constants) in non-`'use client'` modules — see `.claude/rules/sim-queries.md`. Enforced by `scripts/check-client-boundary-imports.ts`.
+4 -2
View File
@@ -762,9 +762,11 @@ tools: {
}
```
#### 3. Create Internal API Route
#### 3. Create Special Internal Tool Execution Route
Create `apps/sim/app/api/tools/{service}/{action}/route.ts`. Internal tool routes are HTTP boundaries and follow the same contract policy as public routes — define the request/response shape in `apps/sim/lib/api/contracts/tools/{service}.ts` (or an existing aggregate) and validate with canonical helpers from `@/lib/api/server`. Never write a route-local Zod schema.
Create `apps/sim/app/api/tools/{service}/{action}/route.ts`. This raw route pattern is only for an integration's provider-execution boundary when it needs special file normalization, large-body handling, or protocol behavior. It is not the pattern for CRUD or other operations on protected Sim resources. For those, use the `migrate-application-operation` skill and an authorized application use case with the ordinary internal/v2 route builders.
Internal tool routes are HTTP boundaries and follow the same contract policy as public routes — define the request/response shape in `apps/sim/lib/api/contracts/tools/{service}.ts` (or an existing aggregate) and validate with canonical helpers from `@/lib/api/server`. Never write a route-local Zod schema. Authenticate and perform cheap admission before parsing or downloading files.
```typescript
// apps/sim/lib/api/contracts/tools/{service}.ts
@@ -0,0 +1,299 @@
# Create Or Migrate Application Operation
Create or migrate one bounded semantic operation at a time. Share authorization and business behavior without forcing internal APIs, public APIs, Copilot, and other tools to share authentication, input schemas, or response shapes.
## Enforce the application boundary
Apply this invariant:
> Every real operation on persisted or protected data enters through an authorized application use case.
This includes mutations, content and metadata reads, canonical resource lookup, and reference-to-resource resolution when the lookup is authorization-sensitive.
Surface helpers may:
- Normalize an already-authenticated surface context into a `Principal`.
- Translate aliases or wire arguments into application input.
- Select a code-defined operation and application use case.
- Call the application use case.
- Translate typed results and errors into the surface contract.
Surface helpers must not:
- Query databases or storage.
- Decide workspace or resource authorization.
- Implement business transactions.
- Record semantic audit or shared domain notifications.
- Infer authoritative identity, workspace, audience, or scope from untrusted arguments.
- Substitute billing attribution for identity.
A helper that resolves a path is valid only when the actual protected lookup runs through an authorized application resolver. If a helper begins doing real data work, move that work into an application use case.
## Read the foundation first
Read these files completely before editing:
- `packages/auth/src/principal.ts`
- `apps/sim/lib/core/application/operation.ts`
- `apps/sim/lib/core/application/workspace-operation.ts`
- `apps/sim/lib/core/application/workspace-authorization.ts`
- `apps/sim/lib/core/application/authorized-workspace-use-case.ts`
- `apps/sim/lib/api/server/routes/definition.ts`
- `apps/sim/lib/api/server/routes/internal-json-route.ts`
- `apps/sim/lib/api/server/routes/v2-json-route.ts`
- `apps/sim/lib/auth/internal-delegation.ts`
- `apps/sim/lib/copilot/application/application-adapter.ts`
- `apps/sim/lib/copilot/auth/application-delegation.ts`
Use the file domain only as a representative golden slice:
- `apps/sim/lib/workspace-files/application/operations.ts`
- `apps/sim/lib/workspace-files/application/authorized-workspace-file-use-case.ts`
- `apps/sim/lib/workspace-files/application/rename-workspace-file.ts`
- `apps/sim/lib/copilot/application/execute-file-use-case.ts`
- `apps/sim/lib/copilot/auth/file-delegation.ts`
Then read the target domain's operation registry, application code, repositories, contracts, adapters, aliases, resume paths, and focused tests. Fail immediately if the shared foundation is absent. Do not recreate it inside the domain.
## Bound the migration
Inventory every entry point for the behavior before editing:
- Internal HTTP routes and contracts.
- Public or versioned API routes and contracts.
- Copilot tools, aliases, resume paths, and polymorphic branches.
- Other tool servers, workflow executors, jobs, or service callers.
- Current authentication, authorization, workspace assertions, and concealment.
- Manager or orchestration call chains.
- Audit, notification, analytics, and billing side effects.
- Error/status/result behavior.
- Rate-limit identity, rollout gates, quota, and concurrency admission.
Classify each as `migrate`, `defer`, or `non-goal`. Do not migrate adjacent operations merely because they share a module. Do not modify v1 unless the request explicitly includes it.
Preserve behavior unless the task explicitly changes it. Stop and report a decision when surfaces currently disagree on security or compatibility behavior; do not silently choose one.
## Keep the layers distinct
Use these responsibilities:
1. Authentication adapter: verify the surface credential or trusted execution context and construct a `Principal`.
2. Route or tool adapter: select rate policy, parse its contract, translate input, call the application use case, and render its own result.
3. Application use case: load canonical context, compare asserted scope, authorize the semantic operation, execute business behavior, project semantic audit, and trigger shared domain effects.
4. Manager or repository: perform database and storage reads or writes using canonical identifiers and scope. Never accept credentials or principals.
5. Presenter: return only the surface success body or typed binary descriptor. Never construct auth, rate, or error behavior.
For ordinary public JSON routes, preserve this order:
```text
IP abuse limit
-> authenticate
-> build Principal
-> operation rate limit
-> parse surface contract
-> application use case
-> canonical load
-> asserted-scope concealment
-> current authorization
-> manager read or mutation
-> semantic audit
-> shared domain effects
-> surface presenter
```
Internal routes may omit the IP bucket or operation limit only through an explicit policy with a reason. Usage billing, storage quota, cost admission, and concurrency are separate from request-rate limiting.
Never query API keys or sessions from the application layer. Never add fallback identity or authorization behavior. Propagate infrastructure failures instead of turning them into not-found or forbidden results.
## Define the semantic operation once
Add one stable entry to the target domain's operation registry:
```ts
rename: defineWorkspaceOperation({
id: 'widgets.rename',
minimumRole: 'write',
workspaceApiKey: 'allow',
principalKinds: ['session', 'personal_api_key', 'workspace_api_key', 'delegated'],
delegatedServices: ['copilot'],
})
```
Do not create internal-, public-, or Copilot-specific versions of the same semantic operation. If two callers have materially different business or transactional semantics, define separate semantic operations and use cases and explain the distinction.
Choose principal kinds from actual behavior. Do not accept every principal merely because the use case is shared. Workspace API keys have a write ceiling and cannot satisfy admin operations. The operation definition must fail fast when its role, workspace-key policy, and principal kinds disagree.
Route declarations, tool adapters, and use cases must use the same literal operation. Runtime operation selection is permitted only from a trusted, code-defined registry. Never accept an operation ID or permission tag from an HTTP body, model argument, or other untrusted input.
## Implement the application use case
Use `defineAuthorizedWorkspaceUseCase` directly or a thin domain binding that supplies domain-specific authorization options:
```ts
export const renameWidget = defineAuthorizedWorkspaceUseCase({
operation: widgetOperations.rename,
resolveContext: ({ input }: { input: RenameWidgetInput }) =>
loadCanonicalWidgetContext(input.id, input.assertedWorkspaceId),
authorizationOptions: { delegation: widgetDelegationPolicy },
execute: async ({ input, context }) => renameWidgetRecord({
workspaceId: context.workspaceId,
widgetId: context.resourceId,
name: input.name,
}),
projectAudit: ({ result }) => ({
action: AuditAction.WIDGET_UPDATED,
resourceType: AuditResourceType.WIDGET,
resourceId: result.id,
resourceName: result.name,
}),
afterSuccess: ({ context }) => notifyWidgetsChanged(context.workspaceId),
})
```
Adapt the example to the domain's real authorization options; do not copy invented field names.
The wrapper must own this lifecycle:
1. Reject disallowed principal kinds before protected loading.
2. Load canonical context and conceal asserted-scope mismatches as required.
3. Authorize the operation using current policy state.
4. Execute the manager or repository primitive.
5. Project audit from authoritative results.
6. Await shared post-success effects.
Do not call shared authorization, principal audit attribution, or `recordAudit` manually from an ordinary migrated use-case body. Use `projectAudit` only when the operation has semantic audit. Return no audit entries for authoritative no-ops. Keep product analytics such as `captureServerEvent` surface-specific through the adapter's success hook.
Inspect legacy orchestration before reusing it. If it already authorizes, audits, notifies, or captures analytics, call a lower-level primitive or remove duplicate responsibility for migrated callers.
Application code must remain surface-neutral. It must not import `app/api/**`, `next/server`, internal/v1/v2 contracts or presenters, or Copilot tool handlers. Return domain values and let each surface presenter project its own wire result.
## Adapt internal APIs
Use `defineInternalJsonRoute` for ordinary JSON routes. Explicitly declare the contract, authentication policy, semantic operation, rate policy, error policy, input mapping, use case, and presenter when the wire result differs.
Use `internalSessionAuth` for session-only routes. Use `createInternalSessionOrExecutorAuth` only when the endpoint genuinely supports signed executor delegation; the semantic operation must then allow `delegated` principals from the `executor` service. Never turn an actorless legacy JWT into a fake session, owner, or user principal.
The internal adapter owns authentication and internal response envelopes. It must not implement workspace authorization. Preserve internal-only analytics through `onSuccess` after application success.
Keep the route module declarative. If several internal routes repeat authentication, parsing, error rendering, or response construction, improve the shared internal route builder instead of adding a domain-specific route wrapper.
## Adapt public or versioned APIs
Use the appropriate public/versioned route builder, such as `defineV2JsonRoute`, with API-key authentication, explicit semantic operation and rate policy, external error projection, input mapping, application use case, and an external presenter. V2 rollout admission is centralized by the builder; do not invent a route-local rollout policy.
Authentication and HTTP formatting may differ from internal APIs; authorization and business behavior must not. Rate-limit using the credential or principal subject, never a billed owner. Resolve billing attribution only for billing, quota, or legacy required-user fields.
Keep surface contracts separate when their wire shapes differ. Reuse shared primitive schemas and domain validators for invariants such as IDs, names, bounds, and formats. Do not maintain duplicate internal and external schemas merely because the routes are separate; import the same schema when the wire shape is genuinely identical. Never cast one surface response into another.
Keep v1 middleware and routes unchanged unless explicitly included.
## Adapt Copilot
Copilot is a surface adapter, not a separate application layer. If an HTTP or other surface already uses an application use case, Copilot must call that exact use case rather than reimplementing protected business behavior under `lib/copilot`.
Create one domain-level Copilot application adapter with `createCopilotApplicationAdapter` instead of constructing delegated principals in every tool:
```ts
executeCopilotWidgetUseCase(context, renameWidget, input, { resourceId })
```
That adapter must:
- Require a trusted server-authored Copilot execution marker.
- Require the authenticated subject, canonical workspace, tool-call or execution identity, and required audience or lifecycle scope.
- Construct the shared delegated `Principal` in one place.
- Optionally bind the canonical resource scope after trusted resolution.
- Verify that the use case exposes a registered code-defined operation.
- Use the domain's exact immutable operation registry so operation-object membership and identity are checked centrally.
- Call the application use case directly.
Never construct authoritative delegation from model-provided workspace IDs, user IDs, operation IDs, resource scope, or permission tags. Model arguments are requested targets only and must be checked against trusted execution context and canonical data.
Tool handlers own argument aliases, resumable legacy names, abort checks, tool-call reporting, and tool-specific presentation. They must not query managers directly for protected operations, manually authorize, or implement protected business behavior. If a Copilot-only compound action expresses real domain behavior, define a surface-neutral domain operation and application use case for it.
A Copilot reference helper may translate a path to a resource only by calling an authorized application resolver under the intended semantic operation. Passing a code-defined operation object is acceptable; passing a model-provided operation string is not. An immediate same-request resolver followed by the operation may reuse one trusted principal, though the application operation still performs its own canonical authorization. Fresh authentication and authorization are required across lifecycle boundaries such as resumed tool calls, executor callbacks, queued or background work, upload control legs and finalization, durable completion, and long-running provider operations. When resolution and execution form one business operation, need a consistent snapshot, or appear repeatedly together, prefer a top-level application use case such as `renameWidgetByReference`.
Surface adapters must not compose protected mutations. An atomic compound action requires one top-level semantic domain operation and application use case that owns the transaction and authoritative result. An explicitly best-effort application command may coordinate multiple operations only when it defines hard input and expansion caps, cancellation checkpoints, partial-result semantics, audit behavior, and rate/quota policy. Keep composition exceptional and explicit; ordinary tools should use the shared execution adapter.
Map expected typed errors to safe tool results. Unknown errors must become generic system/retryable messages while retaining full causes in server logs. Never return raw database or storage errors to the model.
## Adapt other internal or external tools
Treat every tool runtime as a surface adapter:
- Normalize its already-authenticated execution context into an existing `Principal` through one shared adapter for that runtime or domain.
- Call the same application use case used by HTTP and Copilot surfaces.
- Preserve the tool protocol's input, output, retry, and cancellation semantics.
- Keep authoritative workspace and subject scope server-authored.
An internal caller is not automatically trusted to bypass authorization. It must supply an explicit principal or use a deliberately designed service/delegation principal. If the current principal model cannot express its authority, stop and extend the identity model intentionally; do not fall back to an owner, uploader, creator, or arbitrary user ID.
External tool endpoints authenticate at their adapter exactly like public APIs. Do not authenticate again inside the application use case.
## Preserve identity and attribution
- Session and personal-key principals authorize through current human workspace permission.
- Personal API keys also respect the workspace's personal-key policy.
- Workspace keys authorize as the workspace under explicit operation policy and the write ceiling, independent of creator membership.
- Delegated principals re-check the current subject and their workspace, audience, expiry, execution, and resource scope.
- Billing owners are attribution for billing or legacy required columns only, never authorization, rate identity, delegated identity, audit actor, or human analytics identity.
- Preserve structured `PrincipalActor` metadata in semantic audit.
If a required legacy user column cannot represent the real actor, label the compatibility attribution explicitly. Never pretend it is the acting human.
## Handle special operations explicitly
Do not force these through an ordinary JSON migration:
- Upload or multipart lifecycles: bind immutable credential identity, reauthorize control legs and finalization, and make durable completion idempotent.
- Large bodies: authenticate and perform cheap admission before bounded buffering.
- Binary or streaming responses: use binary/stream builders and typed descriptors.
- Bulk or recursive operations: deduplicate and cap inputs and expansion, load all resources canonically, and define atomic versus best-effort behavior.
- Polymorphic tools: select the semantic operation only after trusted target-kind resolution; do not route unrelated branches through one domain registry.
- Multi-resource transactions: keep canonical scope predicates and derive audit from authoritative affected rows.
Stop and report a missing design rather than weakening identity, authorization, limits, or errors.
## Test the complete matrix
Add focused tests for every migrated surface and principal kind allowed by the operation:
- Application: allowed and disallowed roles, principal-kind rejection before canonical loading, workspace assertion mismatch, delegated scope, not found, conflict, no-op, and infrastructure propagation.
- Operation registry: role/workspace-key/principal-kind/delegated-service consistency and fail-fast rejection of invalid definitions.
- Repository: canonical active lookup, workspace-predicated writes, archived resources, authoritative affected rows, and database error propagation.
- Internal API: authentication before parsing, exact contract, typed errors, and surface analytics only after success.
- Public API: personal and workspace keys, rate and rollout behavior, concealment, exact external envelope, and rate headers.
- Copilot or tools: trusted context, exact registered operation membership, rejected forged scope, aliases and resume paths, permission re-check, safe errors, and unchanged tool result shapes.
- Side effects: audit derives from authoritative results; shared notifications follow audit; neither occurs for rejection or no-op.
Run at minimum:
```bash
bunx vitest run <focused test files>
bunx biome check <changed source and test files>
bunx turbo run type-check --filter=sim --filter=@sim/auth
bun run check:api-validation:strict
git diff --check
```
Do not claim a check passed unless it completed successfully.
## Work safely in parallel
- Assign non-overlapping route modules and caller sets. Two methods in one route file are one ownership unit.
- Treat operation registries, contract families, route policies, and shared surface adapters as merge hotspots.
- Keep shared core foundations owned by one task; ordinary domain migrations should consume them without modifying them.
- Preserve unrelated working-tree changes. Never stage proposal docs, lockfile drift, or another agent's edits.
- Do not commit, push, or open a PR unless requested.
## Hand off
Report:
1. Semantic operation, role, workspace-key policy, and principal kinds.
2. Migrated, deferred, and non-goal entry points.
3. Behavior preserved per internal, public, Copilot, and other tool surface.
4. Identity construction and authoritative scope source for each surface.
5. Files changed and shared merge hotspots.
6. Tests and checks run with results.
7. Remaining risks or blockers. Fail fast when an invariant could not be implemented.
+6 -1
View File
@@ -8,7 +8,12 @@ alwaysApply: true
Import `createLogger` from `@sim/logger`. Use `logger.info`, `logger.warn`, `logger.error` instead of `console.log`. Inside API routes wrapped with `withRouteHandler`, loggers automatically include the request ID.
## API Route Handlers
All API route handlers must be wrapped with `withRouteHandler` from `@/lib/core/utils/with-route-handler`. Never export a bare `async function GET/POST/...` — always use `export const METHOD = withRouteHandler(...)`.
All API route handlers must run inside `withRouteHandler`. Ordinary internal and v2 JSON/binary handlers use the shared route builders, which already apply it; never double-wrap them. Use raw `withRouteHandler` only for documented protocol or lifecycle exceptions. Never export a bare `async function GET/POST/...`.
## Application Operation Boundary
Every protected read, write, canonical lookup, or authorization-sensitive reference resolution must enter through an authorized application use case. Surfaces authenticate and build a `Principal`, rate-limit, parse, map input, call the use case, and present their own result. They must not query protected data, decide resource authorization, implement business transactions, or record semantic audit.
Define one stable semantic operation with its role, workspace-key policy, principal kinds, and delegated services. Internal, v2, Copilot, and trusted-tool adapters call the same use case when domain behavior is the same. Copilot uses `createCopilotApplicationAdapter`; it is not a separate protected business layer. Protected compound mutations require one top-level semantic application operation. Never substitute billing attribution, an uploader, creator, or key owner for the acting principal. Use the `migrate-application-operation` skill for new or migrated protected operations.
## Comments
Use TSDoc for documentation. No `====` separators. No non-TSDoc comments.
+15
View File
@@ -37,6 +37,21 @@ packages/ # @sim/* — audit, auth, db, logger, realtime-protocol
- `apps/* → packages/*` only. Packages never import from `apps/*`.
- `apps/realtime` avoids Next.js, React, the block/tool registry, provider SDKs, and the executor; never add `@/lib/webhooks/providers/*`, `@/executor/*`, `@/blocks/*`, or `@/tools/*` imports to any package it consumes. CI enforces this via `scripts/check-monorepo-boundaries.ts` and `scripts/check-realtime-prune-graph.ts`.
## Protected Application Operations
Every real operation on protected or persisted data crosses one authorized application boundary:
1. The surface authenticates its credential or trusted context and constructs a `Principal`.
2. A fixed, code-defined semantic operation declares minimum role, workspace-key policy, allowed principal kinds, and delegated services.
3. The application use case loads canonical context, checks asserted scope, authorizes current access, executes the manager/repository, projects semantic audit, and runs shared domain effects.
4. The surface presents its own internal, v2, Copilot, or tool result.
Routes and tools must not query protected data, authorize resources, implement business transactions, or record semantic audit. Application modules must not import `app/api/**`, `next/server`, route contracts/presenters, or Copilot handlers. Copilot must call the same domain use case through `createCopilotApplicationAdapter`; do not create a second Copilot business implementation. Atomic compound mutations need one top-level semantic application operation rather than sequential surface calls.
Ordinary internal and v2 routes use the shared JSON/binary route builders. Those builders already apply `withRouteHandler`; do not double-wrap them. Use raw `withRouteHandler` only for explicit protocol, streaming, large-body, multipart, or lifecycle exceptions, while keeping protected business work inside application use cases.
Use the `migrate-application-operation` skill before creating or migrating a protected endpoint, tool command, or resource method.
## Feature Organization
Features live under `app/workspace/[workspaceId]/`:
+34 -31
View File
@@ -6,7 +6,7 @@ You are a professional software engineer. All code must follow best practices: a
- **Linting / Audit**: `bun run check:api-validation` must pass on PRs. Do not introduce route-local boundary Zod schemas, direct route Zod imports, or ad-hoc client wire types — see "API Contracts" and "API Route Pattern" below
- **Logging**: Import `createLogger` from `@sim/logger`. Use `logger.info`, `logger.warn`, `logger.error` instead of `console.log`. Inside API routes wrapped with `withRouteHandler`, loggers automatically include the request ID — no manual `withMetadata({ requestId })` needed
- **API Route Handlers**: All API route handlers (`GET`, `POST`, `PUT`, `DELETE`, `PATCH`) must be wrapped with `withRouteHandler` from `@/lib/core/utils/with-route-handler`. This provides request ID tracking, automatic error logging for 4xx/5xx responses, and unhandled error catching. See "API Route Pattern" section below
- **API Route Handlers**: All API route handlers (`GET`, `POST`, `PUT`, `DELETE`, `PATCH`) must run inside `withRouteHandler`. Ordinary internal and v2 handlers use the shared JSON/binary route builders, which already apply it; never double-wrap a builder. Use raw `withRouteHandler` only for documented protocol or lifecycle exceptions. See "API Route Pattern" below
- **Comments**: Use TSDoc for documentation. No `====` separators. No non-TSDoc comments
- **Styling**: Never update global styles. Keep all styling local to components
- **ID Generation**: Never use `crypto.randomUUID()`, `nanoid`, or `uuid` package. Use `generateId()` (UUID v4) or `generateShortId()` (compact) from `@sim/utils/id`
@@ -29,6 +29,17 @@ You are a professional software engineer. All code must follow best practices: a
3. Type Safety First: TypeScript interfaces for all props, state, return types
4. Predictable State: Zustand for global state, useState for UI-only concerns
### Application Operation Boundary
- Every protected read, write, canonical resource lookup, or authorization-sensitive reference resolution enters through an authorized application use case.
- Define one stable semantic operation with its minimum role, workspace-key policy, allowed principal kinds, and delegated services. Internal APIs, v2 APIs, Copilot, and trusted tools call the same use case when the domain behavior is the same.
- Surface adapters authenticate and construct a `Principal`, apply request-rate policy, parse contracts, map input, and present results. They never query protected data, decide resource authorization, implement business transactions, or record semantic audit.
- Application use cases load canonical context, compare asserted scope, authorize current access, execute managers/repositories, project semantic audit, and trigger shared domain effects. Managers accept canonical IDs and scope, never credentials or principals.
- Copilot is a surface adapter. Use `createCopilotApplicationAdapter` and the domain's registered operation object; do not create Copilot-only authorization or business implementations.
- Protected compound mutations belong in one top-level semantic application operation. Do not sequence independently committing mutations in a route or tool adapter.
- Never substitute a billing owner, uploader, creator, or API-key owner for the acting principal. Fail fast when the identity model or operation policy cannot express the caller.
- Use the `migrate-application-operation` skill whenever creating or migrating a protected endpoint, tool command, or resource method.
### Root Structure
```
@@ -166,56 +177,49 @@ const provider = config as unknown as LegacyProvider
## API Route Pattern
Every API route handler must be wrapped with `withRouteHandler`. This sets up `AsyncLocalStorage`-based request context so all loggers in the request lifecycle automatically include the request ID.
Every route method must run inside `withRouteHandler`. Ordinary internal and v2 JSON/binary routes use `defineInternalJsonRoute`, `defineV2JsonRoute`, or the matching binary/stream builder. These builders already apply `withRouteHandler`; never wrap them again. Use raw `withRouteHandler` only for explicit protocol or lifecycle exceptions such as streaming, multipart control, large-body admission, OAuth, or public execution.
Routes never `import { z } from 'zod'` and never define route-local boundary schemas. They consume the contract from `@/lib/api/contracts/**` and validate with canonical helpers from `@/lib/api/server`:
Routes never `import { z } from 'zod'` and never define route-local boundary schemas. Declarative builders consume contracts and own authentication, admission, parsing, use-case execution, response validation, and error projection. A raw special route consumes the same contracts and validates with canonical helpers from `@/lib/api/server`, after authentication and cheap admission:
- `parseRequest(contract, request, context, options?)` — fully contract-bound routes; parses params, query, body, and headers in one call. Pass `{}` for `context` on routes without route params, or the route's `context` argument when route params exist. Returns a discriminated union; check `parsed.success` and return `parsed.response` on failure
- `validationErrorResponse(error)` and `getValidationErrorMessage(error, fallback)` — produce 400 responses from a `ZodError`
- `validationErrorResponseFromError(error)` — when handling unknown caught errors that may or may not be a `ZodError`
- `isZodError(error)` — type guard. Routes never use `instanceof z.ZodError`
### Fully contract-bound route (`parseRequest`)
### Ordinary authorized JSON route
```typescript
import { createLogger } from '@sim/logger'
import type { NextRequest } from 'next/server'
import { NextResponse } from 'next/server'
import { createFolderContract } from '@/lib/api/contracts/folders'
import { parseRequest } from '@/lib/api/server'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
const logger = createLogger('FoldersAPI')
export const POST = withRouteHandler(async (request: NextRequest) => {
const parsed = await parseRequest(createFolderContract, request, {})
if (!parsed.success) return parsed.response
const { body } = parsed.data
logger.info('Creating folder', { workspaceId: body.workspaceId })
return NextResponse.json({ ok: true })
export const PATCH = defineInternalJsonRoute({
contract: renameWidgetContract,
auth: internalSessionAuth,
operation: widgetOperations.rename,
rateLimit: internalRateLimits.none({ reason: 'Preserve existing internal behavior' }),
errorPolicy: internalWidgetErrorPolicy,
mapInput: ({ params, body }) => ({
widgetId: params.widgetId,
assertedWorkspaceId: params.workspaceId,
name: body.name,
}),
useCase: renameWidget,
present: ({ widget }) => ({ success: true, widget }),
})
```
### Composing with other middleware
```typescript
export const POST = withRouteHandler(withAdminAuth(async (request) => {
return NextResponse.json({ ok: true })
}))
```
The contract, operation, and use case must agree at definition time. Authentication and request-rate admission happen before parsing; canonical loading and authorization happen in the application use case. The presenter returns only the surface success body.
Routes under `apps/sim/app/api/v1/**` use the shared middleware in `apps/sim/app/api/v1/middleware.ts` for auth, rate-limit, and workspace access. Compose contract validation inside that middleware — never reimplement auth/rate-limit per-route.
Never export a bare `async function GET/POST/...` — always use `export const METHOD = withRouteHandler(...)`.
Never export a bare `async function GET/POST/...`. Export the result of a shared builder or, for a documented special route, `withRouteHandler(...)`.
### Adding a new boundary feature end-to-end
When adding a new route + client surface, follow this order. Each step has one place it lives.
1. **Author the contract first** in `apps/sim/lib/api/contracts/<domain>.ts` (or a subdirectory for large domains: `knowledge/`, `selectors/`, `tools/`). Define one schema per request slice (`params`, `query`, `body`, `headers`) and one for the response, then wrap with `defineRouteContract`. Export named type aliases (`z.input` for inputs, `z.output` for outputs).
2. **Implement the route** in `apps/sim/app/api/<path>/route.ts`. Auth always runs **before** `parseRequest` — never validate untrusted input before authenticating the caller. The route returns exactly the shape declared in `contract.response.schema`.
3. **Add the React Query hook** in `apps/sim/hooks/queries/<domain>.ts`. Use `requestJson(contract, input)` for the call. Build a hierarchical query-key factory (`all` → `lists()` → `list(workspaceId)` → `details()` → `detail(id)`) so invalidations can target prefixes.
4. **Use the hook in the component**. The mutation's `data` and `error` are fully typed from the contract; surface `error.message` (already extracted from the response body's `error` or `message` field by `requestJson`).
2. **Define the semantic operation and application use case** under `apps/sim/lib/<domain>/application/`. The use case owns canonical loading, asserted-scope checks, current authorization, business behavior, semantic audit, and shared domain effects.
3. **Implement the route adapter** in `apps/sim/app/api/<path>/route.ts` with the appropriate shared builder. Declare auth, operation, rate policy, error policy, input mapping, use case, and presenter. Auth always runs **before** parsing. Use raw `withRouteHandler` only for an explicit special route, and keep protected work in application use cases.
4. **Add the React Query hook** in `apps/sim/hooks/queries/<domain>.ts`. Use `requestJson(contract, input)` for the call. Build a hierarchical query-key factory (`all` → `lists()` → `list(workspaceId)` → `details()` → `detail(id)`) so invalidations can target prefixes.
5. **Use the hook in the component**. The mutation's `data` and `error` are fully typed from the contract; surface `error.message` (already extracted from the response body's `error` or `message` field by `requestJson`).
### Schema review checklist (read the contract diff like a DB migration)
@@ -473,4 +477,3 @@ For the full authoring instructions — SubBlock property tables, `condition`/`d
Table column types are registry entries in `apps/sim/lib/table/column-types/` — one file per type owning its label, icon, storage cast, coercion, validation, conversion compatibility, formatting, and editor. `Record<ColumnType, …>` on `registry.ts` and `registry.server.ts` is a compile-time completeness gate: adding a type to the union errors until both entries exist.
Never add a `case 'sometype':` outside `column-types/` — a missing arm fails silently (a wrong `jsonbCast` breaks every filter on the column). If a consumer needs per-type knowledge, add a registry field. Use `/add-column-type` for the full procedure.
+34 -31
View File
@@ -6,7 +6,7 @@ You are a professional software engineer. All code must follow best practices: a
- **Linting / Audit**: `bun run check:api-validation` must pass on PRs. Do not introduce route-local boundary Zod schemas, direct route Zod imports, or ad-hoc client wire types — see "API Contracts" and "API Route Pattern" below
- **Logging**: Import `createLogger` from `@sim/logger`. Use `logger.info`, `logger.warn`, `logger.error` instead of `console.log`. Inside API routes wrapped with `withRouteHandler`, loggers automatically include the request ID — no manual `withMetadata({ requestId })` needed
- **API Route Handlers**: All API route handlers (`GET`, `POST`, `PUT`, `DELETE`, `PATCH`) must be wrapped with `withRouteHandler` from `@/lib/core/utils/with-route-handler`. This provides request ID tracking, automatic error logging for 4xx/5xx responses, and unhandled error catching. See "API Route Pattern" section below
- **API Route Handlers**: All API route handlers (`GET`, `POST`, `PUT`, `DELETE`, `PATCH`) must run inside `withRouteHandler`. Ordinary internal and v2 handlers use the shared JSON/binary route builders, which already apply it; never double-wrap a builder. Use raw `withRouteHandler` only for documented protocol or lifecycle exceptions. See "API Route Pattern" below
- **Comments**: Use TSDoc for documentation. No `====` separators. No non-TSDoc comments
- **Styling**: Never update global styles. Keep all styling local to components
- **ID Generation**: Never use `crypto.randomUUID()`, `nanoid`, or `uuid` package. Use `generateId()` (UUID v4) or `generateShortId()` (compact) from `@sim/utils/id`
@@ -30,6 +30,17 @@ You are a professional software engineer. All code must follow best practices: a
3. Type Safety First: TypeScript interfaces for all props, state, return types
4. Predictable State: Zustand for global state, useState for UI-only concerns
### Application Operation Boundary
- Every protected read, write, canonical resource lookup, or authorization-sensitive reference resolution enters through an authorized application use case.
- Define one stable semantic operation with its minimum role, workspace-key policy, allowed principal kinds, and delegated services. Internal APIs, v2 APIs, Copilot, and trusted tools call the same use case when the domain behavior is the same.
- Surface adapters authenticate and construct a `Principal`, apply request-rate policy, parse contracts, map input, and present results. They never query protected data, decide resource authorization, implement business transactions, or record semantic audit.
- Application use cases load canonical context, compare asserted scope, authorize current access, execute managers/repositories, project semantic audit, and trigger shared domain effects. Managers accept canonical IDs and scope, never credentials or principals.
- Copilot is a surface adapter. Use `createCopilotApplicationAdapter` and the domain's registered operation object; do not create Copilot-only authorization or business implementations.
- Protected compound mutations belong in one top-level semantic application operation. Do not sequence independently committing mutations in a route or tool adapter.
- Never substitute a billing owner, uploader, creator, or API-key owner for the acting principal. Fail fast when the identity model or operation policy cannot express the caller.
- Use the `migrate-application-operation` skill whenever creating or migrating a protected endpoint, tool command, or resource method.
### Root Structure
```
@@ -169,56 +180,49 @@ const provider = config as unknown as LegacyProvider
## API Route Pattern
Every API route handler must be wrapped with `withRouteHandler`. This sets up `AsyncLocalStorage`-based request context so all loggers in the request lifecycle automatically include the request ID.
Every route method must run inside `withRouteHandler`. Ordinary internal and v2 JSON/binary routes use `defineInternalJsonRoute`, `defineV2JsonRoute`, or the matching binary/stream builder. These builders already apply `withRouteHandler`; never wrap them again. Use raw `withRouteHandler` only for explicit protocol or lifecycle exceptions such as streaming, multipart control, large-body admission, OAuth, or public execution.
Routes never `import { z } from 'zod'` and never define route-local boundary schemas. They consume the contract from `@/lib/api/contracts/**` and validate with canonical helpers from `@/lib/api/server`:
Routes never `import { z } from 'zod'` and never define route-local boundary schemas. Declarative builders consume contracts and own authentication, admission, parsing, use-case execution, response validation, and error projection. A raw special route consumes the same contracts and validates with canonical helpers from `@/lib/api/server`, after authentication and cheap admission:
- `parseRequest(contract, request, context, options?)` — fully contract-bound routes; parses params, query, body, and headers in one call. Pass `{}` for `context` on routes without route params, or the route's `context` argument when route params exist. Returns a discriminated union; check `parsed.success` and return `parsed.response` on failure
- `validationErrorResponse(error)` and `getValidationErrorMessage(error, fallback)` — produce 400 responses from a `ZodError`
- `validationErrorResponseFromError(error)` — when handling unknown caught errors that may or may not be a `ZodError`
- `isZodError(error)` — type guard. Routes never use `instanceof z.ZodError`
### Fully contract-bound route (`parseRequest`)
### Ordinary authorized JSON route
```typescript
import { createLogger } from '@sim/logger'
import type { NextRequest } from 'next/server'
import { NextResponse } from 'next/server'
import { createFolderContract } from '@/lib/api/contracts/folders'
import { parseRequest } from '@/lib/api/server'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
const logger = createLogger('FoldersAPI')
export const POST = withRouteHandler(async (request: NextRequest) => {
const parsed = await parseRequest(createFolderContract, request, {})
if (!parsed.success) return parsed.response
const { body } = parsed.data
logger.info('Creating folder', { workspaceId: body.workspaceId })
return NextResponse.json({ ok: true })
export const PATCH = defineInternalJsonRoute({
contract: renameWidgetContract,
auth: internalSessionAuth,
operation: widgetOperations.rename,
rateLimit: internalRateLimits.none({ reason: 'Preserve existing internal behavior' }),
errorPolicy: internalWidgetErrorPolicy,
mapInput: ({ params, body }) => ({
widgetId: params.widgetId,
assertedWorkspaceId: params.workspaceId,
name: body.name,
}),
useCase: renameWidget,
present: ({ widget }) => ({ success: true, widget }),
})
```
### Composing with other middleware
```typescript
export const POST = withRouteHandler(withAdminAuth(async (request) => {
return NextResponse.json({ ok: true })
}))
```
The contract, operation, and use case must agree at definition time. Authentication and request-rate admission happen before parsing; canonical loading and authorization happen in the application use case. The presenter returns only the surface success body.
Routes under `apps/sim/app/api/v1/**` use the shared middleware in `apps/sim/app/api/v1/middleware.ts` for auth, rate-limit, and workspace access. Compose contract validation inside that middleware — never reimplement auth/rate-limit per-route.
Never export a bare `async function GET/POST/...` — always use `export const METHOD = withRouteHandler(...)`.
Never export a bare `async function GET/POST/...`. Export the result of a shared builder or, for a documented special route, `withRouteHandler(...)`.
### Adding a new boundary feature end-to-end
When adding a new route + client surface, follow this order. Each step has one place it lives.
1. **Author the contract first** in `apps/sim/lib/api/contracts/<domain>.ts` (or a subdirectory for large domains: `knowledge/`, `selectors/`, `tools/`). Define one schema per request slice (`params`, `query`, `body`, `headers`) and one for the response, then wrap with `defineRouteContract`. Export named type aliases (`z.input` for inputs, `z.output` for outputs).
2. **Implement the route** in `apps/sim/app/api/<path>/route.ts`. Auth always runs **before** `parseRequest` — never validate untrusted input before authenticating the caller. The route returns exactly the shape declared in `contract.response.schema`.
3. **Add the React Query hook** in `apps/sim/hooks/queries/<domain>.ts`. Use `requestJson(contract, input)` for the call. Build a hierarchical query-key factory (`all` → `lists()` → `list(workspaceId)` → `details()` → `detail(id)`) so invalidations can target prefixes.
4. **Use the hook in the component**. The mutation's `data` and `error` are fully typed from the contract; surface `error.message` (already extracted from the response body's `error` or `message` field by `requestJson`).
2. **Define the semantic operation and application use case** under `apps/sim/lib/<domain>/application/`. The use case owns canonical loading, asserted-scope checks, current authorization, business behavior, semantic audit, and shared domain effects.
3. **Implement the route adapter** in `apps/sim/app/api/<path>/route.ts` with the appropriate shared builder. Declare auth, operation, rate policy, error policy, input mapping, use case, and presenter. Auth always runs **before** parsing. Use raw `withRouteHandler` only for an explicit special route, and keep protected work in application use cases.
4. **Add the React Query hook** in `apps/sim/hooks/queries/<domain>.ts`. Use `requestJson(contract, input)` for the call. Build a hierarchical query-key factory (`all` → `lists()` → `list(workspaceId)` → `details()` → `detail(id)`) so invalidations can target prefixes.
5. **Use the hook in the component**. The mutation's `data` and `error` are fully typed from the contract; surface `error.message` (already extracted from the response body's `error` or `message` field by `requestJson`).
### Schema review checklist (read the contract diff like a DB migration)
@@ -490,4 +494,3 @@ For the full authoring instructions — SubBlock property tables, `condition`/`d
Table column types are registry entries in `apps/sim/lib/table/column-types/` — one file per type owning its label, icon, storage cast, coercion, validation, conversion compatibility, formatting, and editor. `Record<ColumnType, …>` on `registry.ts` and `registry.server.ts` is a compile-time completeness gate: adding a type to the union errors until both entries exist.
Never add a `case 'sometype':` outside `column-types/` — a missing arm fails silently (a wrong `jsonbCast` breaks every filter on the column). If a consumer needs per-type knowledge, add a registry field. Use `/add-column-type` for the full procedure.
+6 -1
View File
@@ -11,6 +11,7 @@ import { PageFooter } from '@/components/docs-layout/page-footer'
import { PageNavigationArrows } from '@/components/docs-layout/page-navigation-arrows'
import { LLMCopyButton } from '@/components/page-actions'
import { StructuredData } from '@/components/structured-data'
import { APIExampleSelector } from '@/components/ui/api-example-selector'
import { CodeBlock } from '@/components/ui/code-block'
import { Heading } from '@/components/ui/heading'
import { ResponseSection } from '@/components/ui/response-section'
@@ -70,12 +71,16 @@ function stripLocalePrefix(url: string, lang: string): string {
const APIPage = createAPIPage(openapi, {
playground: { enabled: false },
client: {
operation: { APIExampleSelector },
},
content: {
renderOperationLayout: async (slots) => {
renderOperationLayout: (slots) => {
return (
<div className='flex @4xl:flex-row flex-col @4xl:items-start gap-x-6 gap-y-4'>
<div className='min-w-0 flex-1'>
{slots.header}
{slots.description}
{slots.apiPlayground}
{slots.authSchemes && <div className='api-section-divider'>{slots.authSchemes}</div>}
{slots.parameters}
+40 -29
View File
@@ -3,6 +3,8 @@ import { defineI18nUI } from 'fumadocs-ui/i18n'
import { DocsLayout } from 'fumadocs-ui/layouts/docs'
import { RootProvider } from 'fumadocs-ui/provider/next'
import { Geist_Mono, Inter } from 'next/font/google'
import Script from 'next/script'
import { ThemeProvider } from 'next-themes'
import {
SidebarFolder,
SidebarItem,
@@ -90,40 +92,49 @@ export default async function Layout({ children, params }: LayoutProps) {
suppressHydrationWarning
>
<head>
<script
<Script
id='website-json-ld'
type='application/ld+json'
strategy='beforeInteractive'
dangerouslySetInnerHTML={{ __html: serializeJsonLd(structuredData) }}
/>
</head>
<body className='flex min-h-screen flex-col font-sans'>
<RootProvider i18n={provider(lang)}>
<Navbar />
<DocsLayout
tree={source.pageTree[lang]}
nav={{
title: <SimWordmark className='h-[18px]' />,
}}
sidebar={{
tabs: false,
defaultOpenLevel: 0,
collapsible: false,
footer: null,
banner: null,
prefetch: false,
components: {
Item: SidebarItem,
Folder: SidebarFolder,
Separator: SidebarSeparator,
},
}}
containerProps={{
className: '!pt-0',
}}
>
{children}
</DocsLayout>
<Footer />
</RootProvider>
<ThemeProvider
attribute='class'
defaultTheme='system'
enableSystem
disableTransitionOnChange
>
<RootProvider i18n={provider(lang)} theme={{ enabled: false }}>
<Navbar />
<DocsLayout
tree={source.pageTree[lang]}
nav={{
title: <SimWordmark className='h-[18px]' />,
}}
sidebar={{
tabs: false,
defaultOpenLevel: 0,
collapsible: false,
footer: null,
banner: null,
prefetch: false,
components: {
Item: SidebarItem,
Folder: SidebarFolder,
Separator: SidebarSeparator,
},
}}
containerProps={{
className: '!pt-0',
}}
>
{children}
</DocsLayout>
<Footer />
</RootProvider>
</ThemeProvider>
</body>
</html>
)
+8 -3
View File
@@ -1,3 +1,4 @@
import Script from 'next/script'
import { serializeJsonLd } from '@/lib/json-ld'
import { DOCS_BASE_URL } from '@/lib/urls'
@@ -19,6 +20,7 @@ export function StructuredData({
breadcrumb,
}: StructuredDataProps) {
const baseUrl = DOCS_BASE_URL
const structuredDataId = encodeURIComponent(url)
const articleStructuredData = {
'@context': 'https://schema.org',
@@ -101,14 +103,16 @@ export function StructuredData({
return (
<>
<script
<Script
id={`article-json-ld-${structuredDataId}`}
type='application/ld+json'
dangerouslySetInnerHTML={{
__html: serializeJsonLd(articleStructuredData),
}}
/>
{breadcrumbStructuredData && (
<script
<Script
id={`breadcrumb-json-ld-${structuredDataId}`}
type='application/ld+json'
dangerouslySetInnerHTML={{
__html: serializeJsonLd(breadcrumbStructuredData),
@@ -116,7 +120,8 @@ export function StructuredData({
/>
)}
{(url === baseUrl || url === `${baseUrl}/`) && (
<script
<Script
id={`software-json-ld-${structuredDataId}`}
type='application/ld+json'
dangerouslySetInnerHTML={{
__html: serializeJsonLd(softwareStructuredData),
@@ -0,0 +1,47 @@
'use client'
import type { ComponentProps } from 'react'
import { useId } from 'react'
import { ChevronDown } from '@sim/emcn/icons'
import type { APIPageClientOptions } from 'fumadocs-openapi/ui/client'
type FumadocsAPIExampleSelector = NonNullable<
NonNullable<APIPageClientOptions['operation']>['APIExampleSelector']
>
interface APIExampleSelectorProps extends ComponentProps<FumadocsAPIExampleSelector> {}
export function APIExampleSelector({ items, value, onValueChange }: APIExampleSelectorProps) {
const id = useId()
if (items.length <= 1) return null
const selectedValue = value ?? items[0].id
const selectedItem = items.find((item) => item.id === selectedValue)
return (
<div className='not-prose mb-2 flex flex-col gap-1.5'>
<label htmlFor={id} className='sr-only'>
Request example
</label>
<div className='relative'>
<select
id={id}
value={selectedValue}
onChange={(event) => onValueChange(event.target.value)}
className='w-full appearance-none rounded-md border bg-fd-secondary py-2 ps-3 pe-9 text-start font-medium text-fd-secondary-foreground text-sm outline-none transition-colors hover:bg-fd-accent focus-visible:ring-2 focus-visible:ring-fd-ring'
>
{items.map((item) => (
<option key={item.id} value={item.id}>
{item.name}
</option>
))}
</select>
<ChevronDown className='-translate-y-1/2 pointer-events-none absolute end-3 top-1/2 size-[14px] text-fd-muted-foreground' />
</div>
{selectedItem?.description && (
<p className='text-fd-muted-foreground text-xs'>{selectedItem.description}</p>
)}
</div>
)
}
+5 -2
View File
@@ -1,7 +1,8 @@
'use client'
import { useState } from 'react'
import { useId, useState } from 'react'
import { ChevronRight } from '@sim/emcn/icons'
import Script from 'next/script'
import { serializeJsonLd } from '@/lib/json-ld'
import { cn } from '@/lib/utils'
@@ -58,6 +59,7 @@ function FAQItemRow({
}
export function FAQ({ items, title = 'Common Questions' }: FAQProps) {
const structuredDataId = useId()
/**
* Rows open independently rather than as a single-open accordion. Auto-closing
* a sibling collapses content *above* the row being opened, which yanks that
@@ -89,7 +91,8 @@ export function FAQ({ items, title = 'Common Questions' }: FAQProps) {
return (
<div className='mt-12'>
<script
<Script
id={`faq-json-ld-${structuredDataId}`}
type='application/ld+json'
dangerouslySetInnerHTML={{ __html: serializeJsonLd(faqSchema) }}
/>
@@ -41,23 +41,23 @@ Pass your API key in the `X-API-Key` header with every request:
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
'https://www.sim.ai/api/workflows/{workflowId}/execute',
'https://www.sim.ai/api/v2/workflows/{workflowId}/execute',
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
```
@@ -67,12 +67,12 @@ Pass your API key in the `X-API-Key` header with every request:
import requests
response = requests.post(
"https://www.sim.ai/api/workflows/{workflowId}/execute",
"https://www.sim.ai/api/v2/workflows/{workflowId}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
```
</Tab>
@@ -49,28 +49,28 @@ A workflow must be deployed before it can be executed via the API. Click the **D
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
`https://www.sim.ai/api/workflows/${workflowId}/execute`,
`https://www.sim.ai/api/v2/workflows/${workflowId}/execute`,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
const data = await response.json()
console.log(data.output)
console.log(data.data.output)
```
</Tab>
<Tab value="Python">
@@ -79,16 +79,16 @@ A workflow must be deployed before it can be executed via the API. Click the **D
import os
response = requests.post(
f"https://www.sim.ai/api/workflows/{workflow_id}/execute",
f"https://www.sim.ai/api/v2/workflows/{workflow_id}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
data = response.json()
print(data["output"])
print(data["data"]["output"])
```
</Tab>
</Tabs>
@@ -103,77 +103,63 @@ By default, workflow executions are **synchronous** — the API blocks until the
For long-running workflows, use **asynchronous execution** by passing `async: true`:
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}, "async": true}'
-d '{"input": {}, "async": true}'
```
This returns immediately with a `jobId` and `statusUrl`:
Keyed callers can optionally provide `X-Run-Id: my-run-123` to choose the run ID. Run IDs cannot be reused; a duplicate returns `409`.
This returns immediately with a `runId` and `statusUrl`:
```json
{
"success": true,
"jobId": "job_abc123",
"statusUrl": "https://www.sim.ai/api/jobs/job_abc123",
"message": "Workflow execution started",
"async": true
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"statusUrl": "https://www.sim.ai/api/v2/workflows/{workflowId}/runs/c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74"
}
}
```
Poll the [Get Job Status](/api-reference/workflows/getJobStatus) endpoint until the status is `completed` or `failed`:
Poll the run status endpoint until the status is terminal:
```bash
curl https://www.sim.ai/api/jobs/{jobId} \
curl https://www.sim.ai/api/v2/workflows/{workflowId}/runs/{runId}?includeOutput=true \
-H "X-API-Key: YOUR_API_KEY"
```
<Callout type="info">
Job status transitions follow: `queued` → `processing` → `completed` or `failed`. The `output` field is only present when status is `completed`.
Execution status transitions follow: `queued` → `running` → `completed`, `failed`, `cancelled`, or `paused`. The `data.output` field is populated for completed executions when `includeOutput=true`.
</Callout>
## Response Format
Successful responses include an `output` object with your workflow results and a `limits` object with your current rate limit and usage status:
Successful v2 responses wrap the run resource in `data`:
```json
{
"success": true,
"output": {
"result": "Hello, world!"
},
"limits": {
"workflowExecutionRateLimit": {
"sync": {
"requestsPerMinute": 60,
"maxBurst": 10,
"remaining": 59,
"resetAt": "2025-01-01T00:01:00Z"
},
"async": {
"requestsPerMinute": 30,
"maxBurst": 5,
"remaining": 30,
"resetAt": "2025-01-01T00:01:00Z"
}
},
"usage": {
"currentPeriodCost": 1.25,
"limit": 50.00,
"plan": "pro",
"isExceeded": false
}
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"workflowId": "{workflowId}",
"status": "completed",
"output": { "result": "Hello, world!" },
"error": null,
"durationMs": 842
}
}
```
## Error Handling
The API uses standard HTTP status codes. Error responses include a human-readable `error` message:
The API uses standard HTTP status codes. v2 errors include a stable code and human-readable message:
```json
{
"error": "Workflow not found"
"error": {
"code": "NOT_FOUND",
"message": "Workflow not found"
}
}
```
@@ -186,12 +172,12 @@ The API uses standard HTTP status codes. Error responses include a human-readabl
| `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header |
<Callout type="info">
Use the [Get Usage Limits](/api-reference/usage/getUsageLimits) endpoint to check your current rate limit status and billing usage at any time.
Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage.
</Callout>
## Rate Limits
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions. Every execution response includes a `limits` object showing your current rate limit status.
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions.
When rate limited, the API returns a `429` response with a `Retry-After` header indicating how many seconds to wait before retrying.
@@ -201,11 +187,11 @@ List endpoints (workflows, logs, audit logs) use **cursor-based pagination**:
```bash
# First page
curl "https://www.sim.ai/api/v1/logs?limit=20" \
curl "https://www.sim.ai/api/v2/logs?limit=20" \
-H "X-API-Key: YOUR_API_KEY"
# Next page — use the nextCursor from the previous response
curl "https://www.sim.ai/api/v1/logs?limit=20&cursor=abc123" \
curl "https://www.sim.ai/api/v2/logs?limit=20&cursor=abc123" \
-H "X-API-Key: YOUR_API_KEY"
```
@@ -2,6 +2,7 @@
"title": "API Reference",
"root": true,
"pages": [
"---Getting Started---",
"getting-started",
"authentication",
"---SDKs---",
@@ -9,10 +10,18 @@
"typescript",
"---Endpoints---",
"(generated)/workflows",
"(generated)/workflow-runs",
"(generated)/logs",
"(generated)/usage",
"(generated)/audit-logs",
"(generated)/tables",
"(generated)/files"
"(generated)/files",
"(generated)/knowledge-bases",
"(generated)/workspaces",
"(generated)/mcp-servers",
"(generated)/skills",
"(generated)/custom-tools",
"(generated)/credentials",
"(generated)/secrets",
"(generated)/billing",
"(generated)/audit-logs"
]
}
@@ -656,7 +656,7 @@ def stream_workflow():
def generate():
response = requests.post(
'https://sim.ai/api/workflows/WORKFLOW_ID/execute',
'https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute',
headers={
'Content-Type': 'application/json',
'X-API-Key': os.getenv('SIM_API_KEY')
@@ -763,4 +763,4 @@ Konfigurieren Sie den Client mit Umgebungsvariablen:
## Lizenz
Apache-2.0
Apache-2.0
@@ -965,7 +965,7 @@ function StreamingWorkflow() {
// IMPORTANT: Make this API call from your backend server, not the browser
// Never expose your API key in client-side code
const response = await fetch('https://sim.ai/api/workflows/WORKFLOW_ID/execute', {
const response = await fetch('https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
@@ -1049,4 +1049,4 @@ function StreamingWorkflow() {
## Lizenz
Apache-2.0
Apache-2.0
@@ -1,9 +0,0 @@
{
"pages": [
"listPausedExecutions",
"getPausedExecution",
"getPausedExecutionByResumePath",
"getPauseContext",
"resumeExecution"
]
}
@@ -0,0 +1,3 @@
{
"pages": ["listWorkflowRunsV2", "getWorkflowRunV2", "resumeWorkflowRunV2", "cancelRunV2"]
}
@@ -1,15 +1,21 @@
{
"pages": [
"executeWorkflow",
"getWorkflowExecution",
"cancelExecution",
"listWorkflows",
"createWorkflowV2",
"getWorkflow",
"updateWorkflowV2",
"deleteWorkflowV2",
"listWorkflowVersionsV2",
"getWorkflowVersionV2",
"exportWorkflow",
"importWorkflow",
"deployWorkflow",
"undeployWorkflow",
"rollbackWorkflow",
"getJobStatus"
"executeWorkflowV2",
"listWorkflowsFolders",
"createWorkflowsFolder",
"relocateWorkflowsFolder",
"deleteWorkflowsFolder"
]
}
@@ -41,23 +41,23 @@ Pass your API key in the `X-API-Key` header with every request:
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
'https://www.sim.ai/api/workflows/{workflowId}/execute',
'https://www.sim.ai/api/v2/workflows/{workflowId}/execute',
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
```
@@ -67,12 +67,12 @@ Pass your API key in the `X-API-Key` header with every request:
import requests
response = requests.post(
"https://www.sim.ai/api/workflows/{workflowId}/execute",
"https://www.sim.ai/api/v2/workflows/{workflowId}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
```
</Tab>
@@ -49,28 +49,28 @@ A workflow must be deployed before it can be executed via the API. Click the **D
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
`https://www.sim.ai/api/workflows/${workflowId}/execute`,
`https://www.sim.ai/api/v2/workflows/${workflowId}/execute`,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
const data = await response.json()
console.log(data.output)
console.log(data.data.output)
```
</Tab>
<Tab value="Python">
@@ -79,16 +79,16 @@ A workflow must be deployed before it can be executed via the API. Click the **D
import os
response = requests.post(
f"https://www.sim.ai/api/workflows/{workflow_id}/execute",
f"https://www.sim.ai/api/v2/workflows/{workflow_id}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
data = response.json()
print(data["output"])
print(data["data"]["output"])
```
</Tab>
</Tabs>
@@ -103,77 +103,63 @@ By default, workflow executions are **synchronous** — the API blocks until the
For long-running workflows, use **asynchronous execution** by passing `async: true`:
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}, "async": true}'
-d '{"input": {}, "async": true}'
```
This returns immediately with a `jobId` and `statusUrl`:
Keyed callers can optionally provide `X-Run-Id: my-run-123` to choose the run ID. Run IDs cannot be reused; a duplicate returns `409`.
This returns immediately with a `runId` and `statusUrl`:
```json
{
"success": true,
"jobId": "job_abc123",
"statusUrl": "https://www.sim.ai/api/jobs/job_abc123",
"message": "Workflow execution started",
"async": true
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"statusUrl": "https://www.sim.ai/api/v2/workflows/{workflowId}/runs/c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74"
}
}
```
Poll the [Get Job Status](/api-reference/workflows/getJobStatus) endpoint until the status is `completed` or `failed`:
Poll the run status endpoint until the status is terminal:
```bash
curl https://www.sim.ai/api/jobs/{jobId} \
curl https://www.sim.ai/api/v2/workflows/{workflowId}/runs/{runId}?includeOutput=true \
-H "X-API-Key: YOUR_API_KEY"
```
<Callout type="info">
Job status transitions follow: `queued` → `processing` → `completed` or `failed`. The `output` field is only present when status is `completed`.
Execution status transitions follow: `queued` → `running` → `completed`, `failed`, `cancelled`, or `paused`. The `data.output` field is populated for completed executions when `includeOutput=true`.
</Callout>
## Response Format
Successful responses include an `output` object with your workflow results and a `limits` object with your current rate limit and usage status:
Successful v2 responses wrap the run resource in `data`:
```json
{
"success": true,
"output": {
"result": "Hello, world!"
},
"limits": {
"workflowExecutionRateLimit": {
"sync": {
"requestsPerMinute": 60,
"maxBurst": 10,
"remaining": 59,
"resetAt": "2025-01-01T00:01:00Z"
},
"async": {
"requestsPerMinute": 30,
"maxBurst": 5,
"remaining": 30,
"resetAt": "2025-01-01T00:01:00Z"
}
},
"usage": {
"currentPeriodCost": 1.25,
"limit": 50.00,
"plan": "pro",
"isExceeded": false
}
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"workflowId": "{workflowId}",
"status": "completed",
"output": { "result": "Hello, world!" },
"error": null,
"durationMs": 842
}
}
```
## Error Handling
The API uses standard HTTP status codes. Error responses include a human-readable `error` message:
The API uses standard HTTP status codes. v2 errors include a stable code and human-readable message:
```json
{
"error": "Workflow not found"
"error": {
"code": "NOT_FOUND",
"message": "Workflow not found"
}
}
```
@@ -186,12 +172,12 @@ The API uses standard HTTP status codes. Error responses include a human-readabl
| `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header |
<Callout type="info">
Use the [Get Usage Limits](/api-reference/usage/getUsageLimits) endpoint to check your current rate limit status and billing usage at any time.
Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage.
</Callout>
## Rate Limits
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions. Every execution response includes a `limits` object showing your current rate limit status.
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions.
When rate limited, the API returns a `429` response with a `Retry-After` header indicating how many seconds to wait before retrying.
@@ -201,11 +187,11 @@ List endpoints (workflows, logs, audit logs) use **cursor-based pagination**:
```bash
# First page
curl "https://www.sim.ai/api/v1/logs?limit=20" \
curl "https://www.sim.ai/api/v2/logs?limit=20" \
-H "X-API-Key: YOUR_API_KEY"
# Next page — use the nextCursor from the previous response
curl "https://www.sim.ai/api/v1/logs?limit=20&cursor=abc123" \
curl "https://www.sim.ai/api/v2/logs?limit=20&cursor=abc123" \
-H "X-API-Key: YOUR_API_KEY"
```
@@ -10,12 +10,18 @@
"typescript",
"---Endpoints---",
"(generated)/workflows",
"(generated)/human-in-the-loop",
"(generated)/workflow-runs",
"(generated)/logs",
"(generated)/usage",
"(generated)/audit-logs",
"(generated)/tables",
"(generated)/files",
"(generated)/knowledge-bases"
"(generated)/knowledge-bases",
"(generated)/workspaces",
"(generated)/mcp-servers",
"(generated)/skills",
"(generated)/custom-tools",
"(generated)/credentials",
"(generated)/secrets",
"(generated)/billing",
"(generated)/audit-logs"
]
}
@@ -81,7 +81,7 @@ result = client.execute_workflow(
**Returns:** `WorkflowExecutionResult | AsyncExecutionResult`
When `async_execution=True`, returns immediately with a `job_id` and `status_url` for polling. Otherwise, waits for completion.
When `async_execution=True`, returns immediately with a `run_id` and `status_url` for polling. Otherwise, waits for completion.
##### get_workflow_status()
@@ -113,30 +113,42 @@ if is_ready:
**Returns:** `bool`
##### get_job_status()
##### get_workflow_run()
Get the status of an async job execution.
Get the status and optional outputs of a workflow execution.
```python
status = client.get_job_status("job-id-from-async-execution")
print("Status:", status["status"]) # 'queued', 'processing', 'completed', 'failed'
status = client.get_workflow_run("workflow-id", "run-id", include_output=True)
print("Status:", status["status"]) # 'queued', 'running', 'completed', 'failed'
if status["status"] == "completed":
print("Output:", status["output"])
```
**Parameters:**
- `task_id` (str): The job ID returned from async execution
- `workflow_id` (str): The workflow ID
- `run_id` (str): The run ID returned from async execution
- `include_output` (bool, optional): Include the final output for completed executions
- `selected_outputs` (list[str], optional): Block output selectors to include
**Returns:** `Dict[str, Any]`
**Response fields:**
- `success` (bool): Whether the request was successful
- `taskId` (str): The job ID
- `status` (str): One of `'queued'`, `'processing'`, `'completed'`, `'failed'`, `'cancelled'`
- `metadata` (dict): Contains `startedAt`, `completedAt`, and `duration`
- `output` (any, optional): The workflow output (when completed)
- `error` (any, optional): Error details (when failed)
- `estimatedDuration` (int, optional): Estimated duration in milliseconds (when processing/queued)
- `runId` (str): The run ID
- `workflowId` (str): The workflow ID
- `status` (str): One of `'queued'`, `'pending'`, `'running'`, `'paused'`, `'completed'`, `'failed'`, `'cancelled'`
- `startedAt` / `endedAt` (str): Execution timestamps
- `durationMs` (int, optional): Duration in milliseconds
- `output` (any, optional): The workflow output when requested for a completed execution
- `blockOutputs` (dict, optional): Requested block outputs
- `error` (dict, optional): Structured failure details with `code`, `message`, and optional `details`
##### get_job_status()
Get the status of a job created through the legacy async execution endpoint. New integrations should use `get_workflow_run()` with the run ID instead.
```python
status = client.get_job_status("legacy-job-id")
```
##### execute_with_retry()
@@ -271,9 +283,8 @@ class WorkflowExecutionResult:
@dataclass
class AsyncExecutionResult:
success: bool
job_id: str
run_id: str
status_url: str
execution_id: Optional[str] = None
message: str = ""
async_execution: bool = True
```
@@ -495,22 +506,26 @@ def execute_async():
)
# Check if result is an async execution
if hasattr(result, 'job_id'):
print(f"Job ID: {result.job_id}")
if hasattr(result, 'async_execution') and result.async_execution:
print(f"Run ID: {result.run_id}")
print(f"Status endpoint: {result.status_url}")
# Poll for completion
status = client.get_job_status(result.job_id)
status = client.get_workflow_run(
"workflow-id", result.run_id, include_output=True
)
while status["status"] in ["queued", "processing"]:
while status["status"] in ["queued", "pending", "running"]:
print(f"Current status: {status['status']}")
time.sleep(2) # Wait 2 seconds
status = client.get_job_status(result.job_id)
status = client.get_workflow_run(
"workflow-id", result.run_id, include_output=True
)
if status["status"] == "completed":
print("Workflow completed!")
print(f"Output: {status['output']}")
print(f"Duration: {status['metadata']['duration']}")
print(f"Duration: {status['durationMs']}")
else:
print(f"Workflow failed: {status['error']}")
@@ -657,13 +672,13 @@ def stream_workflow():
def generate():
response = requests.post(
'https://sim.ai/api/workflows/WORKFLOW_ID/execute',
'https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute',
headers={
'Content-Type': 'application/json',
'X-API-Key': os.getenv('SIM_API_KEY')
},
json={
'message': 'Generate a story',
'input': {'message': 'Generate a story'},
'stream': True,
'selectedOutputs': ['agent1.content']
},
@@ -766,7 +781,7 @@ import { FAQ } from '@/components/ui/faq'
<FAQ items={[
{ question: "Do I need to deploy a workflow before I can execute it via the SDK?", answer: "Yes. Workflows must be deployed before they can be executed through the SDK. You can use the validate_workflow() method to check whether a workflow is deployed and ready. If it returns False, deploy the workflow from the Sim UI first and create or select an API key during deployment." },
{ question: "What is the difference between sync and async execution?", answer: "Sync execution (the default) blocks until the workflow completes and returns the full result. Async execution (async_execution=True) returns immediately with a job ID and status URL that you can poll using get_job_status(). Use async mode for long-running workflows to avoid request timeouts. Async job statuses include queued, processing, completed, failed, and cancelled." },
{ question: "What is the difference between sync and async execution?", answer: "Sync execution (the default) blocks until the workflow completes and returns the full result. Async execution (async_execution=True) returns immediately with a run ID and status URL that you can poll using get_workflow_run(). Use async mode for long-running workflows to avoid request timeouts. Run statuses include queued, pending, running, paused, completed, failed, and cancelled." },
{ question: "How does the SDK handle rate limiting?", answer: "The SDK provides built-in rate limiting support through the execute_with_retry() method. It uses exponential backoff (1s, 2s, 4s, 8s...) with 25% jitter to avoid thundering herd problems. If the API returns a retry-after header, that value is used instead. You can configure max_retries, initial_delay, max_delay, and backoff_multiplier. Use get_rate_limit_info() to check your current rate limit status." },
{ question: "Can I use the Python SDK as a context manager?", answer: "Yes. The SimStudioClient supports Python's context manager protocol. Use it with the 'with' statement to automatically close the underlying HTTP session when you are done, which is especially useful for scripts that create and discard client instances." },
{ question: "How do I handle different types of errors from the SDK?", answer: "The SDK raises SimStudioError with a code property for API-specific errors. Common error codes are UNAUTHORIZED (invalid API key), TIMEOUT (request timed out), RATE_LIMIT_EXCEEDED (too many requests), USAGE_LIMIT_EXCEEDED (billing limit reached), and EXECUTION_ERROR (workflow failed). Use the error code to implement targeted error handling and recovery logic." },
@@ -95,7 +95,7 @@ const result = await client.executeWorkflow('workflow-id', { message: 'Hello, wo
**Returns:** `Promise<WorkflowExecutionResult | AsyncExecutionResult>`
When `async: true`, returns immediately with a `jobId` and `statusUrl` for polling. Otherwise, waits for completion.
When `async: true`, returns immediately with a `runId` and `statusUrl` for polling. Otherwise, waits for completion.
##### getWorkflowStatus()
@@ -127,31 +127,45 @@ if (isReady) {
**Returns:** `Promise<boolean>`
##### getJobStatus()
##### getWorkflowRun()
Get the status of an async job execution.
Get the status and optional outputs of a workflow run.
```typescript
const status = await client.getJobStatus('job-id-from-async-execution');
console.log('Status:', status.status); // 'queued', 'processing', 'completed', 'failed'
const status = await client.getWorkflowRun('workflow-id', 'run-id', {
includeOutput: true
});
console.log('Status:', status.status); // 'queued', 'running', 'completed', 'failed'
if (status.status === 'completed') {
console.log('Output:', status.output);
}
```
**Parameters:**
- `jobId` (string): The job ID returned from async execution
- `workflowId` (string): The workflow ID
- `runId` (string): The run ID returned from async execution
- `options.includeOutput` (boolean, optional): Include the final output for completed executions
- `options.selectedOutputs` (string[], optional): Block output selectors to include
**Returns:** `Promise<JobStatus>`
**Returns:** `Promise<WorkflowRunStatus>`
**Response fields:**
- `success` (boolean): Whether the request was successful
- `taskId` (string): The job ID
- `status` (string): One of `'queued'`, `'processing'`, `'completed'`, `'failed'`, `'cancelled'`
- `metadata` (object): Contains `startedAt`, `completedAt`, and `duration`
- `output` (any, optional): The workflow output (when completed)
- `error` (any, optional): Error details (when failed)
- `estimatedDuration` (number, optional): Estimated duration in milliseconds (when processing/queued)
- `runId` (string): The run ID
- `workflowId` (string): The workflow ID
- `status` (string): One of `'queued'`, `'pending'`, `'running'`, `'paused'`, `'completed'`, `'failed'`, `'cancelled'`
- `startedAt` / `endedAt` (string): Execution timestamps
- `durationMs` (number, nullable): Duration in milliseconds
- `output` (any, nullable): The workflow output when requested for a completed execution
- `blockOutputs` (object, nullable): Requested block outputs
- `error` (object, nullable): Structured failure details with `code`, `message`, and optional `details`
##### getJobStatus()
Get the status of a job created through the legacy async execution endpoint. New integrations should use `getWorkflowRun()` with the run ID instead.
```typescript
const status = await client.getJobStatus('legacy-job-id');
```
##### executeWithRetry()
@@ -266,7 +280,7 @@ interface WorkflowExecutionResult {
logs?: any[];
metadata?: {
duration?: number;
executionId?: string;
runId?: string;
[key: string]: any;
};
traceSpans?: any[];
@@ -279,9 +293,8 @@ interface WorkflowExecutionResult {
```typescript
interface AsyncExecutionResult {
success: boolean;
jobId: string;
runId: string;
statusUrl: string;
executionId?: string;
message: string;
async: true;
}
@@ -767,23 +780,27 @@ async function executeAsync() {
});
// Check if result is an async execution
if ('jobId' in result) {
console.log('Job ID:', result.jobId);
if ('async' in result && result.async) {
console.log('Run ID:', result.runId);
console.log('Status endpoint:', result.statusUrl);
// Poll for completion
let status = await client.getJobStatus(result.jobId);
let status = await client.getWorkflowRun('workflow-id', result.runId, {
includeOutput: true
});
while (status.status === 'queued' || status.status === 'processing') {
while (status.status === 'queued' || status.status === 'pending' || status.status === 'running') {
console.log('Current status:', status.status);
await new Promise(resolve => setTimeout(resolve, 2000)); // Wait 2 seconds
status = await client.getJobStatus(result.jobId);
status = await client.getWorkflowRun('workflow-id', result.runId, {
includeOutput: true
});
}
if (status.status === 'completed') {
console.log('Workflow completed!');
console.log('Output:', status.output);
console.log('Duration:', status.metadata.duration);
console.log('Duration:', status.durationMs);
} else {
console.error('Workflow failed:', status.error);
}
@@ -932,14 +949,14 @@ function StreamingWorkflow() {
// IMPORTANT: Make this API call from your backend server, not the browser
// Never expose your API key in client-side code
const response = await fetch('https://sim.ai/api/workflows/WORKFLOW_ID/execute', {
const response = await fetch('https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY! // Server-side environment variable only
},
body: JSON.stringify({
message: 'Generate a story',
input: { message: 'Generate a story' },
stream: true,
selectedOutputs: ['agent1.content']
})
@@ -1022,7 +1039,7 @@ import { FAQ } from '@/components/ui/faq'
<FAQ items={[
{ question: "Do I need to deploy a workflow before I can execute it via the SDK?", answer: "Yes. Workflows must be deployed before they can be executed through the SDK. You can use the validateWorkflow() method to check whether a workflow is deployed and ready. If it returns false, deploy the workflow from the Sim UI first and create or select an API key during deployment." },
{ question: "What is the difference between sync and async execution?", answer: "Sync execution (the default) blocks until the workflow completes and returns the full result. Async execution returns immediately with a job ID and status URL that you can poll using getJobStatus(). Use async mode for long-running workflows to avoid request timeouts. Async job statuses include queued, processing, completed, failed, and cancelled." },
{ question: "What is the difference between sync and async execution?", answer: "Sync execution (the default) blocks until the workflow completes and returns the full result. Async execution returns immediately with a run ID and status URL that you can poll using getWorkflowRun(). Use async mode for long-running workflows to avoid request timeouts. Run statuses include queued, pending, running, paused, completed, failed, and cancelled." },
{ question: "How does streaming work with the SDK?", answer: "Enable streaming by setting stream: true and specifying selectedOutputs with block names and attributes in blockName.attribute format (e.g., ['agent1.content']). The response uses Server-Sent Events (SSE) format, sending incremental chunks as the workflow executes. Each chunk includes the blockId and the text content. A final done event includes the execution metadata." },
{ question: "How does the SDK handle rate limiting?", answer: "The SDK provides built-in rate limiting support through the executeWithRetry() method. It uses exponential backoff (1s, 2s, 4s, 8s...) with 25% jitter to avoid thundering herd problems. If the API returns a retry-after header, that value is used instead. You can configure maxRetries, initialDelay, maxDelay, and backoffMultiplier. Use getRateLimitInfo() to check your current rate limit status." },
{ question: "Is it safe to use the SDK in browser-side code?", answer: "You can use the SDK in the browser, but you should not expose your API key in client-side code. In production, use a backend proxy server to handle SDK calls, or use a public API key with limited permissions. The SDK works with both Node.js and browser environments, but sensitive keys should stay server-side." },
@@ -5,7 +5,7 @@ description: Manage Zoho Desk tickets, comments, threads, and contacts
import { BlockInfoCard } from "@/components/ui/block-info-card"
<BlockInfoCard
<BlockInfoCard
type="zoho_desk"
color="#FFFFFF"
/>
@@ -522,4 +522,3 @@ Trigger a workflow when a Zoho Desk event occurs (ticket, comment, thread, conta
| `orgId` | string | Zoho Desk organization ID |
| `payload` | json | The full resource that changed \(ticket, comment, thread, etc.\). Comment and thread events gain a derived plain-text `contentText` alongside the raw `content` + `contentType`; ticket events gain `descriptionText` alongside `description`. |
| `prevState` | json | Previous state of the resource \(update events only\) |
@@ -33,7 +33,7 @@ Audit logs are also accessible through the Sim API for integration with external
```http
GET /api/v1/audit-logs
Authorization: Bearer <api-key>
X-API-Key: <api-key>
```
**Query parameters:**
@@ -71,11 +71,18 @@ Authorization: Bearer <api-key>
"createdAt": "2026-04-20T21:16:00.000Z"
}
],
"nextCursor": "eyJpZCI6ImFiYzEyMyJ9"
"nextCursor": "eyJpZCI6ImFiYzEyMyJ9",
"limits": {
"workflowExecutionRateLimit": {
"sync": { "requestsPerMinute": 60, "maxBurst": 10, "remaining": 59, "resetAt": "2026-04-20T21:17:00.000Z" },
"async": { "requestsPerMinute": 30, "maxBurst": 5, "remaining": 30, "resetAt": "2026-04-20T21:17:00.000Z" }
},
"usage": { "currentPeriodCost": 1.25, "limit": 50, "plan": "enterprise", "isExceeded": false }
}
}
```
Paginate by passing the `nextCursor` value as the `cursor` parameter in the next request. When `nextCursor` is absent, you have reached the last page.
Paginate by passing the `nextCursor` value as the `cursor` parameter in the next request. When `nextCursor` is absent, you have reached the last page. Each entry also includes `actorName`; `metadata` is an arbitrary per-action JSON object. The `limits` object reports your current rate-limit and usage status.
The API accepts both personal and workspace-scoped API keys. Rate limits apply — the response includes `X-RateLimit-*` headers with your current limit and remaining quota.
@@ -78,7 +78,6 @@ cat > /tmp/cors.json <<'EOF'
"AllowedOrigins": ["https://sim.yourdomain.com"],
"AllowedMethods": ["GET", "PUT"],
"AllowedHeaders": ["*"],
"ExposeHeaders": ["ETag"],
"MaxAgeSeconds": 3600
}
]
@@ -91,10 +90,6 @@ for name in workspace-files knowledge-base execution-files chat-files \
done
```
<Callout type="warn">
`ExposeHeaders` **must** include `ETag`. Files larger than 50 MB use multipart uploads, and the browser reads each part's `ETag` to complete the upload — CORS hides the header otherwise and large uploads fail at the final step.
</Callout>
Set `AllowedOrigins` to your exact Sim origin (scheme + host, no trailing slash). Add every origin users reach Sim from, including an apex/`www` pair if both are live.
</Step>
@@ -237,20 +232,26 @@ AZURE_STORAGE_OG_IMAGES_CONTAINER_NAME=og-images
AZURE_STORAGE_WORKSPACE_LOGOS_CONTAINER_NAME=workspace-logos
```
Browser uploads also require an account-level Blob service CORS rule. This rule explicitly allows Sim's create-only `If-None-Match` precondition, Azure upload headers, and multipart `ETag` reads:
Direct browser uploads require a Blob service CORS rule on the storage account. Allow your exact
Sim origin, `GET` and `PUT`, the `Content-Type` header, and the `x-ms-*` prefix used by signed blob
and metadata headers. Small-file uploads also send `If-None-Match` so a signed URL cannot overwrite
an existing final object. Multipart uploads also read `ETag` from the browser response:
```bash
az storage cors add \
--services b \
--methods GET PUT OPTIONS \
--origins "https://your-sim-domain.com" \
--allowed-headers "Content-Type" "If-None-Match" "x-ms-*" \
--exposed-headers "ETag" \
--origins https://sim.yourdomain.com \
--allowed-headers content-type if-none-match 'x-ms-*' \
--exposed-headers ETag \
--max-age 3600 \
--account-name mystorageaccount
--account-name mystorageaccount \
--account-key '<account-key>'
```
The CORS rule applies to every blob container in the storage account, so it only needs to be added once per account.
If you authenticate with a connection string, replace the last two options with
`--connection-string "$AZURE_CONNECTION_STRING"`. CORS is configured once for the account's Blob
service and applies to all of its containers.
A full Helm example lives at `helm/sim/examples/values-azure.yaml`.
@@ -288,15 +289,17 @@ cat > /tmp/cors.json <<'EOF'
{
"origin": ["https://your-sim-domain.com"],
"method": ["GET", "PUT"],
"responseHeader": [
"Content-Type",
"ETag",
"x-goog-if-generation-match",
"responseHeader": [
"Content-Type",
"ETag",
"x-goog-if-generation-match",
"x-goog-meta-uploadid",
"x-goog-meta-originalname",
"x-goog-meta-uploadedat",
"x-goog-meta-purpose",
"x-goog-meta-userid",
"x-goog-meta-workspaceid",
"x-goog-meta-knowledgebaseid",
"x-goog-meta-folderid",
"x-goog-meta-workflowid",
"x-goog-meta-executionid",
@@ -462,6 +465,27 @@ The same browser-reachability and CORS requirements apply.
</Tab>
</Tabs>
## Configure incomplete multipart cleanup
Sim uploads directly to a create-only final object key and keeps upload-session state in PostgreSQL.
The cleanup cron claims expired sessions before deleting an uploaded object or aborting its provider
multipart state. Configure provider lifecycle cleanup as a second line of defense for multipart
state that outlives its database row:
- On AWS S3 and Google Cloud Storage, abort incomplete multipart uploads after two days on every
purpose-specific bucket.
- Azure automatically removes uncommitted blocks after seven days.
- For an S3-compatible provider, configure incomplete-multipart cleanup when its lifecycle
implementation supports it. Check the provider's documentation because support varies.
The provider window should exceed the 24-hour upload-session lifetime so an in-progress completion
can still recover. Do not add an object-expiration rule for final upload keys.
<Callout type="warning">
Object expiration and incomplete-multipart cleanup are different lifecycle operations. Configure
the incomplete-multipart operation; expiring objects does not remove abandoned multipart parts.
</Callout>
## Verify it works
After restarting with the new configuration:
@@ -49,4 +49,4 @@ Recommended for any production deployment. The requirement is **pgvector**.
DATABASE_URL="postgresql://user:pass@host:5432/simstudio?sslmode=require"
```
For the Helm chart, disable the bundled Postgres and use `externalDatabase` — see [Kubernetes](/platform/self-hosting/kubernetes#external-database).
For the Helm chart, disable the bundled Postgres and use `externalDatabase` — see [Kubernetes](/platform/self-hosting/kubernetes#external-database).
@@ -193,7 +193,7 @@ Both pods must have `REDIS_URL`. On Helm they share one Secret, so setting it un
The bucket's CORS policy does not allow your Sim origin. Uploads go directly from the browser to object storage via presigned `PUT`, so server-side configuration being correct is not enough.
If small uploads succeed but files over 50 MB fail at the last step, `ETag` is missing from the CORS exposed headers — multipart uploads read it from the browser. See [Object Storage](/platform/self-hosting/object-storage).
If small uploads succeed but files over 50 MB fail during completion, check the app logs for the provider's part-listing request. The server completes multipart uploads from provider-authoritative state; for S3, its identity needs `s3:ListMultipartUploadParts`. See [Object Storage](/platform/self-hosting/object-storage).
## Agent Output Arrives All at Once
@@ -18,7 +18,7 @@ Run this after a first install, after an upgrade, and after a restore. Each step
| 4 | Open the same workflow in a second browser window and edit | Cross-replica collaboration | With >1 replica this needs [Redis](/platform/self-hosting/redis) |
| 5 | Paste a model API key in settings and run a two-block workflow | Execution engine, credential encryption, outbound network | App logs; check `ENCRYPTION_KEY` is set and outbound egress is allowed |
| 6 | Upload a small file in Files | File storage end to end | With object storage configured: presigned URL + bucket CORS. On local disk: the upload proxies through the app |
| 7 | Upload a file larger than 50 MB | Multipart upload path (object storage only) | Confirm `ETag` is in the bucket's CORS exposed headers |
| 7 | Upload a file larger than 50 MB | Multipart upload path (object storage only) | Check app logs for provider part-listing or completion errors |
| 8 | Create a knowledge base and upload a PDF | Document parsing, embeddings, pgvector | Needs a hosted embedding provider — see below |
| 9 | Invite a teammate from workspace settings | Email delivery | App logs for the mailer; see [Email](/platform/self-hosting/email) |
| 10 | Connect an integration account | OAuth configuration | Redirect URI mismatch → see [Integrations & OAuth](/platform/self-hosting/integrations-oauth) |
@@ -72,7 +72,7 @@ All six should be present on Compose: `simstudio`, `realtime`, `db`, `redis`, `c
**Step 5 fails — execution errors.** Check outbound connectivity to the model provider, then the app logs. If the error is about decrypting a credential, `ENCRYPTION_KEY` differs from the one that encrypted it.
**Step 6 or 7 fails.** With object storage configured, a CORS error in the browser console means the bucket policy does not allow your Sim origin; step 7 failing while step 6 passes specifically means `ETag` is missing from the exposed headers. On local-disk storage there is no CORS involved — uploads proxy through the app, so look at the app logs and the proxy body-size limit instead.
**Step 6 or 7 fails.** With object storage configured, a CORS error in the browser console means the bucket policy does not allow your Sim origin or the signed upload headers. If step 7 fails only during completion, check the app logs and verify the server identity can list multipart parts (for S3, `s3:ListMultipartUploadParts`). On local-disk storage there is no CORS involved — uploads proxy through the app, so look at the app logs and the proxy body-size limit instead.
**Step 8 fails — knowledge base upload errors.** Knowledge bases need a hosted embedding provider — OpenAI, Azure OpenAI, or Gemini. There is no local embedding backend. If a key is set, check pgvector is installed on the database.
@@ -87,14 +87,15 @@ Access resume data in downstream blocks using `<blockId.fieldName>`.
<Tab>
### REST API
Programmatically resume workflows using the resume endpoint. The `contextId` is available from the block's `resumeEndpoint` output or from the `_resume` object in the paused execution response.
Programmatically resume workflows through the v2 run resource. The `contextId` is available from the block's `resumeEndpoint` output or from the `_resume` object in the paused run response.
```bash
POST /api/resume/{workflowId}/{executionId}/{contextId}
POST /api/v2/workflows/{workflowId}/runs/{runId}/resume
Content-Type: application/json
X-API-Key: your-api-key
{
"contextId": "<contextId>",
"input": {
"approved": true,
"comments": "Looks good to proceed"
@@ -108,11 +109,16 @@ Access resume data in downstream blocks using `<blockId.fieldName>`.
```json
{
"success": true,
"status": "completed",
"executionId": "<resumeExecutionId>",
"output": { ... },
"metadata": { "duration": 1234, "startTime": "...", "endTime": "..." }
"data": {
"runId": "<resumeRunId>",
"workflowId": "<workflowId>",
"status": "completed",
"output": { ... },
"error": null,
"startedAt": "...",
"endedAt": "...",
"durationMs": 1234
}
}
```
@@ -120,29 +126,35 @@ Access resume data in downstream blocks using `<blockId.fieldName>`.
- **Stream mode** (`stream: true` on the original execute call) — The resume response streams SSE events with `selectedOutputs` chunks, just like the initial execution.
- **Async mode** (`X-Execution-Mode: async` on the original execute call) — The resume dispatches execution to a background worker and returns immediately with `202`, including a `jobId` and `statusUrl` for polling:
- **Async mode** (`async: true` on the original v2 execute call) — The resume dispatches the run to a background worker and returns immediately with `202`, including the resume attempt's `runId` and v2 `statusUrl` for polling:
```json
{
"success": true,
"async": true,
"jobId": "<jobId>",
"executionId": "<resumeExecutionId>",
"message": "Resume execution queued",
"statusUrl": "/api/jobs/<jobId>"
"data": {
"runId": "<resumeRunId>",
"statusUrl": "/api/v2/workflows/<workflowId>/runs/<resumeRunId>"
}
}
```
#### Polling execution status
#### Polling run status
Poll the `statusUrl` from the async response to check when the resume completes:
```bash
GET /api/jobs/{jobId}
GET /api/v2/workflows/{workflowId}/runs/{resumeRunId}?includeOutput=true
X-API-Key: your-api-key
```
Returns job status and, when completed, the full workflow output.
Returns the run status and, when completed, the full workflow output.
The legacy endpoint remains available without behavior changes for existing integrations:
```bash
POST /api/resume/{workflowId}/{executionId}/{contextId}
```
Its async response continues to expose `jobId` and the legacy `/api/jobs/{jobId}` polling URL.
To check on a paused execution's pause points and resume links:
@@ -162,7 +174,7 @@ Access resume data in downstream blocks using `<blockId.fieldName>`.
## API Execute Behavior
When triggering a workflow via the execute API (`POST /api/workflows/{id}/execute`), HITL blocks cause the execution to pause and return the `_resume` data in the response:
When triggering a workflow through `POST /api/v2/workflows/{id}/execute`, HITL blocks cause the execution to pause and return the `_resume` data in the v2 response envelope. The legacy `POST /api/workflows/{id}/execute` endpoint remains available for existing integrations.
<Tabs items={['Sync (JSON)', 'Stream (SSE)', 'Async']}>
<Tab>
@@ -170,19 +182,23 @@ When triggering a workflow via the execute API (`POST /api/workflows/{id}/execut
```json
{
"success": true,
"executionId": "<executionId>",
"output": {
"data": {
"operation": "human",
"_resume": {
"apiUrl": "/api/resume/{workflowId}/{executionId}/{contextId}",
"uiUrl": "/resume/{workflowId}/{executionId}",
"contextId": "<contextId>",
"executionId": "<executionId>",
"workflowId": "<workflowId>"
"data": {
"runId": "<runId>",
"workflowId": "<workflowId>",
"status": "paused",
"output": {
"data": {
"operation": "human",
"_resume": {
"apiUrl": "/api/resume/{workflowId}/{executionId}/{contextId}",
"uiUrl": "/resume/{workflowId}/{executionId}",
"contextId": "<contextId>",
"executionId": "<executionId>",
"workflowId": "<workflowId>"
}
}
}
},
"error": null
}
}
```
@@ -26,7 +26,7 @@ Click **Deploy** to publish your workflow for the first time, or **Update** to p
Once deployed, your workflow is available at:
```
POST https://sim.ai/api/workflows/{workflow-id}/execute
POST https://sim.ai/api/v2/workflows/{workflow-id}/execute
```
<Callout type="info">
@@ -96,10 +96,10 @@ At the bottom of the tab, two buttons give you quick access to key settings:
By default, API endpoints require an API key passed in the `x-api-key` header. Generate keys in **Settings → Sim Keys** or via the **Generate API Key** button in the API tab.
```bash
curl -X POST https://sim.ai/api/workflows/{workflow-id}/execute \
curl -X POST https://sim.ai/api/v2/workflows/{workflow-id}/execute \
-H "Content-Type: application/json" \
-H "x-api-key: $SIM_API_KEY" \
-d '{ "input": "Hello" }'
-d '{ "input": { "message": "Hello" } }'
```
### API Info and Public Access
@@ -128,10 +128,10 @@ The default mode. Send a request and wait for the complete response:
<Tabs items={['cURL', 'Python', 'TypeScript']}>
<Tab value="cURL">
```bash
curl -X POST https://sim.ai/api/workflows/{workflow-id}/execute \
curl -X POST https://sim.ai/api/v2/workflows/{workflow-id}/execute \
-H "Content-Type: application/json" \
-H "x-api-key: $SIM_API_KEY" \
-d '{ "input": "Summarize this article" }'
-d '{ "input": { "message": "Summarize this article" } }'
```
</Tab>
<Tab value="Python">
@@ -139,25 +139,25 @@ curl -X POST https://sim.ai/api/workflows/{workflow-id}/execute \
import requests, os
response = requests.post(
"https://sim.ai/api/workflows/{workflow-id}/execute",
"https://sim.ai/api/v2/workflows/{workflow-id}/execute",
headers={
"Content-Type": "application/json",
"x-api-key": os.environ["SIM_API_KEY"]
},
json={"input": "Summarize this article"}
json={"input": {"message": "Summarize this article"}}
)
print(response.json())
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch('https://sim.ai/api/workflows/{workflow-id}/execute', {
const response = await fetch('https://sim.ai/api/v2/workflows/{workflow-id}/execute', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-api-key': process.env.SIM_API_KEY!
},
body: JSON.stringify({ input: 'Summarize this article' })
body: JSON.stringify({ input: { message: 'Summarize this article' } })
});
console.log(await response.json());
```
@@ -179,11 +179,11 @@ The `selectedOutputs` values in the request body follow the format `blockName.fi
<Tabs items={['cURL', 'Python', 'TypeScript']}>
<Tab value="cURL">
```bash
curl -X POST https://sim.ai/api/workflows/{workflow-id}/execute \
curl -X POST https://sim.ai/api/v2/workflows/{workflow-id}/execute \
-H "Content-Type: application/json" \
-H "x-api-key: $SIM_API_KEY" \
-d '{
"input": "Write a long essay",
"input": { "prompt": "Write a long essay" },
"stream": true,
"selectedOutputs": ["agent_1.content"]
}'
@@ -194,13 +194,13 @@ curl -X POST https://sim.ai/api/workflows/{workflow-id}/execute \
import requests, os
response = requests.post(
"https://sim.ai/api/workflows/{workflow-id}/execute",
"https://sim.ai/api/v2/workflows/{workflow-id}/execute",
headers={
"Content-Type": "application/json",
"x-api-key": os.environ["SIM_API_KEY"]
},
json={
"input": "Write a long essay",
"input": {"prompt": "Write a long essay"},
"stream": True,
"selectedOutputs": ["agent_1.content"]
},
@@ -213,14 +213,14 @@ for line in response.iter_lines():
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch('https://sim.ai/api/workflows/{workflow-id}/execute', {
const response = await fetch('https://sim.ai/api/v2/workflows/{workflow-id}/execute', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-api-key': process.env.SIM_API_KEY!
},
body: JSON.stringify({
input: 'Write a long essay',
input: { prompt: 'Write a long essay' },
stream: true,
selectedOutputs: ['agent_1.content']
})
@@ -242,12 +242,12 @@ while (true) {
By default a streaming run carries answer text only. To also receive the Agent block's reasoning and its tool-call lifecycle, set `includeThinking` / `includeToolCalls`:
```bash
curl -N -X POST https://sim.ai/api/workflows/{workflow-id}/execute \
curl -N -X POST https://sim.ai/api/v2/workflows/{workflow-id}/execute \
-H "Content-Type: application/json" \
-H "x-api-key: $SIM_API_KEY" \
-H "X-Sim-Stream-Protocol: agent-events-v1" \
-d '{
"input": "Research this topic",
"input": { "prompt": "Research this topic" },
"stream": true,
"selectedOutputs": ["agent_1.content"],
"includeThinking": true,
@@ -280,79 +280,81 @@ The `version` field is part of the external API contract. Treat the reference as
### Asynchronous
For long-running workflows, async mode returns a job ID immediately so you don't need to hold the connection open. Add the `X-Execution-Mode: async` header to your request. The API returns HTTP 202 with a job ID and status URL. Poll the status URL until the job completes.
For long-running workflows, async mode returns a run ID immediately so you don't need to hold the connection open. Set `"async": true` in the v2 request body. The API returns HTTP 202 with a run ID and v2 status URL. Poll that run resource until it completes.
To stop an individual async request sooner than the workspace policy, also send `X-Execution-Timeout-Seconds` with an integer from `1` to `604800` (seven days). The effective limit is the smaller of this request value and the account's configured workflow execution timeout. This header is rejected unless `X-Execution-Mode` is `async`. It controls server-side workflow runtime; your HTTP client's own timeout remains separate.
To stop an individual async request sooner than the workspace policy, set `executionTimeoutSeconds`
to an integer from `1` to `604800` (seven days) in the v2 request body. The effective limit is the
smaller of this request value and the account's configured workflow execution timeout. This field
is rejected unless `async` is `true`; your HTTP client's timeout remains separate.
<Tabs items={['Start Job', 'Check Status']}>
<Tab value="Start Job">
<Tabs items={['Start Execution', 'Check Status']}>
<Tab value="Start Execution">
```bash
curl -X POST https://sim.ai/api/workflows/{workflow-id}/execute \
curl -X POST https://sim.ai/api/v2/workflows/{workflow-id}/execute \
-H "Content-Type: application/json" \
-H "x-api-key: $SIM_API_KEY" \
-H "X-Execution-Mode: async" \
-H "X-Execution-Timeout-Seconds: 3600" \
-d '{ "input": "Process this large dataset" }'
-d '{ "input": { "task": "Process this large dataset" }, "async": true, "executionTimeoutSeconds": 3600 }'
```
**Response** (HTTP 202):
```json
{
"success": true,
"async": true,
"jobId": "run_abc123",
"executionId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"message": "Workflow execution queued",
"statusUrl": "https://sim.ai/api/jobs/run_abc123"
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"statusUrl": "https://sim.ai/api/v2/workflows/{workflow-id}/runs/c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74"
}
}
```
</Tab>
<Tab value="Check Status">
```bash
curl https://sim.ai/api/jobs/{jobId} \
curl "https://sim.ai/api/v2/workflows/{workflow-id}/runs/{runId}?includeOutput=true" \
-H "x-api-key: $SIM_API_KEY"
```
**While processing:**
```json
{
"success": true,
"taskId": "run_abc123",
"status": "processing",
"metadata": {
"createdAt": "2025-09-10T12:00:00.000Z",
"startedAt": "2025-09-10T12:00:01.000Z"
},
"estimatedDuration": 300000
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"workflowId": "{workflow-id}",
"status": "running",
"startedAt": "2025-09-10T12:00:01.000Z",
"endedAt": null,
"durationMs": null,
"output": null
}
}
```
**When completed:**
```json
{
"success": true,
"taskId": "run_abc123",
"status": "completed",
"metadata": {
"createdAt": "2025-09-10T12:00:00.000Z",
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"workflowId": "{workflow-id}",
"status": "completed",
"startedAt": "2025-09-10T12:00:01.000Z",
"completedAt": "2025-09-10T12:00:05.000Z",
"duration": 4000
},
"output": { "result": "..." }
"endedAt": "2025-09-10T12:00:05.000Z",
"durationMs": 4000,
"output": { "result": "..." }
}
}
```
</Tab>
</Tabs>
#### Job Status Values
#### Execution Status Values
| Status | Description |
|--------|-------------|
| `queued` | Job is waiting to be picked up |
| `processing` | Workflow is actively executing |
| `completed` | Finished successfully — `output` field contains the result |
| `queued` | Execution is waiting to be picked up |
| `pending` | The durable execution record exists but has not started |
| `running` | Workflow is actively executing |
| `paused` | Workflow is waiting for a resume condition or input |
| `completed` | Finished successfully — `output` is populated when requested |
| `failed` | Execution failed — `error` field contains the message |
| `cancelled` | Execution was cancelled |
Poll the `statusUrl` from the initial response until the status is `completed` or `failed`.
@@ -364,11 +366,11 @@ Poll the `statusUrl` from the initial response until the status is `completed` o
| **Pro / Max / Team** | 50 minutes | 90 minutes |
| **Enterprise** | 50 minutes | 90 minutes by default; configurable up to 7 days |
If a job exceeds its time limit it is automatically marked as `failed`. An async API request may use `X-Execution-Timeout-Seconds` to shorten its account policy, but never to extend it.
If an execution exceeds its time limit it is automatically marked as `failed`. An async API request may use `executionTimeoutSeconds` to shorten its account policy, but never to extend it.
#### Job Retention
#### Execution Retention
Completed and failed job results are retained for **24 hours**. After that, the status endpoint returns `404`. Retrieve and store results on your end if you need them longer.
Completed and failed runs are read from execution logs and follow the workspace's execution-log retention policy.
#### Capacity Limits
@@ -41,23 +41,23 @@ Pass your API key in the `X-API-Key` header with every request:
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
'https://www.sim.ai/api/workflows/{workflowId}/execute',
'https://www.sim.ai/api/v2/workflows/{workflowId}/execute',
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
```
@@ -67,12 +67,12 @@ Pass your API key in the `X-API-Key` header with every request:
import requests
response = requests.post(
"https://www.sim.ai/api/workflows/{workflowId}/execute",
"https://www.sim.ai/api/v2/workflows/{workflowId}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
```
</Tab>
@@ -49,28 +49,28 @@ A workflow must be deployed before it can be executed via the API. Click the **D
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
`https://www.sim.ai/api/workflows/${workflowId}/execute`,
`https://www.sim.ai/api/v2/workflows/${workflowId}/execute`,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
const data = await response.json()
console.log(data.output)
console.log(data.data.output)
```
</Tab>
<Tab value="Python">
@@ -79,16 +79,16 @@ A workflow must be deployed before it can be executed via the API. Click the **D
import os
response = requests.post(
f"https://www.sim.ai/api/workflows/{workflow_id}/execute",
f"https://www.sim.ai/api/v2/workflows/{workflow_id}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
data = response.json()
print(data["output"])
print(data["data"]["output"])
```
</Tab>
</Tabs>
@@ -103,77 +103,63 @@ By default, workflow executions are **synchronous** — the API blocks until the
For long-running workflows, use **asynchronous execution** by passing `async: true`:
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}, "async": true}'
-d '{"input": {}, "async": true}'
```
This returns immediately with a `jobId` and `statusUrl`:
Keyed callers can optionally provide `X-Run-Id: my-run-123` to choose the run ID. Run IDs cannot be reused; a duplicate returns `409`.
This returns immediately with a `runId` and `statusUrl`:
```json
{
"success": true,
"jobId": "job_abc123",
"statusUrl": "https://www.sim.ai/api/jobs/job_abc123",
"message": "Workflow execution started",
"async": true
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"statusUrl": "https://www.sim.ai/api/v2/workflows/{workflowId}/runs/c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74"
}
}
```
Poll the [Get Job Status](/api-reference/workflows/getJobStatus) endpoint until the status is `completed` or `failed`:
Poll the run status endpoint until the status is terminal:
```bash
curl https://www.sim.ai/api/jobs/{jobId} \
curl https://www.sim.ai/api/v2/workflows/{workflowId}/runs/{runId}?includeOutput=true \
-H "X-API-Key: YOUR_API_KEY"
```
<Callout type="info">
Job status transitions follow: `queued` → `processing` → `completed` or `failed`. The `output` field is only present when status is `completed`.
Execution status transitions follow: `queued` → `running` → `completed`, `failed`, `cancelled`, or `paused`. The `data.output` field is populated for completed executions when `includeOutput=true`.
</Callout>
## Response Format
Successful responses include an `output` object with your workflow results and a `limits` object with your current rate limit and usage status:
Successful v2 responses wrap the run resource in `data`:
```json
{
"success": true,
"output": {
"result": "Hello, world!"
},
"limits": {
"workflowExecutionRateLimit": {
"sync": {
"requestsPerMinute": 60,
"maxBurst": 10,
"remaining": 59,
"resetAt": "2025-01-01T00:01:00Z"
},
"async": {
"requestsPerMinute": 30,
"maxBurst": 5,
"remaining": 30,
"resetAt": "2025-01-01T00:01:00Z"
}
},
"usage": {
"currentPeriodCost": 1.25,
"limit": 50.00,
"plan": "pro",
"isExceeded": false
}
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"workflowId": "{workflowId}",
"status": "completed",
"output": { "result": "Hello, world!" },
"error": null,
"durationMs": 842
}
}
```
## Error Handling
The API uses standard HTTP status codes. Error responses include a human-readable `error` message:
The API uses standard HTTP status codes. v2 errors include a stable code and human-readable message:
```json
{
"error": "Workflow not found"
"error": {
"code": "NOT_FOUND",
"message": "Workflow not found"
}
}
```
@@ -186,12 +172,12 @@ The API uses standard HTTP status codes. Error responses include a human-readabl
| `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header |
<Callout type="info">
Use the [Get Usage Limits](/api-reference/usage/getUsageLimits) endpoint to check your current rate limit status and billing usage at any time.
Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage.
</Callout>
## Rate Limits
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions. Every execution response includes a `limits` object showing your current rate limit status.
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions.
When rate limited, the API returns a `429` response with a `Retry-After` header indicating how many seconds to wait before retrying.
@@ -201,11 +187,11 @@ List endpoints (workflows, logs, audit logs) use **cursor-based pagination**:
```bash
# First page
curl "https://www.sim.ai/api/v1/logs?limit=20" \
curl "https://www.sim.ai/api/v2/logs?limit=20" \
-H "X-API-Key: YOUR_API_KEY"
# Next page — use the nextCursor from the previous response
curl "https://www.sim.ai/api/v1/logs?limit=20&cursor=abc123" \
curl "https://www.sim.ai/api/v2/logs?limit=20&cursor=abc123" \
-H "X-API-Key: YOUR_API_KEY"
```
@@ -2,6 +2,7 @@
"title": "API Reference",
"root": true,
"pages": [
"---Getting Started---",
"getting-started",
"authentication",
"---SDKs---",
@@ -9,8 +10,18 @@
"typescript",
"---Endpoints---",
"(generated)/workflows",
"(generated)/workflow-runs",
"(generated)/logs",
"(generated)/usage",
"(generated)/tables",
"(generated)/files",
"(generated)/knowledge-bases",
"(generated)/workspaces",
"(generated)/mcp-servers",
"(generated)/skills",
"(generated)/custom-tools",
"(generated)/credentials",
"(generated)/secrets",
"(generated)/billing",
"(generated)/audit-logs"
]
}
@@ -656,7 +656,7 @@ def stream_workflow():
def generate():
response = requests.post(
'https://sim.ai/api/workflows/WORKFLOW_ID/execute',
'https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute',
headers={
'Content-Type': 'application/json',
'X-API-Key': os.getenv('SIM_API_KEY')
@@ -763,4 +763,4 @@ Configura el cliente usando variables de entorno:
## Licencia
Apache-2.0
Apache-2.0
@@ -965,7 +965,7 @@ function StreamingWorkflow() {
// IMPORTANT: Make this API call from your backend server, not the browser
// Never expose your API key in client-side code
const response = await fetch('https://sim.ai/api/workflows/WORKFLOW_ID/execute', {
const response = await fetch('https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
@@ -1049,4 +1049,4 @@ function StreamingWorkflow() {
## Licencia
Apache-2.0
Apache-2.0
@@ -41,23 +41,23 @@ Pass your API key in the `X-API-Key` header with every request:
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
'https://www.sim.ai/api/workflows/{workflowId}/execute',
'https://www.sim.ai/api/v2/workflows/{workflowId}/execute',
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
```
@@ -67,12 +67,12 @@ Pass your API key in the `X-API-Key` header with every request:
import requests
response = requests.post(
"https://www.sim.ai/api/workflows/{workflowId}/execute",
"https://www.sim.ai/api/v2/workflows/{workflowId}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
```
</Tab>
@@ -49,28 +49,28 @@ A workflow must be deployed before it can be executed via the API. Click the **D
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
`https://www.sim.ai/api/workflows/${workflowId}/execute`,
`https://www.sim.ai/api/v2/workflows/${workflowId}/execute`,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
const data = await response.json()
console.log(data.output)
console.log(data.data.output)
```
</Tab>
<Tab value="Python">
@@ -79,16 +79,16 @@ A workflow must be deployed before it can be executed via the API. Click the **D
import os
response = requests.post(
f"https://www.sim.ai/api/workflows/{workflow_id}/execute",
f"https://www.sim.ai/api/v2/workflows/{workflow_id}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
data = response.json()
print(data["output"])
print(data["data"]["output"])
```
</Tab>
</Tabs>
@@ -103,77 +103,63 @@ By default, workflow executions are **synchronous** — the API blocks until the
For long-running workflows, use **asynchronous execution** by passing `async: true`:
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}, "async": true}'
-d '{"input": {}, "async": true}'
```
This returns immediately with a `jobId` and `statusUrl`:
Keyed callers can optionally provide `X-Run-Id: my-run-123` to choose the run ID. Run IDs cannot be reused; a duplicate returns `409`.
This returns immediately with a `runId` and `statusUrl`:
```json
{
"success": true,
"jobId": "job_abc123",
"statusUrl": "https://www.sim.ai/api/jobs/job_abc123",
"message": "Workflow execution started",
"async": true
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"statusUrl": "https://www.sim.ai/api/v2/workflows/{workflowId}/runs/c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74"
}
}
```
Poll the [Get Job Status](/api-reference/workflows/getJobStatus) endpoint until the status is `completed` or `failed`:
Poll the run status endpoint until the status is terminal:
```bash
curl https://www.sim.ai/api/jobs/{jobId} \
curl https://www.sim.ai/api/v2/workflows/{workflowId}/runs/{runId}?includeOutput=true \
-H "X-API-Key: YOUR_API_KEY"
```
<Callout type="info">
Job status transitions follow: `queued` → `processing` → `completed` or `failed`. The `output` field is only present when status is `completed`.
Execution status transitions follow: `queued` → `running` → `completed`, `failed`, `cancelled`, or `paused`. The `data.output` field is populated for completed executions when `includeOutput=true`.
</Callout>
## Response Format
Successful responses include an `output` object with your workflow results and a `limits` object with your current rate limit and usage status:
Successful v2 responses wrap the run resource in `data`:
```json
{
"success": true,
"output": {
"result": "Hello, world!"
},
"limits": {
"workflowExecutionRateLimit": {
"sync": {
"requestsPerMinute": 60,
"maxBurst": 10,
"remaining": 59,
"resetAt": "2025-01-01T00:01:00Z"
},
"async": {
"requestsPerMinute": 30,
"maxBurst": 5,
"remaining": 30,
"resetAt": "2025-01-01T00:01:00Z"
}
},
"usage": {
"currentPeriodCost": 1.25,
"limit": 50.00,
"plan": "pro",
"isExceeded": false
}
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"workflowId": "{workflowId}",
"status": "completed",
"output": { "result": "Hello, world!" },
"error": null,
"durationMs": 842
}
}
```
## Error Handling
The API uses standard HTTP status codes. Error responses include a human-readable `error` message:
The API uses standard HTTP status codes. v2 errors include a stable code and human-readable message:
```json
{
"error": "Workflow not found"
"error": {
"code": "NOT_FOUND",
"message": "Workflow not found"
}
}
```
@@ -186,12 +172,12 @@ The API uses standard HTTP status codes. Error responses include a human-readabl
| `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header |
<Callout type="info">
Use the [Get Usage Limits](/api-reference/usage/getUsageLimits) endpoint to check your current rate limit status and billing usage at any time.
Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage.
</Callout>
## Rate Limits
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions. Every execution response includes a `limits` object showing your current rate limit status.
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions.
When rate limited, the API returns a `429` response with a `Retry-After` header indicating how many seconds to wait before retrying.
@@ -201,11 +187,11 @@ List endpoints (workflows, logs, audit logs) use **cursor-based pagination**:
```bash
# First page
curl "https://www.sim.ai/api/v1/logs?limit=20" \
curl "https://www.sim.ai/api/v2/logs?limit=20" \
-H "X-API-Key: YOUR_API_KEY"
# Next page — use the nextCursor from the previous response
curl "https://www.sim.ai/api/v1/logs?limit=20&cursor=abc123" \
curl "https://www.sim.ai/api/v2/logs?limit=20&cursor=abc123" \
-H "X-API-Key: YOUR_API_KEY"
```
@@ -2,6 +2,7 @@
"title": "API Reference",
"root": true,
"pages": [
"---Getting Started---",
"getting-started",
"authentication",
"---SDKs---",
@@ -9,8 +10,18 @@
"typescript",
"---Endpoints---",
"(generated)/workflows",
"(generated)/workflow-runs",
"(generated)/logs",
"(generated)/usage",
"(generated)/tables",
"(generated)/files",
"(generated)/knowledge-bases",
"(generated)/workspaces",
"(generated)/mcp-servers",
"(generated)/skills",
"(generated)/custom-tools",
"(generated)/credentials",
"(generated)/secrets",
"(generated)/billing",
"(generated)/audit-logs"
]
}
@@ -656,7 +656,7 @@ def stream_workflow():
def generate():
response = requests.post(
'https://sim.ai/api/workflows/WORKFLOW_ID/execute',
'https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute',
headers={
'Content-Type': 'application/json',
'X-API-Key': os.getenv('SIM_API_KEY')
@@ -763,4 +763,4 @@ Configurez le client en utilisant des variables d'environnement :
## Licence
Apache-2.0
Apache-2.0
@@ -965,7 +965,7 @@ function StreamingWorkflow() {
// IMPORTANT: Make this API call from your backend server, not the browser
// Never expose your API key in client-side code
const response = await fetch('https://sim.ai/api/workflows/WORKFLOW_ID/execute', {
const response = await fetch('https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
@@ -41,23 +41,23 @@ Pass your API key in the `X-API-Key` header with every request:
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
'https://www.sim.ai/api/workflows/{workflowId}/execute',
'https://www.sim.ai/api/v2/workflows/{workflowId}/execute',
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
```
@@ -67,12 +67,12 @@ Pass your API key in the `X-API-Key` header with every request:
import requests
response = requests.post(
"https://www.sim.ai/api/workflows/{workflowId}/execute",
"https://www.sim.ai/api/v2/workflows/{workflowId}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
```
</Tab>
@@ -49,28 +49,28 @@ A workflow must be deployed before it can be executed via the API. Click the **D
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
`https://www.sim.ai/api/workflows/${workflowId}/execute`,
`https://www.sim.ai/api/v2/workflows/${workflowId}/execute`,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
const data = await response.json()
console.log(data.output)
console.log(data.data.output)
```
</Tab>
<Tab value="Python">
@@ -79,16 +79,16 @@ A workflow must be deployed before it can be executed via the API. Click the **D
import os
response = requests.post(
f"https://www.sim.ai/api/workflows/{workflow_id}/execute",
f"https://www.sim.ai/api/v2/workflows/{workflow_id}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
data = response.json()
print(data["output"])
print(data["data"]["output"])
```
</Tab>
</Tabs>
@@ -103,77 +103,63 @@ By default, workflow executions are **synchronous** — the API blocks until the
For long-running workflows, use **asynchronous execution** by passing `async: true`:
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}, "async": true}'
-d '{"input": {}, "async": true}'
```
This returns immediately with a `jobId` and `statusUrl`:
Keyed callers can optionally provide `X-Run-Id: my-run-123` to choose the run ID. Run IDs cannot be reused; a duplicate returns `409`.
This returns immediately with a `runId` and `statusUrl`:
```json
{
"success": true,
"jobId": "job_abc123",
"statusUrl": "https://www.sim.ai/api/jobs/job_abc123",
"message": "Workflow execution started",
"async": true
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"statusUrl": "https://www.sim.ai/api/v2/workflows/{workflowId}/runs/c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74"
}
}
```
Poll the [Get Job Status](/api-reference/workflows/getJobStatus) endpoint until the status is `completed` or `failed`:
Poll the run status endpoint until the status is terminal:
```bash
curl https://www.sim.ai/api/jobs/{jobId} \
curl https://www.sim.ai/api/v2/workflows/{workflowId}/runs/{runId}?includeOutput=true \
-H "X-API-Key: YOUR_API_KEY"
```
<Callout type="info">
Job status transitions follow: `queued` → `processing` → `completed` or `failed`. The `output` field is only present when status is `completed`.
Execution status transitions follow: `queued` → `running` → `completed`, `failed`, `cancelled`, or `paused`. The `data.output` field is populated for completed executions when `includeOutput=true`.
</Callout>
## Response Format
Successful responses include an `output` object with your workflow results and a `limits` object with your current rate limit and usage status:
Successful v2 responses wrap the run resource in `data`:
```json
{
"success": true,
"output": {
"result": "Hello, world!"
},
"limits": {
"workflowExecutionRateLimit": {
"sync": {
"requestsPerMinute": 60,
"maxBurst": 10,
"remaining": 59,
"resetAt": "2025-01-01T00:01:00Z"
},
"async": {
"requestsPerMinute": 30,
"maxBurst": 5,
"remaining": 30,
"resetAt": "2025-01-01T00:01:00Z"
}
},
"usage": {
"currentPeriodCost": 1.25,
"limit": 50.00,
"plan": "pro",
"isExceeded": false
}
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"workflowId": "{workflowId}",
"status": "completed",
"output": { "result": "Hello, world!" },
"error": null,
"durationMs": 842
}
}
```
## Error Handling
The API uses standard HTTP status codes. Error responses include a human-readable `error` message:
The API uses standard HTTP status codes. v2 errors include a stable code and human-readable message:
```json
{
"error": "Workflow not found"
"error": {
"code": "NOT_FOUND",
"message": "Workflow not found"
}
}
```
@@ -186,12 +172,12 @@ The API uses standard HTTP status codes. Error responses include a human-readabl
| `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header |
<Callout type="info">
Use the [Get Usage Limits](/api-reference/usage/getUsageLimits) endpoint to check your current rate limit status and billing usage at any time.
Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage.
</Callout>
## Rate Limits
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions. Every execution response includes a `limits` object showing your current rate limit status.
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions.
When rate limited, the API returns a `429` response with a `Retry-After` header indicating how many seconds to wait before retrying.
@@ -201,11 +187,11 @@ List endpoints (workflows, logs, audit logs) use **cursor-based pagination**:
```bash
# First page
curl "https://www.sim.ai/api/v1/logs?limit=20" \
curl "https://www.sim.ai/api/v2/logs?limit=20" \
-H "X-API-Key: YOUR_API_KEY"
# Next page — use the nextCursor from the previous response
curl "https://www.sim.ai/api/v1/logs?limit=20&cursor=abc123" \
curl "https://www.sim.ai/api/v2/logs?limit=20&cursor=abc123" \
-H "X-API-Key: YOUR_API_KEY"
```
@@ -2,6 +2,7 @@
"title": "API Reference",
"root": true,
"pages": [
"---Getting Started---",
"getting-started",
"authentication",
"---SDKs---",
@@ -9,8 +10,18 @@
"typescript",
"---Endpoints---",
"(generated)/workflows",
"(generated)/workflow-runs",
"(generated)/logs",
"(generated)/usage",
"(generated)/tables",
"(generated)/files",
"(generated)/knowledge-bases",
"(generated)/workspaces",
"(generated)/mcp-servers",
"(generated)/skills",
"(generated)/custom-tools",
"(generated)/credentials",
"(generated)/secrets",
"(generated)/billing",
"(generated)/audit-logs"
]
}
@@ -656,7 +656,7 @@ def stream_workflow():
def generate():
response = requests.post(
'https://sim.ai/api/workflows/WORKFLOW_ID/execute',
'https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute',
headers={
'Content-Type': 'application/json',
'X-API-Key': os.getenv('SIM_API_KEY')
@@ -763,4 +763,4 @@ if __name__ == '__main__':
## ライセンス
Apache-2.0
Apache-2.0
@@ -965,7 +965,7 @@ function StreamingWorkflow() {
// IMPORTANT: Make this API call from your backend server, not the browser
// Never expose your API key in client-side code
const response = await fetch('https://sim.ai/api/workflows/WORKFLOW_ID/execute', {
const response = await fetch('https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
@@ -41,23 +41,23 @@ Pass your API key in the `X-API-Key` header with every request:
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
'https://www.sim.ai/api/workflows/{workflowId}/execute',
'https://www.sim.ai/api/v2/workflows/{workflowId}/execute',
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
```
@@ -67,12 +67,12 @@ Pass your API key in the `X-API-Key` header with every request:
import requests
response = requests.post(
"https://www.sim.ai/api/workflows/{workflowId}/execute",
"https://www.sim.ai/api/v2/workflows/{workflowId}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
```
</Tab>
@@ -49,28 +49,28 @@ A workflow must be deployed before it can be executed via the API. Click the **D
<Tabs items={['curl', 'TypeScript', 'Python']}>
<Tab value="curl">
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}}'
-d '{"input": {}}'
```
</Tab>
<Tab value="TypeScript">
```typescript
const response = await fetch(
`https://www.sim.ai/api/workflows/${workflowId}/execute`,
`https://www.sim.ai/api/v2/workflows/${workflowId}/execute`,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': process.env.SIM_API_KEY!,
},
body: JSON.stringify({ inputs: {} }),
body: JSON.stringify({ input: {} }),
}
)
const data = await response.json()
console.log(data.output)
console.log(data.data.output)
```
</Tab>
<Tab value="Python">
@@ -79,16 +79,16 @@ A workflow must be deployed before it can be executed via the API. Click the **D
import os
response = requests.post(
f"https://www.sim.ai/api/workflows/{workflow_id}/execute",
f"https://www.sim.ai/api/v2/workflows/{workflow_id}/execute",
headers={
"Content-Type": "application/json",
"X-API-Key": os.environ["SIM_API_KEY"],
},
json={"inputs": {}},
json={"input": {}},
)
data = response.json()
print(data["output"])
print(data["data"]["output"])
```
</Tab>
</Tabs>
@@ -103,77 +103,63 @@ By default, workflow executions are **synchronous** — the API blocks until the
For long-running workflows, use **asynchronous execution** by passing `async: true`:
```bash
curl -X POST https://www.sim.ai/api/workflows/{workflowId}/execute \
curl -X POST https://www.sim.ai/api/v2/workflows/{workflowId}/execute \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"inputs": {}, "async": true}'
-d '{"input": {}, "async": true}'
```
This returns immediately with a `jobId` and `statusUrl`:
Keyed callers can optionally provide `X-Run-Id: my-run-123` to choose the run ID. Run IDs cannot be reused; a duplicate returns `409`.
This returns immediately with a `runId` and `statusUrl`:
```json
{
"success": true,
"jobId": "job_abc123",
"statusUrl": "https://www.sim.ai/api/jobs/job_abc123",
"message": "Workflow execution started",
"async": true
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"statusUrl": "https://www.sim.ai/api/v2/workflows/{workflowId}/runs/c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74"
}
}
```
Poll the [Get Job Status](/api-reference/workflows/getJobStatus) endpoint until the status is `completed` or `failed`:
Poll the run status endpoint until the status is terminal:
```bash
curl https://www.sim.ai/api/jobs/{jobId} \
curl https://www.sim.ai/api/v2/workflows/{workflowId}/runs/{runId}?includeOutput=true \
-H "X-API-Key: YOUR_API_KEY"
```
<Callout type="info">
Job status transitions follow: `queued` → `processing` → `completed` or `failed`. The `output` field is only present when status is `completed`.
Execution status transitions follow: `queued` → `running` → `completed`, `failed`, `cancelled`, or `paused`. The `data.output` field is populated for completed executions when `includeOutput=true`.
</Callout>
## Response Format
Successful responses include an `output` object with your workflow results and a `limits` object with your current rate limit and usage status:
Successful v2 responses wrap the run resource in `data`:
```json
{
"success": true,
"output": {
"result": "Hello, world!"
},
"limits": {
"workflowExecutionRateLimit": {
"sync": {
"requestsPerMinute": 60,
"maxBurst": 10,
"remaining": 59,
"resetAt": "2025-01-01T00:01:00Z"
},
"async": {
"requestsPerMinute": 30,
"maxBurst": 5,
"remaining": 30,
"resetAt": "2025-01-01T00:01:00Z"
}
},
"usage": {
"currentPeriodCost": 1.25,
"limit": 50.00,
"plan": "pro",
"isExceeded": false
}
"data": {
"runId": "c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74",
"workflowId": "{workflowId}",
"status": "completed",
"output": { "result": "Hello, world!" },
"error": null,
"durationMs": 842
}
}
```
## Error Handling
The API uses standard HTTP status codes. Error responses include a human-readable `error` message:
The API uses standard HTTP status codes. v2 errors include a stable code and human-readable message:
```json
{
"error": "Workflow not found"
"error": {
"code": "NOT_FOUND",
"message": "Workflow not found"
}
}
```
@@ -186,12 +172,12 @@ The API uses standard HTTP status codes. Error responses include a human-readabl
| `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header |
<Callout type="info">
Use the [Get Usage Limits](/api-reference/usage/getUsageLimits) endpoint to check your current rate limit status and billing usage at any time.
Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage.
</Callout>
## Rate Limits
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions. Every execution response includes a `limits` object showing your current rate limit status.
Rate limits depend on your subscription plan and apply separately to synchronous and asynchronous executions.
When rate limited, the API returns a `429` response with a `Retry-After` header indicating how many seconds to wait before retrying.
@@ -201,11 +187,11 @@ List endpoints (workflows, logs, audit logs) use **cursor-based pagination**:
```bash
# First page
curl "https://www.sim.ai/api/v1/logs?limit=20" \
curl "https://www.sim.ai/api/v2/logs?limit=20" \
-H "X-API-Key: YOUR_API_KEY"
# Next page — use the nextCursor from the previous response
curl "https://www.sim.ai/api/v1/logs?limit=20&cursor=abc123" \
curl "https://www.sim.ai/api/v2/logs?limit=20&cursor=abc123" \
-H "X-API-Key: YOUR_API_KEY"
```
@@ -2,6 +2,7 @@
"title": "API Reference",
"root": true,
"pages": [
"---Getting Started---",
"getting-started",
"authentication",
"---SDKs---",
@@ -9,8 +10,18 @@
"typescript",
"---Endpoints---",
"(generated)/workflows",
"(generated)/workflow-runs",
"(generated)/logs",
"(generated)/usage",
"(generated)/tables",
"(generated)/files",
"(generated)/knowledge-bases",
"(generated)/workspaces",
"(generated)/mcp-servers",
"(generated)/skills",
"(generated)/custom-tools",
"(generated)/credentials",
"(generated)/secrets",
"(generated)/billing",
"(generated)/audit-logs"
]
}
@@ -657,7 +657,7 @@ def stream_workflow():
def generate():
response = requests.post(
'https://sim.ai/api/workflows/WORKFLOW_ID/execute',
'https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute',
headers={
'Content-Type': 'application/json',
'X-API-Key': os.getenv('SIM_API_KEY')
@@ -764,4 +764,4 @@ if __name__ == '__main__':
## 许可证
Apache-2.0
Apache-2.0
@@ -964,7 +964,7 @@ function StreamingWorkflow() {
// IMPORTANT: Make this API call from your backend server, not the browser
// Never expose your API key in client-side code
const response = await fetch('https://sim.ai/api/workflows/WORKFLOW_ID/execute', {
const response = await fetch('https://sim.ai/api/v2/workflows/WORKFLOW_ID/execute', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
+9
View File
@@ -0,0 +1,9 @@
export const OPENAPI_SPEC_FILES = [
'openapi-v2-workflows.json',
'openapi-v2-logs.json',
'openapi-v2-files-audit.json',
'openapi-v2-tables.json',
'openapi-v2-knowledge.json',
'openapi-v2-billing.json',
'openapi-v2-resources.json',
] as const
+52 -20
View File
@@ -1,9 +1,10 @@
import { readFileSync } from 'node:fs'
import { join } from 'node:path'
import { createOpenAPI } from 'fumadocs-openapi/server'
import { OPENAPI_SPEC_FILES } from '@/lib/openapi-specs'
export const openapi = createOpenAPI({
input: ['./openapi.json'],
input: OPENAPI_SPEC_FILES.map((file) => `./${file}`),
})
interface OpenAPIOperation {
@@ -24,20 +25,34 @@ function resolveRef(ref: string, spec: Record<string, unknown>): unknown {
return current
}
function resolveRefs(obj: unknown, spec: Record<string, unknown>, depth = 0): unknown {
if (depth > 10) return obj
function resolveRefs(
obj: unknown,
spec: Record<string, unknown>,
seen: Set<string> = new Set(),
depth = 0
): unknown {
// Generous backstop against pathological fan-out; real schemas nest far shallower.
if (depth > 50) return obj
if (Array.isArray(obj)) {
return obj.map((item) => resolveRefs(item, spec, depth + 1))
return obj.map((item) => resolveRefs(item, spec, seen, depth + 1))
}
if (obj && typeof obj === 'object') {
const record = obj as Record<string, unknown>
if ('$ref' in record && typeof record.$ref === 'string') {
const resolved = resolveRef(record.$ref, spec)
return resolveRefs(resolved, spec, depth + 1)
if (typeof record.$ref === 'string') {
const ref = record.$ref
// Break reference cycles: if this $ref is already being expanded above us,
// leave it untouched instead of recursing forever.
if (seen.has(ref)) return record
const resolved = resolveRef(ref, spec)
if (resolved === undefined) return record
seen.add(ref)
const out = resolveRefs(resolved, spec, seen, depth + 1)
seen.delete(ref)
return out
}
const result: Record<string, unknown> = {}
for (const [key, value] of Object.entries(record)) {
result[key] = resolveRefs(value, spec, depth + 1)
result[key] = resolveRefs(value, spec, seen, depth + 1)
}
return result
}
@@ -48,14 +63,34 @@ function formatSchema(schema: unknown): string {
return JSON.stringify(schema, null, 2)
}
let cachedSpec: Record<string, unknown> | null = null
let cachedSpecs: Record<string, unknown>[] | null = null
function getSpec(): Record<string, unknown> {
if (!cachedSpec) {
const specPath = join(process.cwd(), 'openapi.json')
cachedSpec = JSON.parse(readFileSync(specPath, 'utf8')) as Record<string, unknown>
function getSpecs(): Record<string, unknown>[] {
if (!cachedSpecs) {
cachedSpecs = OPENAPI_SPEC_FILES.map(
(file) =>
JSON.parse(readFileSync(join(process.cwd(), file), 'utf8')) as Record<string, unknown>
)
}
return cachedSpec
return cachedSpecs
}
/**
* Locate an operation by path + method across every rendered spec, returning the
* operation together with the spec that owns it so `$ref`s resolve within the
* correct document (each spec carries its own `components`).
*/
function findOperation(
path: string,
method: string
): { operation: Record<string, unknown>; spec: Record<string, unknown> } | undefined {
const key = method.toLowerCase()
for (const spec of getSpecs()) {
const pathObj = (spec.paths as Record<string, Record<string, unknown>> | undefined)?.[path]
const operation = pathObj?.[key] as Record<string, unknown> | undefined
if (operation) return { operation, spec }
}
return undefined
}
export function getApiSpecContent(
@@ -63,22 +98,19 @@ export function getApiSpecContent(
description: string | undefined,
operations: OpenAPIOperation[]
): string {
const spec = getSpec()
if (!operations || operations.length === 0) {
return `# ${title}\n\n${description || ''}`
}
const op = operations[0]
const method = op.method.toUpperCase()
const pathObj = (spec.paths as Record<string, Record<string, unknown>>)?.[op.path]
const operation = pathObj?.[op.method.toLowerCase()] as Record<string, unknown> | undefined
const found = findOperation(op.path, op.method)
if (!operation) {
if (!found) {
return `# ${title}\n\n${description || ''}`
}
const resolved = resolveRefs(operation, spec) as Record<string, unknown>
const resolved = resolveRefs(found.operation, found.spec) as Record<string, unknown>
const lines: string[] = []
lines.push(`# ${title}`)
+1
View File
@@ -11,6 +11,7 @@ const config: NextConfig = {
unoptimized: true,
},
experimental: {
useTypeScriptCli: true,
webpackMemoryOptimizations: true,
webpackBuildWorker: true,
},
+747
View File
@@ -0,0 +1,747 @@
{
"openapi": "3.1.0",
"info": {
"title": "Sim API v2 — Billing",
"description": "Version 2 of the Sim REST API for billing standing, credit allowance, storage quota, and cursor-paginated usage history.",
"version": "2.0.0",
"contact": {
"name": "Sim Support",
"email": "help@sim.ai",
"url": "https://www.sim.ai"
},
"license": {
"name": "Apache 2.0",
"url": "https://www.apache.org/licenses/LICENSE-2.0.html"
}
},
"servers": [
{
"url": "https://www.sim.ai",
"description": "Production"
}
],
"tags": [
{
"name": "Billing",
"description": "Inspect billing standing, credit allowance, storage quota, and usage history."
}
],
"security": [
{
"apiKey": []
}
],
"paths": {
"/api/v2/billing/status": {
"get": {
"operationId": "getBillingStatus",
"summary": "Get Billing Status",
"description": "Return the current plan, billing standing, credit allowance, and storage quota. Billing history lives at `GET /api/v2/billing/logs`.",
"tags": ["Billing"],
"parameters": [
{
"name": "workspaceId",
"in": "query",
"required": false,
"description": "Workspace whose payer should be resolved. Workspace API keys are pinned to their own workspace.",
"schema": {
"description": "Workspace whose payer should be resolved. Workspace API keys are pinned to their own workspace.",
"type": "string"
}
}
],
"responses": {
"200": {
"description": "The current billing and storage status.",
"headers": {
"X-RateLimit-Limit": {
"$ref": "#/components/headers/X-RateLimit-Limit"
},
"X-RateLimit-Remaining": {
"$ref": "#/components/headers/X-RateLimit-Remaining"
},
"X-RateLimit-Reset": {
"$ref": "#/components/headers/X-RateLimit-Reset"
}
},
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2BillingStatusResponse"
}
}
}
},
"400": {
"$ref": "#/components/responses/BadRequest"
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"403": {
"$ref": "#/components/responses/Forbidden"
},
"429": {
"$ref": "#/components/responses/RateLimited"
},
"500": {
"$ref": "#/components/responses/InternalError"
},
"503": {
"$ref": "#/components/responses/ServiceUnavailable"
}
}
}
},
"/api/v2/billing/logs": {
"get": {
"operationId": "listBillingLogs",
"summary": "List Billing Logs",
"description": "List the credit-denominated billing ledger with source filtering and opaque cursor pagination.",
"tags": ["Billing"],
"parameters": [
{
"name": "source",
"in": "query",
"required": false,
"description": "Restrict results to one usage source.",
"schema": {
"description": "Restrict results to one usage source.",
"type": "string",
"enum": [
"workflow",
"wand",
"sim-chat",
"mcp_copilot",
"mothership_block",
"knowledge-base",
"voice-input",
"enrichment",
"voice-output"
]
}
},
{
"name": "workspaceId",
"in": "query",
"required": false,
"description": "Restrict results to one workspace whose payer the caller can inspect.",
"schema": {
"description": "Restrict results to one workspace whose payer the caller can inspect.",
"type": "string"
}
},
{
"name": "period",
"in": "query",
"required": false,
"description": "Relative window, all history, or a custom date range.",
"schema": {
"default": "30d",
"description": "Relative window, all history, or a custom date range.",
"type": "string",
"enum": ["1d", "7d", "30d", "all", "custom"]
}
},
{
"name": "startDate",
"in": "query",
"required": false,
"description": "Start of a custom window as a Date-parseable string.",
"schema": {
"description": "Start of a custom window as a Date-parseable string.",
"type": "string",
"minLength": 1
}
},
{
"name": "endDate",
"in": "query",
"required": false,
"description": "End of a custom window as a Date-parseable string; defaults to now.",
"schema": {
"description": "End of a custom window as a Date-parseable string; defaults to now.",
"type": "string",
"minLength": 1
}
},
{
"name": "limit",
"in": "query",
"required": false,
"description": "Maximum usage events per page, from 1 to 100.",
"schema": {
"default": 50,
"description": "Maximum usage events per page, from 1 to 100.",
"type": "integer",
"minimum": 1,
"maximum": 100
}
},
{
"name": "cursor",
"in": "query",
"required": false,
"description": "Opaque cursor returned by the previous page.",
"schema": {
"description": "Opaque cursor returned by the previous page.",
"type": "string",
"minLength": 1
}
}
],
"responses": {
"200": {
"description": "A page of usage events.",
"headers": {
"X-RateLimit-Limit": {
"$ref": "#/components/headers/X-RateLimit-Limit"
},
"X-RateLimit-Remaining": {
"$ref": "#/components/headers/X-RateLimit-Remaining"
},
"X-RateLimit-Reset": {
"$ref": "#/components/headers/X-RateLimit-Reset"
}
},
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2BillingLogListResponse"
}
}
}
},
"400": {
"$ref": "#/components/responses/BadRequest"
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"403": {
"$ref": "#/components/responses/Forbidden"
},
"429": {
"$ref": "#/components/responses/RateLimited"
},
"500": {
"$ref": "#/components/responses/InternalError"
},
"503": {
"$ref": "#/components/responses/ServiceUnavailable"
}
}
}
}
},
"components": {
"securitySchemes": {
"apiKey": {
"type": "apiKey",
"in": "header",
"name": "X-API-Key",
"description": "Your Sim API key (personal or workspace). Generate one from the Sim dashboard under Settings > API Keys."
}
},
"headers": {
"X-RateLimit-Limit": {
"description": "Maximum requests allowed in the current window.",
"schema": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991,
"title": "Rate limit",
"description": "Maximum requests allowed in the current window."
}
},
"X-RateLimit-Remaining": {
"description": "Requests remaining in the current window.",
"schema": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991,
"title": "Rate limit remaining",
"description": "Requests remaining in the current window."
}
},
"X-RateLimit-Reset": {
"description": "ISO 8601 timestamp when the current rate-limit window resets.",
"schema": {
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$",
"title": "Rate limit reset",
"description": "ISO 8601 timestamp when the current rate-limit window resets."
}
},
"Retry-After": {
"description": "Seconds to wait before retrying a rate-limited request.",
"schema": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991,
"title": "Retry after",
"description": "Seconds to wait before retrying a rate-limited request."
}
},
"X-Run-Id": {
"description": "Identifier assigned to the workflow run.",
"schema": {
"type": "string",
"minLength": 1,
"title": "Run identifier",
"description": "Identifier assigned to the workflow run."
}
}
},
"responses": {
"BadRequest": {
"description": "The request is invalid.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"Unauthorized": {
"description": "The API key is missing or invalid.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"UsageLimitExceeded": {
"description": "The workspace has exceeded its usage or billing limits.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"Forbidden": {
"description": "The caller lacks access to the resource.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"NotFound": {
"description": "The requested resource was not found.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"Conflict": {
"description": "The request conflicts with current resource state.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"RunIdConflict": {
"description": "The run identifier is already associated with a different request.",
"headers": {
"X-Run-Id": {
"$ref": "#/components/headers/X-Run-Id"
}
},
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"Gone": {
"description": "The requested generated resource has expired.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"PayloadTooLarge": {
"description": "The request body exceeds the allowed size.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"UnsupportedMediaType": {
"description": "The request uses an unsupported media type.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"Locked": {
"description": "The resource is locked and cannot be modified.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"RateLimited": {
"description": "The caller exceeded the request rate limit.",
"headers": {
"Retry-After": {
"$ref": "#/components/headers/Retry-After"
}
},
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"InternalError": {
"description": "An unexpected server error occurred.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
},
"ServiceUnavailable": {
"description": "A required service is temporarily unavailable.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
}
}
}
}
},
"schemas": {
"V2Error": {
"type": "object",
"properties": {
"error": {
"type": "object",
"properties": {
"code": {
"type": "string",
"description": "Stable machine-readable error code."
},
"message": {
"type": "string",
"description": "Human-readable explanation of the error."
},
"details": {
"description": "Optional structured error details."
}
},
"required": ["code", "message"],
"additionalProperties": false,
"description": "Canonical error details."
}
},
"required": ["error"],
"additionalProperties": false,
"title": "v2 error response",
"description": "Canonical error envelope returned by the public v2 API.",
"examples": [
{
"error": {
"code": "BAD_REQUEST",
"message": "The request is invalid."
}
}
]
},
"V2BillingStatus": {
"type": "object",
"properties": {
"workspaceId": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Workspace whose payer was resolved, or null for account billing."
},
"period": {
"type": "object",
"properties": {
"start": {
"type": "string",
"description": "ISO 8601 start of the current billing period.",
"format": "date-time"
},
"end": {
"type": "string",
"description": "ISO 8601 end of the current billing period.",
"format": "date-time"
}
},
"required": ["start", "end"],
"additionalProperties": false,
"description": "Current billing period."
},
"plan": {
"type": "string",
"description": "Current billing plan."
},
"status": {
"type": "string",
"enum": ["active", "limit_exceeded", "billing_blocked"],
"description": "Current billing standing."
},
"credits": {
"type": "object",
"properties": {
"used": {
"type": "number",
"description": "Credits consumed during the current billing period."
},
"limit": {
"type": "number",
"description": "Credit allowance for the current billing period."
},
"remaining": {
"type": "number",
"description": "Credits remaining in the current billing period."
}
},
"required": ["used", "limit", "remaining"],
"additionalProperties": false,
"description": "Credit usage and allowance for the current billing period."
},
"storage": {
"type": "object",
"properties": {
"usedBytes": {
"type": "number",
"minimum": 0,
"description": "Storage currently consumed, in bytes."
},
"limitBytes": {
"type": "number",
"minimum": 0,
"description": "Storage quota, in bytes."
},
"percentUsed": {
"type": "number",
"minimum": 0,
"description": "Percentage of the storage quota consumed."
}
},
"required": ["usedBytes", "limitBytes", "percentUsed"],
"additionalProperties": false,
"description": "Current storage consumption and quota."
}
},
"required": ["workspaceId", "period", "plan", "status", "credits", "storage"],
"additionalProperties": false,
"title": "Billing status",
"description": "Current billing standing, credit allowance, and storage quota."
},
"V2BillingStatusResponse": {
"type": "object",
"properties": {
"data": {
"description": "Response data.",
"$ref": "#/components/schemas/V2BillingStatus"
}
},
"required": ["data"],
"additionalProperties": false,
"title": "Billing status response",
"description": "Current billing standing, credit allowance, and storage quota.",
"examples": [
{
"data": {
"workspaceId": null,
"period": {
"start": "2026-07-01T00:00:00.000Z",
"end": "2026-08-01T00:00:00.000Z"
},
"plan": "pro",
"status": "active",
"credits": {
"used": 512,
"limit": 20000,
"remaining": 19488
},
"storage": {
"usedBytes": 5242880,
"limitBytes": 1073741824,
"percentUsed": 0.48828125
}
}
}
]
},
"V2BillingLogEntry": {
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "Unique usage-event identifier."
},
"createdAt": {
"type": "string",
"description": "ISO 8601 timestamp when the usage event was recorded.",
"format": "date-time"
},
"source": {
"type": "string",
"enum": [
"workflow",
"wand",
"sim-chat",
"mcp_copilot",
"mothership_block",
"knowledge-base",
"voice-input",
"enrichment",
"voice-output"
],
"description": "Product surface that consumed the credits."
},
"workspaceId": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Workspace attributed to the event, or null for account-level usage."
},
"workflow": {
"anyOf": [
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "Workflow identifier."
},
"name": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Workflow display name, when available."
}
},
"required": ["id", "name"],
"additionalProperties": false
},
{
"type": "null"
}
],
"description": "Workflow attributed to the event, when applicable."
},
"runId": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Workflow run attributed to the event, when applicable."
},
"creditCost": {
"type": "number",
"description": "Credits apportioned to the event so page rows sum to the rounded page total; may be zero for a sub-credit event."
}
},
"required": ["id", "createdAt", "source", "workspaceId", "workflow", "runId", "creditCost"],
"additionalProperties": false,
"title": "Billing log entry",
"description": "One credit-consuming usage event in the billing ledger."
},
"V2BillingLogListResponse": {
"type": "object",
"properties": {
"data": {
"type": "array",
"items": {
"$ref": "#/components/schemas/V2BillingLogEntry"
},
"description": "Items in the current page."
},
"nextCursor": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Opaque cursor for the next page, or null when no more items remain."
}
},
"required": ["data", "nextCursor"],
"additionalProperties": false,
"title": "Billing log list response",
"description": "A cursor-paginated page of credit-consuming usage events.",
"examples": [
{
"data": [
{
"id": "log_1",
"createdAt": "2026-07-29T18:04:11.000Z",
"source": "sim-chat",
"workspaceId": "ws_1",
"workflow": null,
"runId": null,
"creditCost": 12
}
],
"nextCursor": null
}
]
}
}
},
"x-generated-by": "scripts/generate-openapi.ts"
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+35 -21
View File
@@ -11,6 +11,16 @@ These rules apply to files under `apps/sim/` in addition to the repository root
3. **Type Safety First**: TypeScript interfaces for all props, state, return types
4. **Predictable State**: Zustand for global state, useState for UI-only concerns
### Application Operation Boundary
- Every protected read, write, canonical resource lookup, or authorization-sensitive reference resolution enters through an authorized application use case.
- Define one stable semantic operation with its role, workspace-key policy, principal kinds, and delegated services. Internal, v2, Copilot, and trusted-tool adapters call the same use case when domain behavior is the same.
- Surface adapters authenticate and build a `Principal`, rate-limit, parse, map, and present. They do not query protected data, authorize resources, implement business transactions, or record semantic audit.
- Application use cases canonicalize scope, authorize current access, execute managers/repositories, project semantic audit, and trigger shared domain effects. Application code stays surface-neutral and never imports `app/api/**`, `next/server`, route contracts/presenters, or Copilot handlers.
- Copilot uses `createCopilotApplicationAdapter`; it does not own separate protected business logic. Compound protected mutations require a top-level semantic application operation.
- Never substitute billing attribution, an uploader, creator, or API-key owner for the acting principal. Fail fast when the identity or policy cannot be represented.
- Use the `migrate-application-operation` skill for new and migrated protected endpoints, tool commands, and resource methods.
### Root-Level Structure
```
@@ -130,44 +140,49 @@ output: z.unknown(),
## API Route Pattern
Routes never `import { z } from 'zod'` and never define route-local boundary schemas. They consume the contract from `@/lib/api/contracts/**` and validate with canonical helpers from `@/lib/api/server`:
Every route method must run inside `withRouteHandler`. Ordinary internal and v2 JSON/binary routes use `defineInternalJsonRoute`, `defineV2JsonRoute`, or the matching binary/stream builder. These builders already apply `withRouteHandler`; never wrap them again. Use raw `withRouteHandler` only for explicit protocol or lifecycle exceptions such as streaming, multipart control, large-body admission, OAuth, or public execution.
Routes never `import { z } from 'zod'` and never define route-local boundary schemas. Declarative builders consume contracts and own authentication, admission, parsing, use-case execution, response validation, and error projection. A raw special route consumes the same contracts and validates with canonical helpers from `@/lib/api/server`, after authentication and cheap admission:
- `parseRequest(contract, request, context, options?)` — fully contract-bound routes; parses params, query, body, and headers in one call. Pass `{}` for `context` on routes without route params, or the route's `context` argument when route params exist. Returns a discriminated union; check `parsed.success` and return `parsed.response` on failure.
- `validationErrorResponse(error)` and `getValidationErrorMessage(error, fallback)` — produce 400 responses from a `ZodError`.
- `validationErrorResponseFromError(error)` — when handling unknown caught errors that may or may not be a `ZodError`.
- `isZodError(error)` — type guard. Routes never use `instanceof z.ZodError`.
### Fully contract-bound route (`parseRequest`)
### Ordinary authorized JSON route
```typescript
import { createLogger } from '@sim/logger'
import type { NextRequest } from 'next/server'
import { NextResponse } from 'next/server'
import { createFolderContract } from '@/lib/api/contracts/folders'
import { parseRequest } from '@/lib/api/server'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
const logger = createLogger('FoldersAPI')
export const POST = withRouteHandler(async (request: NextRequest) => {
const parsed = await parseRequest(createFolderContract, request, {})
if (!parsed.success) return parsed.response
const { body } = parsed.data
logger.info('Creating folder', { workspaceId: body.workspaceId })
return NextResponse.json({ ok: true })
export const PATCH = defineInternalJsonRoute({
contract: renameWidgetContract,
auth: internalSessionAuth,
operation: widgetOperations.rename,
rateLimit: internalRateLimits.none({ reason: 'Preserve existing internal behavior' }),
errorPolicy: internalWidgetErrorPolicy,
mapInput: ({ params, body }) => ({
widgetId: params.widgetId,
assertedWorkspaceId: params.workspaceId,
name: body.name,
}),
useCase: renameWidget,
present: ({ widget }) => ({ success: true, widget }),
})
```
The contract, operation, and use case must agree at definition time. Authentication and request-rate admission happen before parsing; canonical loading and authorization happen in the application use case. The presenter returns only the surface success body.
Routes under `apps/sim/app/api/v1/**` use the shared middleware in `apps/sim/app/api/v1/middleware.ts` for auth, rate-limit, and workspace access. Compose contract validation inside that middleware — never reimplement auth/rate-limit per-route.
Never export a bare `async function GET/POST/...`. Export the result of a shared builder or, for a documented special route, `withRouteHandler(...)`.
### Adding a new boundary feature end-to-end
When adding a new route + client surface, follow this order. Each step has one place it lives.
1. **Author the contract first** in `apps/sim/lib/api/contracts/<domain>.ts` (or a subdirectory for large domains: `knowledge/`, `selectors/`, `tools/`). Define one schema per request slice (`params`, `query`, `body`, `headers`) and one for the response, then wrap with `defineRouteContract`. Export named type aliases (`z.input` for inputs, `z.output` for outputs).
2. **Implement the route** in `apps/sim/app/api/<path>/route.ts`. Auth always runs **before** `parseRequest` — never validate untrusted input before authenticating the caller. The route returns exactly the shape declared in `contract.response.schema`.
3. **Add the React Query hook** in `apps/sim/hooks/queries/<domain>.ts`. Use `requestJson(contract, input)` for the call. Build a hierarchical query-key factory (`all` → `lists()` → `list(workspaceId)` → `details()` → `detail(id)`) so invalidations can target prefixes.
4. **Use the hook in the component**. The mutation's `data` and `error` are fully typed from the contract; surface `error.message` (already extracted from the response body's `error` or `message` field by `requestJson`).
2. **Define the semantic operation and application use case** under `apps/sim/lib/<domain>/application/`. The use case owns canonical loading, asserted-scope checks, current authorization, business behavior, semantic audit, and shared domain effects.
3. **Implement the route adapter** in `apps/sim/app/api/<path>/route.ts` with the appropriate shared builder. Declare auth, operation, rate policy, error policy, input mapping, use case, and presenter. Auth always runs **before** parsing. Use raw `withRouteHandler` only for an explicit special route, and keep protected work in application use cases.
4. **Add the React Query hook** in `apps/sim/hooks/queries/<domain>.ts`. Use `requestJson(contract, input)` for the call. Build a hierarchical query-key factory (`all` → `lists()` → `list(workspaceId)` → `details()` → `detail(id)`) so invalidations can target prefixes.
5. **Use the hook in the component**. The mutation's `data` and `error` are fully typed from the contract; surface `error.message` (already extracted from the response body's `error` or `message` field by `requestJson`).
### Schema review checklist (read the contract diff like a DB migration)
@@ -228,4 +243,3 @@ export function useEntityList(workspaceId?: string) {
- **Create `utils.ts` when** 2+ files need the same helper
- **Check existing sources** before duplicating (`lib/` has many utilities)
- **Location**: `lib/` (app-wide) → `feature/utils/` (feature-scoped) → inline (single-use)
@@ -22,7 +22,7 @@ vi.mock('@/app/api/v1/audit-logs/auth', () => ({
validateEnterpriseAuditAccess: mockValidateEnterpriseAuditAccess,
}))
vi.mock('@/app/api/v1/audit-logs/query', () => ({
vi.mock('@/lib/audit-logs/query', () => ({
buildFilterConditions: mockBuildFilterConditions,
buildOrgScopeCondition: mockBuildOrgScopeCondition,
getOrgWorkspaceIds: mockGetOrgWorkspaceIds,
+6 -6
View File
@@ -3,17 +3,17 @@ import { getErrorMessage } from '@sim/utils/errors'
import { type NextRequest, NextResponse } from 'next/server'
import { exportAuditLogsContract } from '@/lib/api/contracts/audit-logs'
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { getSession } from '@/lib/auth'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { formatCsvValue, toCsvRow } from '@/lib/table/export-format'
import { validateEnterpriseAuditAccess } from '@/app/api/v1/audit-logs/auth'
import { formatAuditLogEntry } from '@/app/api/v1/audit-logs/format'
import {
buildFilterConditions,
buildOrgScopeCondition,
getOrgWorkspaceIds,
queryAuditLogs,
} from '@/app/api/v1/audit-logs/query'
} from '@/lib/audit-logs/query'
import { getSession } from '@/lib/auth'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { formatCsvValue, toCsvRow } from '@/lib/table/export-format'
import { validateEnterpriseAuditAccess } from '@/app/api/v1/audit-logs/auth'
import { formatAuditLogEntry } from '@/app/api/v1/audit-logs/format'
const logger = createLogger('AuditLogsExportAPI')
+83
View File
@@ -0,0 +1,83 @@
/**
* @vitest-environment node
*/
import { NextRequest } from 'next/server'
import { beforeEach, describe, expect, it, vi } from 'vitest'
const mocks = vi.hoisted(() => ({
getSession: vi.fn(),
execute: vi.fn(),
}))
vi.mock('@/lib/auth', () => ({ getSession: mocks.getSession }))
vi.mock('@/lib/audit-logs/application/list-audit-logs', () => ({
listAuditLogs: { operation: { id: 'audit_logs.list' }, execute: mocks.execute },
}))
import { OrchestrationError } from '@/lib/core/orchestration/types'
import { GET } from '@/app/api/audit-logs/route'
const log = {
id: 'audit-1',
workspaceId: 'workspace-1',
actorId: 'admin-1',
actorName: 'Ada',
actorEmail: 'ada@example.com',
action: 'workspace.updated',
resourceType: 'workspace',
resourceId: 'workspace-1',
resourceName: 'Engineering',
description: null,
metadata: {},
createdAt: new Date('2026-08-01T00:00:00Z'),
}
describe('GET /api/audit-logs', () => {
beforeEach(() => {
vi.clearAllMocks()
mocks.getSession.mockResolvedValue({
user: { id: 'admin-1' },
session: { id: 'session-1' },
})
mocks.execute.mockResolvedValue({ data: [log], nextCursor: 'next-1' })
})
it('authenticates before parsing the organization query', async () => {
const response = await GET(new NextRequest('http://localhost:3000/api/audit-logs'))
expect(response.status).toBe(400)
expect(mocks.getSession).toHaveBeenCalled()
expect(mocks.execute).not.toHaveBeenCalled()
})
it('keeps the internal envelope while sharing the application operation', async () => {
const request = new NextRequest(
'http://localhost:3000/api/audit-logs?organizationId=organization-1'
)
const response = await GET(request)
expect(response.status).toBe(200)
expect(await response.json()).toMatchObject({
success: true,
data: [{ id: 'audit-1', actorId: 'admin-1' }],
nextCursor: 'next-1',
})
expect(mocks.execute).toHaveBeenCalledWith({
principal: { kind: 'session', userId: 'admin-1', sessionId: 'session-1' },
input: expect.objectContaining({ organizationId: 'organization-1' }),
request,
})
})
it('preserves internal typed error presentation', async () => {
mocks.execute.mockRejectedValueOnce(new OrchestrationError('forbidden', 'Admin required'))
const response = await GET(
new NextRequest('http://localhost:3000/api/audit-logs?organizationId=organization-1')
)
expect(response.status).toBe(403)
expect(await response.json()).toEqual({ error: 'Admin required' })
})
})
+36 -91
View File
@@ -1,97 +1,42 @@
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { type NextRequest, NextResponse } from 'next/server'
import { listAuditLogsContract } from '@/lib/api/contracts/audit-logs'
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { getSession } from '@/lib/auth'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { validateEnterpriseAuditAccess } from '@/app/api/v1/audit-logs/auth'
import { formatAuditLogEntry } from '@/app/api/v1/audit-logs/format'
import {
buildFilterConditions,
buildOrgScopeCondition,
getOrgWorkspaceIds,
queryAuditLogs,
} from '@/app/api/v1/audit-logs/query'
const logger = createLogger('AuditLogsAPI')
defineInternalJsonRoute,
internalPlainOrchestrationErrorPolicy,
internalRateLimits,
internalSessionAuth,
} from '@/lib/api/server/routes'
import { listAuditLogs } from '@/lib/audit-logs/application/list-audit-logs'
import { auditLogOperations } from '@/lib/audit-logs/application/operations'
import { formatAuditLogEntry } from '@/app/api/v1/audit-logs/format'
export const dynamic = 'force-dynamic'
export const GET = withRouteHandler(async (request: NextRequest) => {
try {
const session = await getSession()
if (!session?.user?.id) {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const parsed = await parseRequest(
listAuditLogsContract,
request,
{},
{
validationErrorResponse: (error) =>
NextResponse.json(
{ error: getValidationErrorMessage(error, 'Invalid query parameters') },
{ status: 400 }
),
}
)
if (!parsed.success) return parsed.response
const authResult = await validateEnterpriseAuditAccess(
session.user.id,
parsed.data.query.organizationId
)
if (!authResult.success) {
return authResult.response
}
const { organizationId, orgMemberIds } = authResult.context
const {
organizationId: _targetOrganizationId,
search,
action,
resourceType,
actorId,
startDate,
endDate,
includeDeparted,
limit,
cursor,
} = parsed.data.query
const orgWorkspaceIds = await getOrgWorkspaceIds(organizationId)
const scopeCondition = buildOrgScopeCondition({
organizationId,
orgWorkspaceIds,
orgMemberIds,
includeDeparted,
})
const filterConditions = buildFilterConditions({
action,
resourceType,
actorId,
search,
startDate,
endDate,
})
const { data, nextCursor } = await queryAuditLogs(
[scopeCondition, ...filterConditions],
limit,
cursor
)
return NextResponse.json({
success: true,
data: data.map(formatAuditLogEntry),
nextCursor,
})
} catch (error: unknown) {
const message = getErrorMessage(error, 'Unknown error')
logger.error('Audit logs fetch error', { error: message })
return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
}
export const GET = defineInternalJsonRoute({
contract: listAuditLogsContract,
auth: internalSessionAuth,
operation: auditLogOperations.list,
rateLimit: internalRateLimits.none({
reason: 'Existing authenticated audit-log settings read has no request-rate policy',
}),
errorPolicy: internalPlainOrchestrationErrorPolicy,
mapInput: ({ query }) => ({
organizationId: query.organizationId,
includeDeparted: query.includeDeparted,
filters: {
search: query.search,
action: query.action,
resourceType: query.resourceType,
actorId: query.actorId,
startDate: query.startDate,
endDate: query.endDate,
},
limit: query.limit,
cursor: query.cursor,
}),
useCase: listAuditLogs,
present: ({ data, nextCursor }) => ({
success: true,
data: data.map(formatAuditLogEntry),
nextCursor,
}),
})
@@ -6,7 +6,7 @@ import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { validateMicrosoftGraphId } from '@/lib/core/security/input-validation'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { getCredential, refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { getCredential, refreshAccessTokenIfNeeded } from '@/lib/oauth/credential-service'
export const dynamic = 'force-dynamic'
@@ -6,7 +6,7 @@ import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { validatePathSegment } from '@/lib/core/security/input-validation'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { getCredential, refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { getCredential, refreshAccessTokenIfNeeded } from '@/lib/oauth/credential-service'
import { GRAPH_ID_PATTERN } from '@/tools/microsoft_excel/utils'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
@@ -17,7 +17,7 @@ const { mockAuthorizeCredentialUse, mockResolveServiceAccountToken } = vi.hoiste
mockResolveServiceAccountToken: vi.fn(),
}))
vi.mock('@/app/api/auth/oauth/utils', () => ({
vi.mock('@/lib/oauth/credential-service', () => ({
...authOAuthUtilsMock,
resolveServiceAccountToken: mockResolveServiceAccountToken,
}))
+3 -3
View File
@@ -12,15 +12,15 @@ import { AuthType, checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { TokenServiceAccountValidationError } from '@/lib/credentials/token-service-accounts/errors'
import { extractSalesforceInstanceUrl, isSalesforceOAuthProviderId } from '@/lib/oauth/salesforce'
import { captureServerEvent } from '@/lib/posthog/server'
import {
getCredential,
getOAuthToken,
refreshTokenIfNeeded,
resolveOAuthAccountId,
resolveServiceAccountToken,
} from '@/app/api/auth/oauth/utils'
} from '@/lib/oauth/credential-service'
import { extractSalesforceInstanceUrl, isSalesforceOAuthProviderId } from '@/lib/oauth/salesforce'
import { captureServerEvent } from '@/lib/posthog/server'
import { extractZohoDeskBaseFromScope } from '@/tools/zoho_desk/host-allowlist'
export const dynamic = 'force-dynamic'
+6 -6
View File
@@ -28,17 +28,17 @@ import { db } from '@sim/db'
import { __resetCoalesceLocallyForTests } from '@/lib/concurrency/singleflight'
import { ZOOM_SERVICE_ACCOUNT_PROVIDER_ID } from '@/lib/credentials/client-credential-accounts/descriptors'
import { refreshOAuthToken } from '@/lib/oauth'
import {
ATLASSIAN_SERVICE_ACCOUNT_PROVIDER_ID,
GOOGLE_SERVICE_ACCOUNT_PROVIDER_ID,
SLACK_CUSTOM_BOT_PROVIDER_ID,
} from '@/lib/oauth/types'
import {
getCredential,
refreshAccessTokenIfNeeded,
refreshTokenIfNeeded,
resolveServiceAccountToken,
} from '@/app/api/auth/oauth/utils'
} from '@/lib/oauth/credential-service'
import {
ATLASSIAN_SERVICE_ACCOUNT_PROVIDER_ID,
GOOGLE_SERVICE_ACCOUNT_PROVIDER_ID,
SLACK_CUSTOM_BOT_PROVIDER_ID,
} from '@/lib/oauth/types'
const mockDb = db as any
const mockRefreshOAuthToken = refreshOAuthToken as any
@@ -6,7 +6,7 @@ import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { validateEnum, validatePathSegment } from '@/lib/core/security/input-validation'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { refreshAccessTokenIfNeeded } from '@/lib/oauth/credential-service'
export const dynamic = 'force-dynamic'
@@ -6,7 +6,7 @@ import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { validatePathSegment } from '@/lib/core/security/input-validation'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { refreshAccessTokenIfNeeded } from '@/lib/oauth/credential-service'
export const dynamic = 'force-dynamic'
@@ -18,13 +18,13 @@ import { getBaseUrl } from '@/lib/core/utils/urls'
import { isSameOrigin } from '@/lib/core/utils/validation'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { processCredentialDraft } from '@/lib/credentials/draft-processor'
import { safeAccountInsert } from '@/lib/oauth/credential-service'
import {
parseInstagramLongLivedToken,
parseInstagramProfile,
parseInstagramShortLivedToken,
} from '@/lib/oauth/instagram'
import { getCanonicalScopesForProvider } from '@/lib/oauth/utils'
import { safeAccountInsert } from '@/app/api/auth/oauth/utils'
import { INSTAGRAM_GRAPH_BASE } from '@/tools/instagram/constants'
const logger = createLogger('InstagramCallback')
@@ -12,7 +12,7 @@ import { getBaseUrl } from '@/lib/core/utils/urls'
import { isSameOrigin } from '@/lib/core/utils/validation'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { processCredentialDraft } from '@/lib/credentials/draft-processor'
import { safeAccountInsert } from '@/app/api/auth/oauth/utils'
import { safeAccountInsert } from '@/lib/oauth/credential-service'
import { SHOPIFY_API_VERSION } from '@/tools/shopify/constants'
const logger = createLogger('ShopifyStore')
+1 -1
View File
@@ -9,8 +9,8 @@ import { getSession } from '@/lib/auth'
import { env } from '@/lib/core/config/env'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { processCredentialDraft } from '@/lib/credentials/draft-processor'
import { safeAccountInsert } from '@/lib/oauth/credential-service'
import { getCanonicalScopesForProvider } from '@/lib/oauth/utils'
import { safeAccountInsert } from '@/app/api/auth/oauth/utils'
const logger = createLogger('TrelloStore')
@@ -23,14 +23,16 @@ import {
import { NextRequest } from 'next/server'
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'
const { mockCheckChatAccess, mockValidateChatDeployAuth } = vi.hoisted(() => ({
mockCheckChatAccess: vi.fn(),
mockValidateChatDeployAuth: vi.fn(),
}))
const { mockCheckChatAccess, mockCheckNeedsRedeployment, mockValidateChatDeployAuth } = vi.hoisted(
() => ({
mockCheckChatAccess: vi.fn(),
mockCheckNeedsRedeployment: vi.fn(),
mockValidateChatDeployAuth: vi.fn(),
})
)
const mockCreateSuccessResponse = workflowsApiUtilsMockFns.mockCreateSuccessResponse
const mockCreateErrorResponse = workflowsApiUtilsMockFns.mockCreateErrorResponse
const mockCheckNeedsRedeployment = workflowsApiUtilsMockFns.mockCheckNeedsRedeployment
const mockEncryptSecret = encryptionMockFns.mockEncryptSecret
const mockPerformFullDeploy = workflowsOrchestrationMockFns.mockPerformFullDeploy
const mockPerformChatUndeploy = workflowsOrchestrationMockFns.mockPerformChatUndeploy
@@ -56,6 +58,9 @@ vi.mock('@/ee/access-control/utils/permission-check', () => {
})
vi.mock('@/lib/workflows/persistence/utils', () => workflowsPersistenceUtilsMock)
vi.mock('@/lib/workflows/orchestration', () => workflowsOrchestrationMock)
vi.mock('@/lib/workflows/deployment-status', () => ({
checkNeedsRedeployment: mockCheckNeedsRedeployment,
}))
import { DELETE, GET, PATCH } from '@/app/api/chat/manage/[id]/route'
import { ChatDeployAuthNotAllowedError } from '@/ee/access-control/utils/permission-check'
+2 -5
View File
@@ -12,17 +12,14 @@ import { isDev } from '@/lib/core/config/env-flags'
import { encryptSecret } from '@/lib/core/security/encryption'
import { getEmailDomain } from '@/lib/core/utils/urls'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { checkNeedsRedeployment } from '@/lib/workflows/deployment-status'
import {
getWorkflowDeploymentSummary,
performChatUndeploy,
performFullDeploy,
} from '@/lib/workflows/orchestration'
import { checkChatAccess } from '@/app/api/chat/utils'
import {
checkNeedsRedeployment,
createErrorResponse,
createSuccessResponse,
} from '@/app/api/workflows/utils'
import { createErrorResponse, createSuccessResponse } from '@/app/api/workflows/utils'
import {
ChatDeployAuthNotAllowedError,
validateChatDeployAuth,
+8 -2
View File
@@ -5,7 +5,11 @@ import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { getSession } from '@/lib/auth'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { type CredentialActorContext, getCredentialActorContext } from '@/lib/credentials/access'
import { performDeleteCredential, performUpdateCredential } from '@/lib/credentials/orchestration'
import {
isProviderOutageCode,
performDeleteCredential,
performUpdateCredential,
} from '@/lib/credentials/orchestration'
const logger = createLogger('CredentialByIdAPI')
@@ -105,7 +109,9 @@ export const PUT = withRouteHandler(
? 409
: // A provider outage during reconnect is infra, not a bad
// request — mirror the create route and runtime token route.
result.providerErrorCode === 'provider_unavailable'
// Every provider family names its own outage code, so this
// asks the shared predicate rather than matching one literal.
isProviderOutageCode(result.providerErrorCode)
? 502
: result.errorCode === 'validation'
? 400
+52 -513
View File
@@ -1,38 +1,22 @@
import { AuditAction, AuditResourceType, recordAudit } from '@sim/audit'
import { db } from '@sim/db'
import { account, credential, credentialMember } from '@sim/db/schema'
import { credential } from '@sim/db/schema'
import { createLogger } from '@sim/logger'
import { getPostgresErrorCode } from '@sim/utils/errors'
import { generateId } from '@sim/utils/id'
import { and, eq, inArray, isNotNull, or } from 'drizzle-orm'
import { and, eq } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server'
import {
createWorkspaceCredentialContract,
credentialsListGetQuerySchema,
normalizeCredentialEnvKey,
} from '@/lib/api/contracts/credentials'
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { getSession } from '@/lib/auth'
import { acquireOrganizationUserMutationLocks } from '@/lib/billing/organizations/membership'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import {
getCredentialActorContext,
isSharedCredentialType,
SHARED_CREDENTIAL_TYPES,
} from '@/lib/credentials/access'
import { AtlassianValidationError } from '@/lib/credentials/atlassian-service-account'
import { getCredentialCreationWorkspaceContext } from '@/lib/credentials/environment'
import { syncWorkspaceOAuthCredentialsForUser } from '@/lib/credentials/oauth'
import {
ServiceAccountSecretError,
verifyAndBuildServiceAccountSecret,
} from '@/lib/credentials/service-account-secret'
import { isTokenServiceAccountProviderId } from '@/lib/credentials/token-service-accounts/descriptors'
import { TokenServiceAccountValidationError } from '@/lib/credentials/token-service-accounts/errors'
import { getServiceConfigByProviderId, providerIdsForService } from '@/lib/oauth'
import { SLACK_CUSTOM_BOT_PROVIDER_ID } from '@/lib/oauth/types'
import { captureServerEvent } from '@/lib/posthog/server'
performCreateCredential,
statusForCredentialOrchestrationError,
} from '@/lib/credentials/orchestration/credential-create'
import { listVisibleWorkspaceCredentials } from '@/lib/credentials/queries'
import { checkWorkspaceAccess } from '@/lib/workspaces/permissions/utils'
const logger = createLogger('CredentialsAPI')
@@ -231,59 +215,14 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
await syncWorkspaceOAuthCredentialsForUser({ workspaceId, userId: session.user.id })
}
const whereClauses = [eq(credential.workspaceId, workspaceId)]
if (type) {
whereClauses.push(eq(credential.type, type))
}
if (providerId) {
whereClauses.push(inArray(credential.providerId, providerIdsForService(providerId)))
}
const isWorkspaceAdmin = workspaceAccess.canAdmin
const accessClause = isWorkspaceAdmin
? or(
isNotNull(credentialMember.id),
inArray(credential.type, SHARED_CREDENTIAL_TYPES),
eq(credential.envOwnerUserId, session.user.id)
)
: or(isNotNull(credentialMember.id), eq(credential.envOwnerUserId, session.user.id))
const rows = await db
.select({
id: credential.id,
workspaceId: credential.workspaceId,
type: credential.type,
displayName: credential.displayName,
description: credential.description,
providerId: credential.providerId,
accountId: credential.accountId,
envKey: credential.envKey,
envOwnerUserId: credential.envOwnerUserId,
createdBy: credential.createdBy,
createdAt: credential.createdAt,
updatedAt: credential.updatedAt,
memberRole: credentialMember.role,
})
.from(credential)
.leftJoin(
credentialMember,
and(
eq(credentialMember.credentialId, credential.id),
eq(credentialMember.userId, session.user.id),
eq(credentialMember.status, 'active')
)
)
.where(and(...whereClauses, accessClause))
const credentials = rows.map(({ memberRole, ...rest }) => ({
...rest,
role:
(rest.type === 'env_personal' && rest.envOwnerUserId === session.user.id) ||
(isWorkspaceAdmin && isSharedCredentialType(rest.type))
? 'admin'
: (memberRole ?? 'member'),
}))
const visible = await listVisibleWorkspaceCredentials({
workspaceId,
userId: session.user.id,
workspaceAccess,
types: type ? [type] : undefined,
providerId,
})
const credentials = visible.map(({ hasServiceAccountKey: _hasKey, ...rest }) => rest)
return NextResponse.json({ credentials })
} catch (error) {
@@ -300,444 +239,44 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
try {
const parsed = await parseRequest(
createWorkspaceCredentialContract,
request,
{},
{
validationErrorResponse: (error) =>
NextResponse.json({ error: getValidationErrorMessage(error) }, { status: 400 }),
}
const parsed = await parseRequest(
createWorkspaceCredentialContract,
request,
{},
{
validationErrorResponse: (error) =>
NextResponse.json({ error: getValidationErrorMessage(error) }, { status: 400 }),
}
)
if (!parsed.success) return parsed.response
const result = await performCreateCredential({
...parsed.data.body,
userId: session.user.id,
actorName: session.user.name,
actorEmail: session.user.email,
request,
})
if (!result.success) {
logger.warn(`[${requestId}] Credential create rejected`, {
errorCode: result.errorCode,
providerErrorCode: result.providerErrorCode,
})
const status = statusForCredentialOrchestrationError(result.errorCode, {
providerUnavailable: result.providerUnavailable,
})
return NextResponse.json(
result.providerErrorCode
? { code: result.providerErrorCode, error: result.error }
: { error: result.error },
{ status }
)
if (!parsed.success) return parsed.response
const {
workspaceId,
type,
displayName,
description,
providerId,
accountId,
envKey,
envOwnerUserId,
serviceAccountJson,
apiToken,
domain,
id: clientCredentialId,
signingSecret,
botToken,
clientId,
clientSecret,
orgId,
dataCenter,
authMethod,
privateKey,
username,
} = parsed.data.body
const workspaceAccess = await checkWorkspaceAccess(workspaceId, session.user.id)
if (!workspaceAccess.canWrite) {
return NextResponse.json({ error: 'Write permission required' }, { status: 403 })
}
let resolvedDisplayName = displayName?.trim() ?? ''
const resolvedDescription = description?.trim() || null
let resolvedProviderId: string | null = providerId ?? null
let resolvedAccountId: string | null = accountId ?? null
const resolvedEnvKey: string | null = envKey ? normalizeCredentialEnvKey(envKey) : null
let resolvedEnvOwnerUserId: string | null = null
let resolvedEncryptedServiceAccountKey: string | null = null
const extraAuditMetadata: Record<string, unknown> = {}
if (type === 'oauth') {
const [accountRow] = await db
.select({
id: account.id,
userId: account.userId,
providerId: account.providerId,
accountId: account.accountId,
})
.from(account)
.where(eq(account.id, accountId!))
.limit(1)
if (!accountRow) {
return NextResponse.json({ error: 'OAuth account not found' }, { status: 404 })
}
if (accountRow.userId !== session.user.id) {
return NextResponse.json(
{ error: 'Only account owners can create oauth credentials for an account' },
{ status: 403 }
)
}
if (providerId !== accountRow.providerId) {
return NextResponse.json(
{ error: 'providerId does not match the selected OAuth account' },
{ status: 400 }
)
}
if (!resolvedDisplayName) {
resolvedDisplayName =
getServiceConfigByProviderId(accountRow.providerId)?.name || accountRow.providerId
}
} else if (type === 'service_account') {
try {
const secret = await verifyAndBuildServiceAccountSecret(providerId ?? '', {
signingSecret,
botToken,
apiToken,
domain,
serviceAccountJson,
clientId,
clientSecret,
orgId,
dataCenter,
authMethod,
privateKey,
username,
})
resolvedProviderId = secret.providerId
resolvedAccountId = null
resolvedEnvOwnerUserId = null
if (!resolvedDisplayName) {
resolvedDisplayName = secret.displayName
}
resolvedEncryptedServiceAccountKey = secret.encryptedServiceAccountKey
Object.assign(extraAuditMetadata, secret.auditMetadata)
} catch (error) {
if (error instanceof ServiceAccountSecretError) {
return NextResponse.json({ error: error.message }, { status: 400 })
}
throw error
}
} else if (type === 'env_personal') {
resolvedEnvOwnerUserId = envOwnerUserId ?? session.user.id
if (resolvedEnvOwnerUserId !== session.user.id) {
return NextResponse.json(
{ error: 'Only the current user can create personal env credentials for themselves' },
{ status: 403 }
)
}
resolvedProviderId = null
resolvedAccountId = null
resolvedDisplayName = resolvedEnvKey || ''
} else {
resolvedProviderId = null
resolvedAccountId = null
resolvedEnvOwnerUserId = null
resolvedDisplayName = resolvedEnvKey || ''
}
if (!resolvedDisplayName) {
return NextResponse.json({ error: 'Display name is required' }, { status: 400 })
}
const existingCredential = await findExistingCredentialBySource({
workspaceId,
type,
accountId: resolvedAccountId,
envKey: resolvedEnvKey,
envOwnerUserId: resolvedEnvOwnerUserId,
displayName: resolvedDisplayName,
providerId: resolvedProviderId,
})
if (existingCredential) {
// A retried custom-bot create with the SAME pre-generated id is an
// idempotent replay and falls through to the normal existing-credential
// path. Any other name collision must fail loudly: returning the existing
// row as success would orphan the new id already embedded in the user's
// Slack Request URL (Slack would post to a URL no credential resolves).
if (
resolvedProviderId === SLACK_CUSTOM_BOT_PROVIDER_ID &&
clientCredentialId &&
existingCredential.id !== clientCredentialId
) {
return NextResponse.json(
{
code: 'duplicate_display_name',
error: `A Slack bot named "${resolvedDisplayName}" already exists in this workspace. Give this bot a different name.`,
},
{ status: 409 }
)
}
// Token service-account creates always carry a fresh token that must be
// stored — falling through to the existing-credential path would return
// the old credential as success and silently drop the submitted token.
if (resolvedProviderId && isTokenServiceAccountProviderId(resolvedProviderId)) {
return NextResponse.json(
{
code: 'duplicate_display_name',
error: `A credential named "${resolvedDisplayName}" already exists in this workspace. Give this one a different name.`,
},
{ status: 409 }
)
}
const access = await getCredentialActorContext(existingCredential.id, session.user.id, {
workspaceAccess,
})
if (!access.member && !access.isAdmin) {
return NextResponse.json(
{ error: 'A credential with this source already exists in this workspace' },
{ status: 409 }
)
}
const canUpdateExistingCredential = access.isAdmin
const shouldUpdateDisplayName =
type === 'oauth' &&
resolvedDisplayName &&
resolvedDisplayName !== existingCredential.displayName
const shouldUpdateDescription =
typeof description !== 'undefined' &&
(existingCredential.description ?? null) !== resolvedDescription
if (canUpdateExistingCredential && (shouldUpdateDisplayName || shouldUpdateDescription)) {
await db
.update(credential)
.set({
...(shouldUpdateDisplayName ? { displayName: resolvedDisplayName } : {}),
...(shouldUpdateDescription ? { description: resolvedDescription } : {}),
updatedAt: new Date(),
})
.where(eq(credential.id, existingCredential.id))
const [updatedCredential] = await db
.select()
.from(credential)
.where(eq(credential.id, existingCredential.id))
.limit(1)
return NextResponse.json(
{ credential: updatedCredential ?? existingCredential },
{ status: 200 }
)
}
return NextResponse.json({ credential: existingCredential }, { status: 200 })
}
const now = new Date()
// Honor a client-supplied id only for custom Slack bots — the setup modal
// shows the ingest URL `/api/webhooks/slack/custom/{id}` before secrets exist.
const credentialId =
resolvedProviderId === SLACK_CUSTOM_BOT_PROVIDER_ID && clientCredentialId
? clientCredentialId
: generateId()
const creationResult = await db.transaction(async (tx) => {
/**
* Discover the organization lock scope inside this transaction, then
* acquire the same organization → user → membership locks as org
* removal/transfer and re-authorize from the transaction before writing.
*
* If this insert wins, transfer sees the new source-owned personal
* credential and blocks. If transfer wins, its permission/member cleanup
* is visible to the authoritative re-read below and the insert is
* refused.
*/
const plannedContext = await getCredentialCreationWorkspaceContext({
executor: tx,
workspaceId,
userId: session.user.id,
})
if (!plannedContext) {
return { success: false as const, status: 403 as const, error: 'Write permission required' }
}
await acquireOrganizationUserMutationLocks(tx, {
userId: session.user.id,
organizationIds: plannedContext.organizationId ? [plannedContext.organizationId] : [],
})
const currentContext = await getCredentialCreationWorkspaceContext({
executor: tx,
workspaceId,
userId: session.user.id,
forUpdate: true,
})
if (!currentContext) {
return { success: false as const, status: 403 as const, error: 'Write permission required' }
}
if (currentContext.organizationId !== plannedContext.organizationId) {
return {
success: false as const,
status: 409 as const,
error: 'Workspace organization changed while creating the credential. Please retry.',
}
}
if (!currentContext.canWrite) {
return { success: false as const, status: 403 as const, error: 'Write permission required' }
}
// service_account has no DB-level unique index on (workspaceId, providerId,
// displayName), so we re-check inside the tx. OAuth/env_* are guarded by
// partial unique indexes and fall through to the 23505 handler below.
if (type === 'service_account') {
const innerExisting = await findExistingCredentialBySourceTx(tx, {
workspaceId,
type,
displayName: resolvedDisplayName,
providerId: resolvedProviderId,
})
if (innerExisting) throw new DuplicateCredentialError()
}
await tx.insert(credential).values({
id: credentialId,
workspaceId,
type,
displayName: resolvedDisplayName,
description: resolvedDescription,
providerId: resolvedProviderId,
accountId: resolvedAccountId,
envKey: resolvedEnvKey,
envOwnerUserId: resolvedEnvOwnerUserId,
encryptedServiceAccountKey: resolvedEncryptedServiceAccountKey,
createdBy: session.user.id,
createdAt: now,
updatedAt: now,
})
if ((type === 'env_workspace' || type === 'service_account') && currentContext.ownerId) {
if (currentContext.memberUserIds.length > 0) {
for (const memberUserId of currentContext.memberUserIds) {
const isAdmin = memberUserId === session.user.id
await tx.insert(credentialMember).values({
id: generateId(),
credentialId,
userId: memberUserId,
role: isAdmin ? 'admin' : 'member',
status: 'active',
joinedAt: now,
invitedBy: session.user.id,
createdAt: now,
updatedAt: now,
})
}
}
} else {
await tx.insert(credentialMember).values({
id: generateId(),
credentialId,
userId: session.user.id,
role: 'admin',
status: 'active',
joinedAt: now,
invitedBy: session.user.id,
createdAt: now,
updatedAt: now,
})
}
return { success: true as const }
})
if (!creationResult.success) {
return NextResponse.json({ error: creationResult.error }, { status: creationResult.status })
}
const [created] = await db
.select()
.from(credential)
.where(eq(credential.id, credentialId))
.limit(1)
captureServerEvent(
session.user.id,
'credential_connected',
{ credential_type: type, provider_id: resolvedProviderId ?? type, workspace_id: workspaceId },
{
groups: { workspace: workspaceId },
setOnce: { first_credential_connected_at: new Date().toISOString() },
}
)
recordAudit({
workspaceId,
actorId: session.user.id,
actorName: session.user.name,
actorEmail: session.user.email,
action: AuditAction.CREDENTIAL_CREATED,
resourceType: AuditResourceType.CREDENTIAL,
resourceId: credentialId,
resourceName: resolvedDisplayName,
description: `Created ${type} credential "${resolvedDisplayName}"`,
metadata: {
// Provider metadata spreads first so this route's own keys stay
// authoritative and can never be shadowed, matching the update path in
// `lib/credentials/orchestration`.
...extraAuditMetadata,
credentialType: type,
providerId: resolvedProviderId,
},
request,
})
return NextResponse.json({ credential: created }, { status: 201 })
} catch (error: unknown) {
if (error instanceof AtlassianValidationError) {
logger.warn(`[${requestId}] Atlassian credential rejected: ${error.code}`, {
code: error.code,
upstreamStatus: error.status,
...error.logDetail,
})
return NextResponse.json({ code: error.code, error: error.code }, { status: 400 })
}
if (error instanceof TokenServiceAccountValidationError) {
logger.warn(`[${requestId}] Token service-account credential rejected: ${error.code}`, {
code: error.code,
upstreamStatus: error.status,
...error.logDetail,
})
// A provider outage is an infra failure, not a bad request — mirror the
// runtime token route so monitoring sees a 502, not a 400.
const status = error.code === 'provider_unavailable' ? 502 : 400
return NextResponse.json({ code: error.code, error: error.code }, { status })
}
if (error instanceof DuplicateCredentialError) {
return NextResponse.json(
{
code: 'duplicate_display_name',
error: 'A credential with that name already exists in this workspace.',
},
{ status: 409 }
)
}
const pgCode = getPostgresErrorCode(error)
if (pgCode === '23505') {
return NextResponse.json(
{ error: 'A credential with this source already exists' },
{ status: 409 }
)
}
if (pgCode === '23503') {
return NextResponse.json(
{ error: 'Invalid credential reference or membership target' },
{ status: 400 }
)
}
if (pgCode === '23514') {
return NextResponse.json(
{ error: 'Credential source data failed validation checks' },
{ status: 400 }
)
}
const errAsRecord =
typeof error === 'object' && error !== null ? (error as Record<string, unknown>) : {}
logger.error(`[${requestId}] Credential create failure details`, {
code: pgCode,
detail: errAsRecord.detail,
constraint: errAsRecord.constraint,
table: errAsRecord.table,
message: errAsRecord.message,
})
logger.error(`[${requestId}] Failed to create credential`, error)
return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
}
// An existing credential matched the source: an idempotent replay, not a create.
return NextResponse.json(
{ credential: result.credential },
{ status: result.created ? 201 : 200 }
)
})
@@ -438,7 +438,6 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
})
}
// Delete completed/failed jobs older than retention period
const retentionThreshold = new Date(Date.now() - JOB_RETENTION_HOURS * 60 * 60 * 1000)
let asyncJobsDeleted = 0
+4 -2
View File
@@ -3,6 +3,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { verifyCronAuth } from '@/lib/auth/internal'
import { dispatchCleanupJobs } from '@/lib/billing/cleanup-dispatcher'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { cleanupExpiredUploadSessions } from '@/lib/uploads/upload-session/service'
export const dynamic = 'force-dynamic'
@@ -13,11 +14,12 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
const authError = verifyCronAuth(request, 'task cleanup')
if (authError) return authError
const uploadSessions = await cleanupExpiredUploadSessions()
const result = await dispatchCleanupJobs('cleanup-tasks')
logger.info('Task cleanup jobs dispatched', result)
logger.info('Task cleanup jobs dispatched', { ...result, uploadSessions })
return NextResponse.json({ triggered: true, ...result })
return NextResponse.json({ triggered: true, ...result, uploadSessions })
} catch (error) {
logger.error('Failed to dispatch task cleanup jobs:', { error })
return NextResponse.json({ error: 'Failed to dispatch task cleanup' }, { status: 500 })
@@ -21,7 +21,7 @@ vi.mock('@/lib/auth/internal', () => ({
verifyCronAuth: mockVerifyCronAuth,
}))
vi.mock('@/app/api/auth/oauth/utils', () => authOAuthUtilsMock)
vi.mock('@/lib/oauth/credential-service', () => authOAuthUtilsMock)
import { GET } from './route'
@@ -8,9 +8,9 @@ import { verifyCronAuth } from '@/lib/auth/internal'
import { acquireLock, releaseLock } from '@/lib/core/config/redis'
import { runDetached } from '@/lib/core/utils/background'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/lib/oauth/credential-service'
import { deliverableWebhookPredicate } from '@/lib/webhooks/delivery-predicate'
import { getCredentialOwner, getNotificationUrl } from '@/lib/webhooks/provider-subscription-utils'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
const logger = createLogger('TeamsSubscriptionRenewal')
@@ -1,364 +0,0 @@
/**
* @vitest-environment node
*/
import { authMockFns, permissionsMock, permissionsMockFns } from '@sim/testing'
import { NextRequest } from 'next/server'
import { beforeEach, describe, expect, it, vi } from 'vitest'
const {
mockIsUsingCloudStorage,
mockGetStorageProvider,
mockGetStorageConfig,
mockCompleteS3MultipartUpload,
mockCompleteBlobMultipartUpload,
mockDeriveBlobBlockId,
mockVerifyUploadToken,
mockSignUploadToken,
} = vi.hoisted(() => ({
mockIsUsingCloudStorage: vi.fn(),
mockGetStorageProvider: vi.fn(),
mockGetStorageConfig: vi.fn(),
mockCompleteS3MultipartUpload: vi.fn(),
mockCompleteBlobMultipartUpload: vi.fn(),
mockDeriveBlobBlockId: vi.fn(),
mockVerifyUploadToken: vi.fn(),
mockSignUploadToken: vi.fn(),
}))
vi.mock('@/lib/uploads', () => ({
isUsingCloudStorage: mockIsUsingCloudStorage,
getStorageProvider: mockGetStorageProvider,
getStorageConfig: mockGetStorageConfig,
}))
vi.mock('@/lib/uploads/core/upload-token', () => ({
signUploadToken: mockSignUploadToken,
verifyUploadToken: mockVerifyUploadToken,
}))
vi.mock('@/lib/uploads/providers/s3/client', () => ({
completeS3MultipartUpload: mockCompleteS3MultipartUpload,
initiateS3MultipartUpload: mockInitiateS3MultipartUpload,
getS3MultipartPartUrls: vi.fn(),
abortS3MultipartUpload: vi.fn(),
}))
vi.mock('@/lib/uploads/providers/blob/client', () => ({
completeMultipartUpload: mockCompleteBlobMultipartUpload,
deriveBlobBlockId: mockDeriveBlobBlockId,
initiateMultipartUpload: vi.fn(),
getMultipartPartUrls: vi.fn(),
abortMultipartUpload: vi.fn(),
}))
vi.mock('@/lib/uploads/contexts/execution/utils', () => ({
generateUniqueExecutionFileKey: mockGenerateUniqueExecutionFileKey,
}))
vi.mock('@/lib/workspaces/permissions/utils', () => permissionsMock)
const {
mockCheckStorageQuota,
mockGenerateUniqueExecutionFileKey,
mockInitiateS3MultipartUpload,
mockResolveStorageBillingContext,
} = vi.hoisted(() => ({
mockCheckStorageQuota: vi.fn(),
mockGenerateUniqueExecutionFileKey: vi.fn(),
mockInitiateS3MultipartUpload: vi.fn(),
mockResolveStorageBillingContext: vi.fn(),
}))
vi.mock('@/lib/billing/storage', () => ({
checkStorageQuotaForBillingContext: mockCheckStorageQuota,
resolveStorageBillingContext: mockResolveStorageBillingContext,
}))
import { POST } from '@/app/api/files/multipart/route'
const STORAGE_CONTEXT = {
workspaceId: 'ws-1',
billedAccountUserId: 'workspace-owner',
billingEntity: { type: 'organization' as const, id: 'workspace-org' },
plan: 'team_25000',
customStorageLimitGB: null,
}
const tokenPayload = {
uploadId: 'upload-1',
key: 'workspace/ws-1/123-abc-file.bin',
userId: 'user-1',
workspaceId: 'ws-1',
context: 'workspace' as const,
}
const makeRequest = (action: string, body: unknown) =>
new NextRequest(`http://localhost/api/files/multipart?action=${action}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
})
describe('POST /api/files/multipart action=complete', () => {
beforeEach(() => {
vi.clearAllMocks()
authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
permissionsMockFns.mockGetUserEntityPermissions.mockResolvedValue('write')
mockIsUsingCloudStorage.mockReturnValue(true)
mockGetStorageConfig.mockReturnValue({ bucket: 'b', region: 'r' })
mockVerifyUploadToken.mockReturnValue({ valid: true, payload: tokenPayload })
mockSignUploadToken.mockReturnValue('signed-token')
mockCompleteS3MultipartUpload.mockResolvedValue({
location: 'loc',
path: '/api/files/serve/...',
key: tokenPayload.key,
})
mockCompleteBlobMultipartUpload.mockResolvedValue({
location: 'loc',
path: '/api/files/serve/...',
key: tokenPayload.key,
})
mockDeriveBlobBlockId.mockImplementation(
(n: number) => `block-${n.toString().padStart(6, '0')}`
)
})
it('rejects parts without partNumber', async () => {
mockGetStorageProvider.mockReturnValue('s3')
const res = await POST(
makeRequest('complete', {
uploadToken: 'tok',
parts: [{ etag: 'abc' }],
})
)
expect(res.status).toBe(400)
expect(mockCompleteS3MultipartUpload).not.toHaveBeenCalled()
})
it('S3 path requires etag and forwards { ETag, PartNumber }', async () => {
mockGetStorageProvider.mockReturnValue('s3')
const missingEtag = await POST(
makeRequest('complete', {
uploadToken: 'tok',
parts: [{ partNumber: 1 }],
})
)
expect(missingEtag.status).toBe(500)
mockCompleteS3MultipartUpload.mockClear()
const ok = await POST(
makeRequest('complete', {
uploadToken: 'tok',
parts: [
{ partNumber: 1, etag: 'aaa' },
{ partNumber: 2, etag: 'bbb' },
],
})
)
expect(ok.status).toBe(200)
expect(mockCompleteS3MultipartUpload).toHaveBeenCalledWith(
tokenPayload.key,
tokenPayload.uploadId,
[
{ ETag: 'aaa', PartNumber: 1 },
{ ETag: 'bbb', PartNumber: 2 },
],
expect.any(Object)
)
})
it('Blob path derives blockId from partNumber and ignores etag', async () => {
mockGetStorageProvider.mockReturnValue('blob')
mockGetStorageConfig.mockReturnValue({
containerName: 'c',
accountName: 'a',
accountKey: 'k',
})
const res = await POST(
makeRequest('complete', {
uploadToken: 'tok',
parts: [{ partNumber: 1, etag: 'irrelevant' }, { partNumber: 2 }],
})
)
expect(res.status).toBe(200)
expect(mockDeriveBlobBlockId).toHaveBeenCalledWith(1)
expect(mockDeriveBlobBlockId).toHaveBeenCalledWith(2)
expect(mockCompleteBlobMultipartUpload).toHaveBeenCalledWith(
tokenPayload.key,
tokenPayload.uploadId,
[
{ partNumber: 1, blockId: 'block-000001' },
{ partNumber: 2, blockId: 'block-000002' },
],
expect.objectContaining({ containerName: 'c' })
)
})
it('returns 403 when token is invalid', async () => {
mockGetStorageProvider.mockReturnValue('s3')
mockVerifyUploadToken.mockReturnValueOnce({ valid: false })
const res = await POST(
makeRequest('complete', {
uploadToken: 'bad',
parts: [{ partNumber: 1, etag: 'a' }],
})
)
expect(res.status).toBe(403)
})
it('batch complete normalizes per upload', async () => {
mockGetStorageProvider.mockReturnValue('s3')
const res = await POST(
makeRequest('complete', {
uploads: [
{
uploadToken: 'tok-a',
parts: [{ partNumber: 1, etag: 'aaa' }],
},
{
uploadToken: 'tok-b',
parts: [{ partNumber: 1, etag: 'bbb' }],
},
],
})
)
expect(res.status).toBe(200)
expect(mockCompleteS3MultipartUpload).toHaveBeenCalledTimes(2)
})
})
describe('POST /api/files/multipart action=initiate quota enforcement', () => {
const makeInitiateRequest = (body: unknown) =>
new NextRequest('http://localhost/api/files/multipart?action=initiate', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
})
beforeEach(() => {
vi.clearAllMocks()
authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
permissionsMockFns.mockGetUserEntityPermissions.mockResolvedValue('write')
mockIsUsingCloudStorage.mockReturnValue(true)
mockGetStorageProvider.mockReturnValue('s3')
mockGetStorageConfig.mockReturnValue({ bucket: 'b', region: 'r' })
mockSignUploadToken.mockReturnValue('signed-token')
mockResolveStorageBillingContext.mockResolvedValue(STORAGE_CONTEXT)
mockCheckStorageQuota.mockResolvedValue({ allowed: true })
mockInitiateS3MultipartUpload.mockResolvedValue({ uploadId: 'up-1', key: 'k/file.bin' })
mockGenerateUniqueExecutionFileKey.mockImplementation(
(
context: { workspaceId: string; workflowId: string; executionId: string },
fileName: string
) =>
`execution/${context.workspaceId}/${context.workflowId}/${context.executionId}/unique-${fileName}`
)
})
it('blocks upload when fileSize: 0 exceeds quota', async () => {
mockCheckStorageQuota.mockResolvedValue({ allowed: false, error: 'Storage limit exceeded' })
const res = await makeInitiateRequest({
fileName: 'file.bin',
contentType: 'application/octet-stream',
fileSize: 0,
workspaceId: 'ws-1',
context: 'knowledge-base',
})
const response = await POST(res)
expect(response.status).toBe(413)
const body = await response.json()
expect(body.error).toContain('Storage limit exceeded')
})
it('allows quota-enforced contexts that pass the quota check', async () => {
const res = await makeInitiateRequest({
fileName: 'doc.pdf',
contentType: 'application/pdf',
fileSize: 99999,
workspaceId: 'ws-1',
context: 'knowledge-base',
})
const response = await POST(res)
expect(response.status).toBe(200)
expect(mockResolveStorageBillingContext).toHaveBeenCalledWith('ws-1')
expect(mockCheckStorageQuota).toHaveBeenCalledWith(STORAGE_CONTEXT, 99999)
expect(mockInitiateS3MultipartUpload).toHaveBeenCalled()
})
it('keeps mothership chat uploads outside workspace storage quotas', async () => {
mockCheckStorageQuota.mockResolvedValue({ allowed: false, error: 'Storage limit exceeded' })
const res = await makeInitiateRequest({
fileName: 'conversation.bin',
contentType: 'application/octet-stream',
fileSize: 99999,
workspaceId: 'ws-1',
context: 'mothership',
})
const response = await POST(res)
expect(response.status).toBe(200)
expect(mockResolveStorageBillingContext).not.toHaveBeenCalled()
expect(mockCheckStorageQuota).not.toHaveBeenCalled()
expect(mockInitiateS3MultipartUpload).toHaveBeenCalled()
})
it('allocates distinct multipart keys for duplicate execution attachment names', async () => {
mockGenerateUniqueExecutionFileKey
.mockReturnValueOnce('execution/ws-1/wf-1/exec-1/one-output.bin')
.mockReturnValueOnce('execution/ws-1/wf-1/exec-1/two-output.bin')
mockInitiateS3MultipartUpload.mockImplementation(async ({ customKey }) => ({
uploadId: `upload-${customKey}`,
key: customKey,
}))
const makeExecutionRequest = (fileSize: number) =>
makeInitiateRequest({
fileName: 'output.bin',
contentType: 'application/octet-stream',
fileSize,
workspaceId: 'ws-1',
workflowId: 'wf-1',
executionId: 'exec-1',
context: 'execution',
})
const first = await POST(makeExecutionRequest(60 * 1024 * 1024))
const second = await POST(makeExecutionRequest(70 * 1024 * 1024))
const firstBody = await first.json()
const secondBody = await second.json()
expect(first.status).toBe(200)
expect(second.status).toBe(200)
expect(firstBody.key).toBe('execution/ws-1/wf-1/exec-1/one-output.bin')
expect(secondBody.key).toBe('execution/ws-1/wf-1/exec-1/two-output.bin')
expect(firstBody.key).not.toBe(secondBody.key)
})
it.each(['og-images', 'profile-pictures', 'workspace-logos', 'logs'])(
'rejects quota-exempt context %s — not allowed via the multipart endpoint',
async (context) => {
const res = await makeInitiateRequest({
fileName: 'asset.png',
contentType: 'image/png',
fileSize: 100 * 1024 * 1024 * 1024,
workspaceId: 'ws-1',
context,
})
const response = await POST(res)
expect(response.status).toBe(400)
const body = await response.json()
expect(body.error).toMatch(/invalid storage context/i)
expect(mockCheckStorageQuota).not.toHaveBeenCalled()
expect(mockInitiateS3MultipartUpload).not.toHaveBeenCalled()
}
)
})
-541
View File
@@ -1,541 +0,0 @@
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { type NextRequest, NextResponse } from 'next/server'
import {
abortMultipartUploadContract,
type CompleteMultipartBody,
completeMultipartUploadContract,
getMultipartPartUrlsContract,
initiateMultipartUploadContract,
multipartActionSchema,
} from '@/lib/api/contracts/storage-transfer'
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { getSession } from '@/lib/auth'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import {
getStorageConfig,
getStorageProvider,
isUsingCloudStorage,
type StorageContext,
} from '@/lib/uploads'
import { deleteFile } from '@/lib/uploads/core/storage-service'
import {
signUploadToken,
type UploadTokenPayload,
verifyUploadToken,
} from '@/lib/uploads/core/upload-token'
import { recordKnowledgeBaseFileOwnership } from '@/lib/uploads/server/metadata'
import { QUOTA_EXEMPT_STORAGE_CONTEXTS, type StorageConfig } from '@/lib/uploads/shared/types'
import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
const logger = createLogger('MultipartUploadAPI')
/**
* Contexts the multipart endpoint accepts. Small public assets and internal logs
* are excluded because they have no large-file flow. Mothership remains
* available for large chat attachments but is quota-exempt because chat uploads
* do not count as durable workspace-file storage. Every other accepted context
* is quota-enforced below.
*/
const ALLOWED_UPLOAD_CONTEXTS = new Set<StorageContext>([
'knowledge-base',
'chat',
'copilot',
'mothership',
'execution',
'workspace',
])
/**
* Unified part identity sent by the client when completing a multipart upload.
* `etag` is required for S3 and GCS (CompleteMultipartUpload). For Azure the
* server derives the block id from `partNumber` via {@link deriveBlobBlockId}.
*/
interface ClientCompletedPart {
partNumber: number
etag?: string
}
const isClientCompletedParts = (value: unknown): value is ClientCompletedPart[] =>
Array.isArray(value) &&
value.every(
(p) =>
p !== null &&
typeof p === 'object' &&
typeof (p as ClientCompletedPart).partNumber === 'number' &&
((p as ClientCompletedPart).etag === undefined ||
typeof (p as ClientCompletedPart).etag === 'string')
)
const buildS3CustomConfig = (config: StorageConfig) =>
config.bucket && config.region ? { bucket: config.bucket, region: config.region } : undefined
const buildBlobCustomConfig = (config: StorageConfig) => ({
containerName: config.containerName!,
accountName: config.accountName!,
accountKey: config.accountKey,
connectionString: config.connectionString,
})
const buildGcsCustomConfig = (config: StorageConfig) =>
config.bucket ? { bucket: config.bucket } : undefined
const verifyTokenForUser = (token: string | undefined, userId: string) => {
if (!token || typeof token !== 'string') {
return null
}
const result = verifyUploadToken(token)
if (!result.valid || result.payload.userId !== userId) {
return null
}
return result.payload
}
/**
* Record a trusted storage-key -> workspace ownership binding for completed
* knowledge-base uploads. KB file authorization resolves the owning workspace
* from this binding, so every KB object must have one. No-op for other contexts.
*/
const recordKnowledgeBaseOwnership = async (
payload: UploadTokenPayload,
key: string
): Promise<void> => {
if (payload.context !== 'knowledge-base' || !payload.workspaceId) {
return
}
await recordKnowledgeBaseFileOwnership({
key,
userId: payload.userId,
workspaceId: payload.workspaceId,
originalName: payload.fileName ?? key.split('/').pop() ?? key,
contentType: payload.contentType ?? 'application/octet-stream',
size: typeof payload.fileSize === 'number' ? payload.fileSize : 0,
})
}
export const POST = withRouteHandler(async (request: NextRequest) => {
try {
const session = await getSession()
if (!session?.user?.id) {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const userId = session.user.id
const actionParam = request.nextUrl.searchParams.get('action')
const actionResult = multipartActionSchema.safeParse(actionParam)
const action = actionResult.success ? actionResult.data : null
if (!isUsingCloudStorage()) {
return NextResponse.json(
{
error:
'Multipart upload is only available with cloud storage (S3, Azure Blob, or Google Cloud Storage)',
},
{ status: 400 }
)
}
const storageProvider = getStorageProvider()
switch (action) {
case 'initiate': {
const parsed = await parseRequest(
initiateMultipartUploadContract,
request,
{},
{
validationErrorResponse: (error) =>
NextResponse.json({ error: getValidationErrorMessage(error) }, { status: 400 }),
}
)
if (!parsed.success) return parsed.response
const data = parsed.data.body
const { fileName, contentType, fileSize, workspaceId, context = 'knowledge-base' } = data
if (!workspaceId || typeof workspaceId !== 'string') {
return NextResponse.json({ error: 'workspaceId is required' }, { status: 400 })
}
if (!ALLOWED_UPLOAD_CONTEXTS.has(context as StorageContext)) {
return NextResponse.json({ error: 'Invalid storage context' }, { status: 400 })
}
const storageContext = context as StorageContext
const permission = await getUserEntityPermissions(userId, 'workspace', workspaceId)
if (permission !== 'write' && permission !== 'admin') {
return NextResponse.json({ error: 'Forbidden' }, { status: 403 })
}
const config = getStorageConfig(storageContext)
if (!QUOTA_EXEMPT_STORAGE_CONTEXTS.has(storageContext)) {
const { checkStorageQuotaForBillingContext, resolveStorageBillingContext } = await import(
'@/lib/billing/storage'
)
const storageBillingContext = await resolveStorageBillingContext(workspaceId)
const quotaCheck = await checkStorageQuotaForBillingContext(
storageBillingContext,
fileSize ?? 0
)
if (!quotaCheck.allowed) {
return NextResponse.json(
{ error: quotaCheck.error || 'Storage limit exceeded' },
{ status: 413 }
)
}
}
let customKey: string | undefined
if (context === 'workspace' || context === 'mothership') {
const { MAX_WORKSPACE_FILE_SIZE } = await import('@/lib/uploads/shared/types')
if (typeof fileSize === 'number' && fileSize > MAX_WORKSPACE_FILE_SIZE) {
return NextResponse.json(
{ error: `File size exceeds maximum of ${MAX_WORKSPACE_FILE_SIZE} bytes` },
{ status: 413 }
)
}
const { generateWorkspaceFileKey } = await import(
'@/lib/uploads/contexts/workspace/workspace-file-manager'
)
customKey = generateWorkspaceFileKey(workspaceId, fileName)
} else if (context === 'execution') {
const workflowId = (data as { workflowId?: unknown }).workflowId
const executionId = (data as { executionId?: unknown }).executionId
if (typeof workflowId !== 'string' || !workflowId.trim()) {
return NextResponse.json(
{ error: 'workflowId is required for execution uploads' },
{ status: 400 }
)
}
if (typeof executionId !== 'string' || !executionId.trim()) {
return NextResponse.json(
{ error: 'executionId is required for execution uploads' },
{ status: 400 }
)
}
const { generateUniqueExecutionFileKey } = await import(
'@/lib/uploads/contexts/execution/utils'
)
customKey = generateUniqueExecutionFileKey(
{ workspaceId, workflowId, executionId },
fileName
)
}
let uploadId: string
let key: string
if (storageProvider === 's3') {
const { initiateS3MultipartUpload } = await import('@/lib/uploads/providers/s3/client')
const result = await initiateS3MultipartUpload({
fileName,
contentType,
fileSize,
customConfig: buildS3CustomConfig(config),
customKey,
purpose: context,
})
uploadId = result.uploadId
key = result.key
} else if (storageProvider === 'blob') {
const { initiateMultipartUpload } = await import('@/lib/uploads/providers/blob/client')
const result = await initiateMultipartUpload({
fileName,
contentType,
fileSize,
customConfig: buildBlobCustomConfig(config),
customKey,
})
uploadId = result.uploadId
key = result.key
} else if (storageProvider === 'gcs') {
const { initiateGcsMultipartUpload } = await import('@/lib/uploads/providers/gcs/client')
const result = await initiateGcsMultipartUpload({
fileName,
contentType,
fileSize,
customConfig: buildGcsCustomConfig(config),
customKey,
purpose: context,
})
uploadId = result.uploadId
key = result.key
} else {
return NextResponse.json(
{ error: `Unsupported storage provider: ${storageProvider}` },
{ status: 400 }
)
}
const uploadToken = signUploadToken({
uploadId,
key,
userId,
workspaceId,
context: storageContext,
fileName,
contentType,
...(typeof fileSize === 'number' ? { fileSize } : {}),
})
logger.info(
`Initiated ${storageProvider} multipart upload for ${fileName} (context: ${storageContext}, workspace: ${workspaceId}): ${uploadId}`
)
return NextResponse.json({ uploadId, key, uploadToken })
}
case 'get-part-urls': {
const parsed = await parseRequest(
getMultipartPartUrlsContract,
request,
{},
{
validationErrorResponse: (error) =>
NextResponse.json({ error: getValidationErrorMessage(error) }, { status: 400 }),
}
)
if (!parsed.success) return parsed.response
const data = parsed.data.body
const { partNumbers } = data
const tokenPayload = verifyTokenForUser(data.uploadToken, userId)
if (!tokenPayload) {
return NextResponse.json({ error: 'Invalid or expired upload token' }, { status: 403 })
}
const { uploadId, key, context } = tokenPayload
const config = getStorageConfig(context)
if (storageProvider === 's3') {
const { getS3MultipartPartUrls } = await import('@/lib/uploads/providers/s3/client')
const presignedUrls = await getS3MultipartPartUrls(
key,
uploadId,
partNumbers,
buildS3CustomConfig(config)
)
return NextResponse.json({ presignedUrls })
}
if (storageProvider === 'blob') {
const { getMultipartPartUrls } = await import('@/lib/uploads/providers/blob/client')
const presignedUrls = await getMultipartPartUrls(
key,
partNumbers,
buildBlobCustomConfig(config)
)
return NextResponse.json({ presignedUrls })
}
if (storageProvider === 'gcs') {
const { getGcsMultipartPartUrls } = await import('@/lib/uploads/providers/gcs/client')
const presignedUrls = await getGcsMultipartPartUrls(
key,
uploadId,
partNumbers,
buildGcsCustomConfig(config)
)
return NextResponse.json({ presignedUrls })
}
return NextResponse.json(
{ error: `Unsupported storage provider: ${storageProvider}` },
{ status: 400 }
)
}
case 'complete': {
const parsed = await parseRequest(
completeMultipartUploadContract,
request,
{},
{
validationErrorResponse: (error) =>
NextResponse.json({ error: getValidationErrorMessage(error) }, { status: 400 }),
}
)
if (!parsed.success) return parsed.response
const data: CompleteMultipartBody = parsed.data.body
const s3Module =
storageProvider === 's3' ? await import('@/lib/uploads/providers/s3/client') : null
const blobModule =
storageProvider === 'blob' ? await import('@/lib/uploads/providers/blob/client') : null
const gcsModule =
storageProvider === 'gcs' ? await import('@/lib/uploads/providers/gcs/client') : null
const completeOne = async (payload: UploadTokenPayload, parts: ClientCompletedPart[]) => {
const { uploadId, key, context } = payload
const config = getStorageConfig(context)
let completed: { location: string; path: string; key: string }
if (storageProvider === 's3' && s3Module) {
const { completeS3MultipartUpload } = s3Module
const s3Parts = parts.map((p) => {
if (!p.etag) {
throw new Error(`Missing etag for S3 part ${p.partNumber}`)
}
return { ETag: p.etag, PartNumber: p.partNumber }
})
completed = await completeS3MultipartUpload(
key,
uploadId,
s3Parts,
buildS3CustomConfig(config)
)
} else if (storageProvider === 'blob' && blobModule) {
const { completeMultipartUpload, deriveBlobBlockId } = blobModule
const blobParts = parts.map((p) => ({
partNumber: p.partNumber,
blockId: deriveBlobBlockId(p.partNumber),
}))
completed = await completeMultipartUpload(
key,
uploadId,
blobParts,
buildBlobCustomConfig(config)
)
} else if (storageProvider === 'gcs' && gcsModule) {
const { completeGcsMultipartUpload } = gcsModule
const gcsParts = parts.map((p) => {
if (!p.etag) {
throw new Error(`Missing etag for GCS part ${p.partNumber}`)
}
return { ETag: p.etag, PartNumber: p.partNumber }
})
completed = await completeGcsMultipartUpload(
key,
uploadId,
gcsParts,
buildGcsCustomConfig(config)
)
} else {
throw new Error(`Unsupported storage provider: ${storageProvider}`)
}
try {
await recordKnowledgeBaseOwnership(payload, completed.key)
} catch (error) {
// The object is committed, but without an ownership binding a KB file
// is unreadable and undeletable via the KB paths. Remove the orphan
// best-effort and surface a retryable error so the client re-uploads.
if (payload.context === 'knowledge-base') {
await deleteFile({ key: completed.key, context: 'knowledge-base' }).catch(() => {})
}
throw error
}
return {
success: true as const,
location: completed.location,
path: completed.path,
key: completed.key,
}
}
if ('uploads' in data && Array.isArray(data.uploads)) {
const verified: Array<{ payload: UploadTokenPayload; parts: ClientCompletedPart[] }> = []
for (const upload of data.uploads) {
const payload = verifyTokenForUser(upload.uploadToken, userId)
if (!payload) {
return NextResponse.json(
{ error: 'Invalid or expired upload token' },
{ status: 403 }
)
}
if (!isClientCompletedParts(upload.parts)) {
return NextResponse.json(
{ error: 'Invalid parts payload: expected [{ partNumber, etag? }]' },
{ status: 400 }
)
}
verified.push({ payload, parts: upload.parts })
}
const results = await Promise.all(
verified.map(({ payload, parts }) => completeOne(payload, parts))
)
logger.info(`Completed ${verified.length} multipart uploads`)
return NextResponse.json({ results })
}
const single = data
const tokenPayload = verifyTokenForUser(single.uploadToken, userId)
if (!tokenPayload) {
return NextResponse.json({ error: 'Invalid or expired upload token' }, { status: 403 })
}
if (!isClientCompletedParts(single.parts)) {
return NextResponse.json(
{ error: 'Invalid parts payload: expected [{ partNumber, etag? }]' },
{ status: 400 }
)
}
const result = await completeOne(tokenPayload, single.parts)
logger.info(
`Completed ${storageProvider} multipart upload for key ${tokenPayload.key} (context: ${tokenPayload.context})`
)
return NextResponse.json(result)
}
case 'abort': {
const parsed = await parseRequest(
abortMultipartUploadContract,
request,
{},
{
validationErrorResponse: (error) =>
NextResponse.json({ error: getValidationErrorMessage(error) }, { status: 400 }),
}
)
if (!parsed.success) return parsed.response
const data = parsed.data.body
const tokenPayload = verifyTokenForUser(data.uploadToken, userId)
if (!tokenPayload) {
return NextResponse.json({ error: 'Invalid or expired upload token' }, { status: 403 })
}
const { uploadId, key, context } = tokenPayload
const config = getStorageConfig(context)
if (storageProvider === 's3') {
const { abortS3MultipartUpload } = await import('@/lib/uploads/providers/s3/client')
await abortS3MultipartUpload(key, uploadId, buildS3CustomConfig(config))
logger.info(`Aborted S3 multipart upload for key ${key} (context: ${context})`)
} else if (storageProvider === 'blob') {
const { abortMultipartUpload } = await import('@/lib/uploads/providers/blob/client')
await abortMultipartUpload(key, uploadId, buildBlobCustomConfig(config))
logger.info(`Aborted Azure multipart upload for key ${key} (context: ${context})`)
} else if (storageProvider === 'gcs') {
const { abortGcsMultipartUpload } = await import('@/lib/uploads/providers/gcs/client')
await abortGcsMultipartUpload(key, uploadId, buildGcsCustomConfig(config))
logger.info(`Aborted GCS multipart upload for key ${key} (context: ${context})`)
} else {
return NextResponse.json(
{ error: `Unsupported storage provider: ${storageProvider}` },
{ status: 400 }
)
}
return NextResponse.json({ success: true })
}
default:
return NextResponse.json(
{ error: 'Invalid action. Use: initiate, get-part-urls, complete, or abort' },
{ status: 400 }
)
}
} catch (error) {
logger.error('Multipart upload error:', error)
return NextResponse.json(
{ error: getErrorMessage(error, 'Multipart upload failed') },
{ status: 500 }
)
}
})
@@ -1,208 +0,0 @@
/**
* Tests for the batch presigned upload API route
*
* @vitest-environment node
*/
import { authMockFns, storageServiceMock, storageServiceMockFns } from '@sim/testing'
import { NextRequest } from 'next/server'
import { beforeEach, describe, expect, it, vi } from 'vitest'
const {
mockValidateFileType,
mockGetUserEntityPermissions,
mockRecordKnowledgeBaseFileOwnershipMany,
mockSignUploadToken,
} = vi.hoisted(() => ({
mockValidateFileType: vi.fn().mockReturnValue(null),
mockGetUserEntityPermissions: vi.fn().mockResolvedValue('write'),
mockRecordKnowledgeBaseFileOwnershipMany: vi.fn().mockResolvedValue(undefined),
mockSignUploadToken: vi.fn(),
}))
vi.mock('@/lib/uploads/config', () => ({
getServeStoragePrefix: () => 's3',
}))
vi.mock('@/lib/uploads/core/storage-service', () => storageServiceMock)
vi.mock('@/lib/uploads/core/upload-token', () => ({
signUploadToken: mockSignUploadToken,
}))
vi.mock('@/lib/uploads/utils/validation', () => ({
validateFileType: mockValidateFileType,
SUPPORTED_ARCHIVE_EXTENSIONS: ['zip'] as const,
}))
vi.mock('@/lib/workspaces/permissions/utils', () => ({
getUserEntityPermissions: mockGetUserEntityPermissions,
}))
vi.mock('@/lib/uploads/server/metadata', () => ({
recordKnowledgeBaseFileOwnershipMany: mockRecordKnowledgeBaseFileOwnershipMany,
}))
import { POST } from '@/app/api/files/presigned/batch/route'
const KB_QUERY = 'type=knowledge-base&workspaceId=ws-1'
const buildRequest = (query: string, files?: unknown) =>
new NextRequest(`http://localhost:3000/api/files/presigned/batch?${query}`, {
method: 'POST',
body: JSON.stringify({
files: files ?? [{ fileName: 'doc.pdf', contentType: 'application/pdf', fileSize: 1024 }],
}),
})
describe('/api/files/presigned/batch', () => {
beforeEach(() => {
vi.clearAllMocks()
authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
mockValidateFileType.mockReturnValue(null)
mockGetUserEntityPermissions.mockResolvedValue('write')
mockRecordKnowledgeBaseFileOwnershipMany.mockResolvedValue(undefined)
mockSignUploadToken.mockReturnValue('signed-receipt-token')
storageServiceMockFns.mockHasCloudStorage.mockReturnValue(true)
storageServiceMockFns.mockGenerateBatchPresignedUploadUrls.mockImplementation(
async (files: Array<{ fileName: string }>, context: string) =>
files.map((file) => ({
url: `https://example.com/${context}/${file.fileName}`,
key: `${context}/${file.fileName}`,
uploadId: `receipt-${file.fileName}`,
}))
)
})
it('returns 401 when the caller has no session', async () => {
authMockFns.mockGetSession.mockResolvedValue(null)
const response = await POST(buildRequest(KB_QUERY))
expect(response.status).toBe(401)
expect(storageServiceMockFns.mockGenerateBatchPresignedUploadUrls).not.toHaveBeenCalled()
})
it.each([
'workspace-logos',
'profile-pictures',
'execution',
'mothership',
'chat',
'copilot',
'workspace',
])('refuses to presign the %s context', async (type) => {
const response = await POST(buildRequest(`type=${type}&workspaceId=ws-1`))
const data = await response.json()
expect(response.status).toBe(400)
expect(data.error).toContain('Invalid type parameter')
expect(storageServiceMockFns.mockGenerateBatchPresignedUploadUrls).not.toHaveBeenCalled()
})
it('returns 400 when type is missing', async () => {
const response = await POST(buildRequest('workspaceId=ws-1'))
expect(response.status).toBe(400)
expect(storageServiceMockFns.mockGenerateBatchPresignedUploadUrls).not.toHaveBeenCalled()
})
it('returns 400 when workspaceId is missing', async () => {
const response = await POST(buildRequest('type=knowledge-base'))
const data = await response.json()
expect(response.status).toBe(400)
expect(data.error).toContain('workspaceId')
expect(mockGetUserEntityPermissions).not.toHaveBeenCalled()
expect(storageServiceMockFns.mockGenerateBatchPresignedUploadUrls).not.toHaveBeenCalled()
})
it.each([['read'], [null]])(
'returns 403 when the caller has %s access to the workspace',
async (permission) => {
mockGetUserEntityPermissions.mockResolvedValue(permission)
const response = await POST(buildRequest(KB_QUERY))
expect(response.status).toBe(403)
expect(storageServiceMockFns.mockGenerateBatchPresignedUploadUrls).not.toHaveBeenCalled()
}
)
it('authorizes the workspace before returning the local-storage fallback', async () => {
storageServiceMockFns.mockHasCloudStorage.mockReturnValue(false)
mockGetUserEntityPermissions.mockResolvedValue('read')
const response = await POST(buildRequest(KB_QUERY))
expect(response.status).toBe(403)
})
it('rejects unsupported file types before minting any URL', async () => {
mockValidateFileType.mockReturnValue({
code: 'UNSUPPORTED_FILE_TYPE',
message: 'Unsupported file type: html.',
supportedTypes: ['pdf'],
})
const response = await POST(
buildRequest(KB_QUERY, [{ fileName: 'poc.html', contentType: 'text/html', fileSize: 41 }])
)
const data = await response.json()
expect(response.status).toBe(400)
expect(data.code).toBe('UNSUPPORTED_FILE_TYPE')
expect(storageServiceMockFns.mockGenerateBatchPresignedUploadUrls).not.toHaveBeenCalled()
})
it('mints knowledge-base URLs and records workspace ownership for a permitted caller', async () => {
const response = await POST(buildRequest(KB_QUERY))
const data = await response.json()
expect(response.status).toBe(200)
expect(mockGetUserEntityPermissions).toHaveBeenCalledWith('user-1', 'workspace', 'ws-1')
expect(storageServiceMockFns.mockGenerateBatchPresignedUploadUrls).toHaveBeenCalledWith(
[{ fileName: 'doc.pdf', contentType: 'application/pdf', fileSize: 1024 }],
'knowledge-base',
'user-1',
3600
)
expect(data.files).toHaveLength(1)
expect(data.files[0].fileInfo.key).toBe('knowledge-base/doc.pdf')
expect(data.files[0].fileInfo.path).toContain('?context=knowledge-base')
expect(data.files[0].uploadToken).toBe('signed-receipt-token')
expect(mockSignUploadToken).toHaveBeenCalledWith(
expect.objectContaining({
uploadId: 'receipt-doc.pdf',
key: 'knowledge-base/doc.pdf',
userId: 'user-1',
workspaceId: 'ws-1',
context: 'knowledge-base',
uploadKind: 'direct',
})
)
expect(data.directUploadSupported).toBe(true)
expect(mockRecordKnowledgeBaseFileOwnershipMany).toHaveBeenCalledWith([
{
key: 'knowledge-base/doc.pdf',
userId: 'user-1',
workspaceId: 'ws-1',
originalName: 'doc.pdf',
contentType: 'application/pdf',
size: 1024,
},
])
})
it('returns the fallback response when cloud storage is not configured', async () => {
storageServiceMockFns.mockHasCloudStorage.mockReturnValue(false)
const response = await POST(buildRequest(KB_QUERY))
const data = await response.json()
expect(response.status).toBe(200)
expect(data.directUploadSupported).toBe(false)
expect(data.files[0].presignedUrl).toBe('')
expect(storageServiceMockFns.mockGenerateBatchPresignedUploadUrls).not.toHaveBeenCalled()
})
})
@@ -1,200 +0,0 @@
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
import {
batchPresignedUploadBodyContract,
batchPresignedUploadTypeSchema,
batchPresignedUploadTypes,
} from '@/lib/api/contracts/storage-transfer'
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { getSession } from '@/lib/auth'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { getServeStoragePrefix } from '@/lib/uploads/config'
import {
generateBatchPresignedUploadUrls,
hasCloudStorage,
} from '@/lib/uploads/core/storage-service'
import { signUploadToken } from '@/lib/uploads/core/upload-token'
import { recordKnowledgeBaseFileOwnershipMany } from '@/lib/uploads/server/metadata'
import { validateFileType } from '@/lib/uploads/utils/validation'
import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
import { createErrorResponse } from '@/app/api/files/utils'
const logger = createLogger('BatchPresignedUploadAPI')
/**
* Mints presigned upload URLs for knowledge-base ingest, the only context this
* endpoint can authorize. Every request must name a workspace the caller has
* write access to; other storage contexts are rejected rather than presigned,
* because a presigned PUT is a write grant into a bucket served from a trusted
* origin.
*/
export const POST = withRouteHandler(async (request: NextRequest) => {
try {
const session = await getSession()
if (!session?.user?.id) {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const parsed = await parseRequest(
batchPresignedUploadBodyContract,
request,
{},
{
validationErrorResponse: (error) =>
NextResponse.json(
{ error: getValidationErrorMessage(error, 'Invalid request data') },
{ status: 400 }
),
invalidJsonResponse: () =>
NextResponse.json({ error: 'Invalid JSON in request body' }, { status: 400 }),
}
)
if (!parsed.success) return parsed.response
const { files } = parsed.data.body
const uploadTypeParam = request.nextUrl.searchParams.get('type')
if (!uploadTypeParam) {
return NextResponse.json({ error: 'type query parameter is required' }, { status: 400 })
}
const uploadTypeResult = batchPresignedUploadTypeSchema.safeParse(uploadTypeParam)
if (!uploadTypeResult.success) {
return NextResponse.json(
{
error: `Invalid type parameter. Must be one of: ${batchPresignedUploadTypes.join(', ')}`,
},
{ status: 400 }
)
}
const uploadType = uploadTypeResult.data
const sessionUserId = session.user.id
for (const file of files) {
const fileValidationError = validateFileType(file.fileName, file.contentType)
if (fileValidationError) {
return NextResponse.json(
{
error: fileValidationError.message,
code: fileValidationError.code,
supportedTypes: fileValidationError.supportedTypes,
},
{ status: 400 }
)
}
}
const workspaceId = request.nextUrl.searchParams.get('workspaceId')
if (!workspaceId?.trim()) {
return NextResponse.json(
{ error: 'workspaceId query parameter is required for knowledge-base uploads' },
{ status: 400 }
)
}
const permission = await getUserEntityPermissions(sessionUserId, 'workspace', workspaceId)
if (permission !== 'write' && permission !== 'admin') {
return NextResponse.json(
{ error: 'Write or Admin access required for knowledge-base uploads' },
{ status: 403 }
)
}
if (!hasCloudStorage()) {
logger.info(
`Local storage detected - batch presigned URLs not available, client will use API fallback`
)
return NextResponse.json({
files: files.map((file) => ({
fileName: file.fileName,
presignedUrl: '', // Empty URL signals fallback to API upload
fileInfo: {
path: '',
key: '',
name: file.fileName,
size: file.fileSize,
type: file.contentType,
},
directUploadSupported: false,
})),
directUploadSupported: false,
})
}
logger.info(`Generating batch ${uploadType} presigned URLs for ${files.length} files`)
const startTime = Date.now()
const presignedUrls = await generateBatchPresignedUploadUrls(
files.map((file) => ({
fileName: file.fileName,
contentType: file.contentType,
fileSize: file.fileSize,
})),
uploadType,
sessionUserId,
3600 // 1 hour
)
const duration = Date.now() - startTime
logger.info(
`Generated ${files.length} presigned URLs in ${duration}ms (avg ${Math.round(duration / files.length)}ms per file)`
)
await recordKnowledgeBaseFileOwnershipMany(
presignedUrls.map((urlResponse, index) => ({
key: urlResponse.key,
userId: sessionUserId,
workspaceId,
originalName: files[index].fileName,
contentType: files[index].contentType,
size: files[index].fileSize,
}))
)
const storagePrefix = getServeStoragePrefix()
return NextResponse.json({
files: presignedUrls.map((urlResponse, index) => {
const finalPath = `/api/files/serve/${storagePrefix}/${encodeURIComponent(urlResponse.key)}?context=${uploadType}`
const file = files[index]
return {
fileName: file.fileName,
presignedUrl: urlResponse.url,
fileInfo: {
path: finalPath,
key: urlResponse.key,
name: file.fileName,
size: file.fileSize,
type: file.contentType,
},
uploadHeaders: urlResponse.uploadHeaders,
...(urlResponse.uploadId
? {
uploadToken: signUploadToken({
uploadId: urlResponse.uploadId,
key: urlResponse.key,
userId: sessionUserId,
workspaceId,
context: 'knowledge-base',
fileName: file.fileName,
contentType: file.contentType,
fileSize: file.fileSize,
uploadKind: 'direct',
}),
}
: {}),
directUploadSupported: true,
}
}),
directUploadSupported: true,
})
} catch (error) {
logger.error('Error generating batch presigned URLs:', error)
return createErrorResponse(
error instanceof Error ? error : new Error('Failed to generate batch presigned URLs')
)
}
})
@@ -1,978 +0,0 @@
/**
* Tests for file presigned API route
*
* @vitest-environment node
*/
import { authMockFns, storageServiceMock, storageServiceMockFns } from '@sim/testing'
import { NextRequest } from 'next/server'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
const {
mockVerifyFileAccess,
mockVerifyWorkspaceFileAccess,
mockUseBlobStorage,
mockUseS3Storage,
mockGetStorageConfig,
mockIsUsingCloudStorage,
mockGetStorageProvider,
mockValidateFileType,
mockValidateAttachmentFileType,
mockGenerateCopilotUploadUrl,
mockIsImageFileType,
mockGetStorageProviderUploads,
mockIsUsingCloudStorageUploads,
mockGetUserEntityPermissions,
mockGenerateWorkspaceFileKey,
mockGenerateUniqueExecutionFileKey,
mockInsertFileMetadata,
mockCheckStorageQuotaForBillingContext,
mockDecrementStorageUsageForBillingContext,
mockIncrementStorageUsageForBillingContext,
mockResolveStorageBillingContext,
mockSignUploadToken,
} = vi.hoisted(() => ({
mockVerifyFileAccess: vi.fn().mockResolvedValue(true),
mockVerifyWorkspaceFileAccess: vi.fn().mockResolvedValue(true),
mockUseBlobStorage: { value: false },
mockUseS3Storage: { value: true },
mockGetStorageConfig: vi.fn(),
mockIsUsingCloudStorage: vi.fn(),
mockGetStorageProvider: vi.fn(),
mockValidateFileType: vi.fn().mockReturnValue(null),
mockValidateAttachmentFileType: vi.fn().mockReturnValue(null),
mockGenerateCopilotUploadUrl: vi.fn().mockResolvedValue({
url: 'https://example.com/presigned-url',
key: 'copilot/test-key.txt',
}),
mockIsImageFileType: vi.fn().mockReturnValue(true),
mockGetStorageProviderUploads: vi.fn(),
mockIsUsingCloudStorageUploads: vi.fn(),
mockGetUserEntityPermissions: vi.fn().mockResolvedValue('admin'),
mockGenerateWorkspaceFileKey: vi.fn(
(workspaceId: string, fileName: string) => `workspace/${workspaceId}/${fileName}`
),
mockGenerateUniqueExecutionFileKey: vi.fn(
(ctx: { workspaceId: string; workflowId: string; executionId: string }, fileName: string) =>
`execution/${ctx.workspaceId}/${ctx.workflowId}/${ctx.executionId}/attachment-${fileName}`
),
mockInsertFileMetadata: vi.fn().mockResolvedValue({ id: 'wf_test' }),
mockCheckStorageQuotaForBillingContext: vi.fn(),
mockDecrementStorageUsageForBillingContext: vi.fn(),
mockIncrementStorageUsageForBillingContext: vi.fn(),
mockResolveStorageBillingContext: vi.fn(),
mockSignUploadToken: vi.fn(),
}))
vi.mock('@/app/api/files/authorization', () => ({
verifyFileAccess: mockVerifyFileAccess,
verifyWorkspaceFileAccess: mockVerifyWorkspaceFileAccess,
}))
vi.mock('@/lib/uploads/config', () => ({
get USE_BLOB_STORAGE() {
return mockUseBlobStorage.value
},
get USE_S3_STORAGE() {
return mockUseS3Storage.value
},
UPLOAD_DIR: '/uploads',
getServeStoragePrefix: () => (mockUseBlobStorage.value ? 'blob' : 's3'),
getStorageConfig: mockGetStorageConfig,
isUsingCloudStorage: mockIsUsingCloudStorage,
getStorageProvider: mockGetStorageProvider,
}))
vi.mock('@/lib/uploads/core/storage-service', () => storageServiceMock)
vi.mock('@/lib/uploads/core/upload-token', () => ({
signUploadToken: mockSignUploadToken,
}))
vi.mock('@/lib/billing/storage', () => ({
checkStorageQuotaForBillingContext: mockCheckStorageQuotaForBillingContext,
decrementStorageUsageForBillingContext: mockDecrementStorageUsageForBillingContext,
incrementStorageUsageForBillingContext: mockIncrementStorageUsageForBillingContext,
resolveStorageBillingContext: mockResolveStorageBillingContext,
}))
vi.mock('@/lib/uploads/utils/validation', () => ({
validateFileType: mockValidateFileType,
validateAttachmentFileType: mockValidateAttachmentFileType,
}))
vi.mock('@/lib/workspaces/permissions/utils', () => ({
getUserEntityPermissions: mockGetUserEntityPermissions,
}))
vi.mock('@/lib/uploads/contexts/workspace/workspace-file-manager', () => ({
generateWorkspaceFileKey: mockGenerateWorkspaceFileKey,
}))
vi.mock('@/lib/uploads/contexts/execution/utils', () => ({
generateUniqueExecutionFileKey: mockGenerateUniqueExecutionFileKey,
}))
vi.mock('@/lib/uploads/server/metadata', () => ({
insertImmutableFileMetadata: mockInsertFileMetadata,
recordKnowledgeBaseFileOwnership: (ownership: Record<string, unknown>) =>
mockInsertFileMetadata({ ...ownership, context: 'knowledge-base' }),
}))
vi.mock('@/lib/uploads/utils/file-utils', () => ({
isImageFileType: mockIsImageFileType,
}))
vi.mock('@/lib/uploads', () => ({
CopilotFiles: {
generateCopilotUploadUrl: mockGenerateCopilotUploadUrl,
},
getStorageProvider: mockGetStorageProviderUploads,
isUsingCloudStorage: mockIsUsingCloudStorageUploads,
}))
import { POST } from '@/app/api/files/presigned/route'
const defaultMockUser = {
id: 'test-user-id',
name: 'Test User',
email: 'test@example.com',
}
function setupFileApiMocks(
options: {
authenticated?: boolean
storageProvider?: 's3' | 'blob' | 'local'
cloudEnabled?: boolean
} = {}
) {
const { authenticated = true, storageProvider = 's3', cloudEnabled = true } = options
if (authenticated) {
authMockFns.mockGetSession.mockResolvedValue({ user: defaultMockUser })
} else {
authMockFns.mockGetSession.mockResolvedValue(null)
}
const useBlobStorage = storageProvider === 'blob' && cloudEnabled
const useS3Storage = storageProvider === 's3' && cloudEnabled
mockUseBlobStorage.value = useBlobStorage
mockUseS3Storage.value = useS3Storage
mockGetStorageConfig.mockReturnValue(
useBlobStorage
? {
accountName: 'testaccount',
accountKey: 'testkey',
connectionString: 'testconnection',
containerName: 'testcontainer',
}
: {
bucket: 'test-bucket',
region: 'us-east-1',
}
)
mockIsUsingCloudStorage.mockReturnValue(cloudEnabled)
mockGetStorageProvider.mockReturnValue(
storageProvider === 'blob' ? 'Azure Blob' : storageProvider === 's3' ? 'S3' : 'Local'
)
storageServiceMockFns.mockHasCloudStorage.mockReturnValue(cloudEnabled)
storageServiceMockFns.mockGeneratePresignedUploadUrl.mockImplementation(
async (opts: { fileName: string; context: string; customKey?: string }) => {
const timestamp = Date.now()
const safeFileName = opts.fileName.replace(/[^a-zA-Z0-9.-]/g, '_')
const key = opts.customKey ?? `${opts.context}/${timestamp}-ik3a6w4-${safeFileName}`
return {
url: 'https://example.com/presigned-url',
key,
uploadId: `receipt-${key}`,
}
}
)
storageServiceMockFns.mockGeneratePresignedDownloadUrl.mockResolvedValue(
'https://example.com/presigned-url'
)
mockValidateFileType.mockReturnValue(null)
mockValidateAttachmentFileType.mockReturnValue(null)
mockGetUserEntityPermissions.mockResolvedValue('admin')
mockGetStorageProviderUploads.mockReturnValue(
storageProvider === 'blob' ? 'Azure Blob' : storageProvider === 's3' ? 'S3' : 'Local'
)
mockIsUsingCloudStorageUploads.mockReturnValue(cloudEnabled)
}
describe('/api/files/presigned', () => {
beforeEach(() => {
vi.clearAllMocks()
vi.useFakeTimers()
vi.setSystemTime(new Date('2024-01-01T00:00:00Z'))
vi.stubGlobal('crypto', {
randomUUID: vi.fn().mockReturnValue('mock-uuid-1234-5678'),
})
mockSignUploadToken.mockReturnValue('signed-receipt-token')
})
afterEach(() => {
vi.useRealTimers()
})
describe('POST', () => {
it('should return graceful fallback response when cloud storage is not enabled', async () => {
setupFileApiMocks({
cloudEnabled: false,
storageProvider: 's3',
})
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=profile-pictures',
{
method: 'POST',
body: JSON.stringify({
fileName: 'avatar.png',
contentType: 'image/png',
fileSize: 1024,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(200)
expect(data.directUploadSupported).toBe(false)
expect(data.presignedUrl).toBe('')
expect(data.fileName).toBe('avatar.png')
expect(data.fileInfo).toBeDefined()
expect(data.fileInfo.name).toBe('avatar.png')
expect(data.fileInfo.size).toBe(1024)
expect(data.fileInfo.type).toBe('image/png')
})
it('should return error when fileName is missing', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 's3',
})
const request = new NextRequest('http://localhost:3000/api/files/presigned', {
method: 'POST',
body: JSON.stringify({
contentType: 'text/plain',
fileSize: 1024,
}),
})
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(400)
expect(data.error).toBe('fileName is required and cannot be empty')
expect(data.code).toBe('VALIDATION_ERROR')
})
it('should return error when contentType is missing', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 's3',
})
const request = new NextRequest('http://localhost:3000/api/files/presigned', {
method: 'POST',
body: JSON.stringify({
fileName: 'test.txt',
fileSize: 1024,
}),
})
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(400)
expect(data.error).toBe('contentType is required and cannot be empty')
expect(data.code).toBe('VALIDATION_ERROR')
})
it('should return error when fileSize is invalid', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 's3',
})
const request = new NextRequest('http://localhost:3000/api/files/presigned', {
method: 'POST',
body: JSON.stringify({
fileName: 'test.txt',
contentType: 'text/plain',
fileSize: 0,
}),
})
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(400)
expect(data.error).toBe('fileSize must be a positive number')
expect(data.code).toBe('VALIDATION_ERROR')
})
it('should return error when file size exceeds limit', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 's3',
})
const largeFileSize = 150 * 1024 * 1024 // 150MB (exceeds 100MB limit)
const request = new NextRequest('http://localhost:3000/api/files/presigned', {
method: 'POST',
body: JSON.stringify({
fileName: 'large-file.txt',
contentType: 'text/plain',
fileSize: largeFileSize,
}),
})
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(400)
expect(data.error).toContain('exceeds maximum allowed size')
expect(data.code).toBe('VALIDATION_ERROR')
})
it('should generate S3 presigned URL successfully', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 's3',
})
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=profile-pictures',
{
method: 'POST',
body: JSON.stringify({
fileName: 'test avatar.png',
contentType: 'image/png',
fileSize: 1024,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(200)
expect(data.presignedUrl).toBe('https://example.com/presigned-url')
expect(data.fileInfo).toMatchObject({
path: expect.stringMatching(/\/api\/files\/serve\/s3\/.+\?context=profile-pictures$/),
key: expect.stringMatching(/.*test.avatar\.png$/),
name: 'test avatar.png',
size: 1024,
type: 'image/png',
})
expect(data.directUploadSupported).toBe(true)
})
it('should generate knowledge-base S3 presigned URL with kb prefix', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 's3',
})
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=knowledge-base&workspaceId=ws-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'knowledge-doc.pdf',
contentType: 'application/pdf',
fileSize: 2048,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(200)
expect(data.fileInfo.key).toMatch(/^kb\/.*knowledge-doc\.pdf$/)
expect(data.directUploadSupported).toBe(true)
})
it('should generate profile-pictures S3 presigned URL with its prefix and direct path', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 's3',
})
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=profile-pictures',
{
method: 'POST',
body: JSON.stringify({
fileName: 'avatar.png',
contentType: 'image/png',
fileSize: 4096,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(200)
expect(data.fileInfo.key).toMatch(/^profile-pictures\/.*avatar\.png$/)
expect(data.fileInfo.path).toMatch(/\/api\/files\/serve\/s3\/.+\?context=profile-pictures$/)
expect(data.presignedUrl).toBeTruthy()
expect(data.directUploadSupported).toBe(true)
})
it('should generate Azure Blob presigned URL successfully', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 'blob',
})
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=profile-pictures',
{
method: 'POST',
body: JSON.stringify({
fileName: 'test avatar.png',
contentType: 'image/png',
fileSize: 1024,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(200)
expect(data.presignedUrl).toBeTruthy()
expect(typeof data.presignedUrl).toBe('string')
expect(data.fileInfo).toMatchObject({
key: expect.stringMatching(/.*test.avatar\.png$/),
name: 'test avatar.png',
size: 1024,
type: 'image/png',
})
expect(data.directUploadSupported).toBe(true)
})
it('should generate profile-pictures Azure Blob presigned URL with its prefix and direct path', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 'blob',
})
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=profile-pictures',
{
method: 'POST',
body: JSON.stringify({
fileName: 'avatar.png',
contentType: 'image/png',
fileSize: 4096,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(200)
expect(data.fileInfo.key).toMatch(/^profile-pictures\/.*avatar\.png$/)
expect(data.fileInfo.path).toMatch(/\/api\/files\/serve\/blob\/.+\?context=profile-pictures$/)
expect(data.presignedUrl).toBeTruthy()
expect(data.directUploadSupported).toBe(true)
})
it('should return error for unknown storage provider', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 's3',
})
storageServiceMockFns.mockGeneratePresignedUploadUrl.mockRejectedValue(
new Error('Unknown storage provider: unknown')
)
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=profile-pictures',
{
method: 'POST',
body: JSON.stringify({
fileName: 'avatar.png',
contentType: 'image/png',
fileSize: 1024,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(500)
expect(data.error).toBeTruthy()
expect(typeof data.error).toBe('string')
})
it('should handle S3 errors gracefully', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 's3',
})
storageServiceMockFns.mockGeneratePresignedUploadUrl.mockRejectedValue(
new Error('S3 service unavailable')
)
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=profile-pictures',
{
method: 'POST',
body: JSON.stringify({
fileName: 'avatar.png',
contentType: 'image/png',
fileSize: 1024,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(500)
expect(data.error).toBeTruthy()
expect(typeof data.error).toBe('string')
})
it('should handle Azure Blob errors gracefully', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 'blob',
})
storageServiceMockFns.mockGeneratePresignedUploadUrl.mockRejectedValue(
new Error('Azure service unavailable')
)
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=profile-pictures',
{
method: 'POST',
body: JSON.stringify({
fileName: 'avatar.png',
contentType: 'image/png',
fileSize: 1024,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(500)
expect(data.error).toBeTruthy()
expect(typeof data.error).toBe('string')
})
it('should handle malformed JSON gracefully', async () => {
setupFileApiMocks({
cloudEnabled: true,
storageProvider: 's3',
})
const request = new NextRequest('http://localhost:3000/api/files/presigned', {
method: 'POST',
body: 'invalid json',
})
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(400) // Changed from 500 to 400 (ValidationError)
expect(data.error).toBe('Invalid JSON in request body') // Updated error message
expect(data.code).toBe('VALIDATION_ERROR')
})
it('rejects the unauthorizable chat context without minting a URL', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
const request = new NextRequest('http://localhost:3000/api/files/presigned?type=chat', {
method: 'POST',
body: JSON.stringify({
fileName: 'poc.html',
contentType: 'text/html',
fileSize: 41,
}),
})
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(400)
expect(data.error).toContain('Invalid type parameter')
expect(storageServiceMockFns.mockGeneratePresignedUploadUrl).not.toHaveBeenCalled()
})
})
describe('mothership uploads', () => {
it('uses validateAttachmentFileType (not validateFileType) — accepts images', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=mothership&workspaceId=ws-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'screenshot.png',
contentType: 'image/png',
fileSize: 4096,
}),
}
)
const response = await POST(request)
expect(response.status).toBe(200)
expect(mockValidateAttachmentFileType).toHaveBeenCalledWith('screenshot.png', {
allowArchives: true,
})
expect(mockValidateFileType).not.toHaveBeenCalled()
})
it('rejects unsupported types when validator returns an error', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
mockValidateAttachmentFileType.mockReturnValue({
code: 'UNSUPPORTED_FILE_TYPE',
message: 'Unsupported file type: exe.',
supportedTypes: [],
})
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=mothership&workspaceId=ws-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'virus.exe',
contentType: 'application/octet-stream',
fileSize: 4096,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(400)
expect(data.code).toBe('VALIDATION_ERROR')
expect(data.error).toContain('exe')
})
it('returns 403 when user lacks workspace write permission', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
mockGetUserEntityPermissions.mockResolvedValue('read')
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=mothership&workspaceId=ws-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'doc.pdf',
contentType: 'application/pdf',
fileSize: 4096,
}),
}
)
const response = await POST(request)
expect(response.status).toBe(403)
})
it('issues an unbilled pending mothership upload binding', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=mothership&workspaceId=ws-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'screenshot.png',
contentType: 'image/png',
fileSize: 4096,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(200)
expect(mockInsertFileMetadata).toHaveBeenCalledTimes(1)
expect(mockInsertFileMetadata).toHaveBeenCalledWith({
key: data.fileInfo.key,
userId: 'test-user-id',
workspaceId: 'ws-1',
context: 'mothership',
originalName: 'screenshot.png',
contentType: 'image/png',
size: 4096,
})
expect(mockCheckStorageQuotaForBillingContext).not.toHaveBeenCalled()
expect(mockResolveStorageBillingContext).not.toHaveBeenCalled()
expect(mockIncrementStorageUsageForBillingContext).not.toHaveBeenCalled()
expect(mockDecrementStorageUsageForBillingContext).not.toHaveBeenCalled()
})
it('returns 500 when insertFileMetadata fails so callers do not get an unauthorizable URL', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
mockInsertFileMetadata.mockRejectedValueOnce(new Error('DB connection lost'))
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=mothership&workspaceId=ws-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'screenshot.png',
contentType: 'image/png',
fileSize: 4096,
}),
}
)
const response = await POST(request)
expect(response.status).toBe(500)
})
})
describe('execution uploads', () => {
it('allocates distinct create-only keys for duplicate attachment names', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
mockGenerateUniqueExecutionFileKey
.mockReturnValueOnce('execution/ws-1/wf-1/exec-1/one-output.txt')
.mockReturnValueOnce('execution/ws-1/wf-1/exec-1/two-output.txt')
const makeExecutionRequest = (fileSize: number) =>
new NextRequest(
'http://localhost:3000/api/files/presigned?type=execution&workspaceId=ws-1&workflowId=wf-1&executionId=exec-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'output.txt',
contentType: 'text/plain',
fileSize,
}),
}
)
const first = await POST(makeExecutionRequest(3))
const second = await POST(makeExecutionRequest(12))
const firstBody = await first.json()
const secondBody = await second.json()
expect(first.status).toBe(200)
expect(second.status).toBe(200)
expect(firstBody.fileInfo.key).toBe('execution/ws-1/wf-1/exec-1/one-output.txt')
expect(secondBody.fileInfo.key).toBe('execution/ws-1/wf-1/exec-1/two-output.txt')
expect(firstBody.fileInfo.key).not.toBe(secondBody.fileInfo.key)
expect(firstBody.uploadToken).toBe('signed-receipt-token')
expect(secondBody.uploadToken).toBe('signed-receipt-token')
})
it('uses validateAttachmentFileType — accepts video', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=execution&workspaceId=ws-1&workflowId=wf-1&executionId=exec-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'output.mp4',
contentType: 'video/mp4',
fileSize: 4096,
}),
}
)
const response = await POST(request)
expect(response.status).toBe(200)
expect(mockValidateAttachmentFileType).toHaveBeenCalledWith('output.mp4')
expect(mockValidateFileType).not.toHaveBeenCalled()
})
it('rejects when validator returns an error', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
mockValidateAttachmentFileType.mockReturnValue({
code: 'UNSUPPORTED_FILE_TYPE',
message: 'Unsupported file type: bin.',
supportedTypes: [],
})
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=execution&workspaceId=ws-1&workflowId=wf-1&executionId=exec-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'blob.bin',
contentType: 'application/octet-stream',
fileSize: 4096,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(400)
expect(data.code).toBe('VALIDATION_ERROR')
})
it('returns 400 when missing workflowId/executionId', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=execution&workspaceId=ws-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'output.mp4',
contentType: 'video/mp4',
fileSize: 4096,
}),
}
)
const response = await POST(request)
expect(response.status).toBe(400)
})
it('inserts a workspaceFiles row with context=execution so previews authorize', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=execution&workspaceId=ws-1&workflowId=wf-1&executionId=exec-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'output.mp4',
contentType: 'video/mp4',
fileSize: 4096,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(200)
expect(mockInsertFileMetadata).toHaveBeenCalledTimes(1)
expect(mockInsertFileMetadata).toHaveBeenCalledWith({
key: data.fileInfo.key,
userId: 'test-user-id',
workspaceId: 'ws-1',
context: 'execution',
originalName: 'output.mp4',
contentType: 'video/mp4',
size: 4096,
})
})
})
describe('workspace-logos uploads', () => {
it('inserts a workspaceFiles row with context=workspace-logos so logos authorize', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=workspace-logos&workspaceId=ws-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'logo.png',
contentType: 'image/png',
fileSize: 4096,
}),
}
)
const response = await POST(request)
const data = await response.json()
expect(response.status).toBe(200)
expect(mockInsertFileMetadata).toHaveBeenCalledTimes(1)
expect(mockInsertFileMetadata).toHaveBeenCalledWith({
key: data.fileInfo.key,
userId: 'test-user-id',
workspaceId: 'ws-1',
context: 'workspace-logos',
originalName: 'logo.png',
contentType: 'image/png',
size: 4096,
})
})
})
describe('knowledge-base uploads', () => {
it('uses validateFileType (docs-only), not validateAttachmentFileType', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=knowledge-base&workspaceId=ws-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'doc.pdf',
contentType: 'application/pdf',
fileSize: 4096,
}),
}
)
const response = await POST(request)
expect(response.status).toBe(200)
expect(mockValidateFileType).toHaveBeenCalledWith('doc.pdf', 'application/pdf')
expect(mockValidateAttachmentFileType).not.toHaveBeenCalled()
})
it('requires workspaceId for knowledge-base uploads', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=knowledge-base',
{
method: 'POST',
body: JSON.stringify({
fileName: 'doc.pdf',
contentType: 'application/pdf',
fileSize: 4096,
}),
}
)
const response = await POST(request)
expect(response.status).toBe(400)
})
it('returns 403 when the user lacks write access to the workspace', async () => {
setupFileApiMocks({ cloudEnabled: true, storageProvider: 's3' })
mockGetUserEntityPermissions.mockResolvedValue('read')
const request = new NextRequest(
'http://localhost:3000/api/files/presigned?type=knowledge-base&workspaceId=ws-1',
{
method: 'POST',
body: JSON.stringify({
fileName: 'doc.pdf',
contentType: 'application/pdf',
fileSize: 4096,
}),
}
)
const response = await POST(request)
expect(response.status).toBe(403)
})
})
})
-404
View File
@@ -1,404 +0,0 @@
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { type NextRequest, NextResponse } from 'next/server'
import {
presignedUploadBodyContract,
presignedUploadTypeSchema,
presignedUploadTypes,
} from '@/lib/api/contracts/storage-transfer'
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { getSession } from '@/lib/auth'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { CopilotFiles } from '@/lib/uploads'
import { getServeStoragePrefix } from '@/lib/uploads/config'
import { generateUniqueExecutionFileKey } from '@/lib/uploads/contexts/execution/utils'
import { generateKnowledgeBaseFileKey } from '@/lib/uploads/contexts/knowledge-base/knowledge-base-file-manager'
import { generateWorkspaceFileKey } from '@/lib/uploads/contexts/workspace/workspace-file-manager'
import { generatePresignedUploadUrl, hasCloudStorage } from '@/lib/uploads/core/storage-service'
import { signUploadToken } from '@/lib/uploads/core/upload-token'
import {
insertImmutableFileMetadata,
recordKnowledgeBaseFileOwnership,
} from '@/lib/uploads/server/metadata'
import type { PresignedUrlResponse, StorageContext } from '@/lib/uploads/shared/types'
import { isImageFileType } from '@/lib/uploads/utils/file-utils'
import { validateAttachmentFileType, validateFileType } from '@/lib/uploads/utils/validation'
import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
import { createErrorResponse } from '@/app/api/files/utils'
const logger = createLogger('PresignedUploadAPI')
class PresignedUrlError extends Error {
constructor(
message: string,
public code: string,
public statusCode = 400
) {
super(message)
this.name = 'PresignedUrlError'
}
}
class ValidationError extends PresignedUrlError {
constructor(message: string) {
super(message, 'VALIDATION_ERROR', 400)
}
}
function createDirectUploadToken(options: {
response: PresignedUrlResponse
userId: string
workspaceId: string
context: StorageContext
fileName: string
contentType: string
fileSize: number
}): string | undefined {
if (!options.response.uploadId) return undefined
return signUploadToken({
uploadId: options.response.uploadId,
key: options.response.key,
userId: options.userId,
workspaceId: options.workspaceId,
context: options.context,
fileName: options.fileName,
contentType: options.contentType,
fileSize: options.fileSize,
uploadKind: 'direct',
})
}
export const POST = withRouteHandler(async (request: NextRequest) => {
try {
const session = await getSession()
if (!session?.user?.id) {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const parsed = await parseRequest(
presignedUploadBodyContract,
request,
{},
{
validationErrorResponse: (error) => {
throw new ValidationError(getValidationErrorMessage(error, 'Invalid request data'))
},
invalidJsonResponse: () => {
throw new ValidationError('Invalid JSON in request body')
},
}
)
if (!parsed.success) return parsed.response
const { fileName, contentType, fileSize } = parsed.data.body
const uploadTypeParam = request.nextUrl.searchParams.get('type')
if (!uploadTypeParam) {
throw new ValidationError('type query parameter is required')
}
const uploadTypeResult = presignedUploadTypeSchema.safeParse(uploadTypeParam)
if (!uploadTypeResult.success) {
throw new ValidationError(
`Invalid type parameter. Must be one of: ${presignedUploadTypes.join(', ')}`
)
}
const uploadType = uploadTypeResult.data
if (uploadType === 'knowledge-base') {
const fileValidationError = validateFileType(fileName, contentType)
if (fileValidationError) {
throw new ValidationError(`${fileValidationError.message}`)
}
}
const sessionUserId = session.user.id
if (!hasCloudStorage()) {
logger.info(
`Local storage detected - presigned URL not available for ${fileName}, client will use API fallback`
)
return NextResponse.json({
fileName,
presignedUrl: '', // Empty URL signals fallback to API upload
fileInfo: {
path: '',
key: '',
name: fileName,
size: fileSize,
type: contentType,
},
directUploadSupported: false,
})
}
logger.info(`Generating ${uploadType} presigned URL for ${fileName}`)
let presignedUrlResponse: PresignedUrlResponse
let uploadToken: string | undefined
if (uploadType === 'copilot') {
try {
presignedUrlResponse = await CopilotFiles.generateCopilotUploadUrl({
fileName,
contentType,
fileSize,
userId: sessionUserId,
expirationSeconds: 3600,
})
} catch (error) {
throw new ValidationError(getErrorMessage(error, 'Chat validation failed'))
}
} else if (uploadType === 'mothership') {
const workspaceId = request.nextUrl.searchParams.get('workspaceId')
if (!workspaceId?.trim()) {
throw new ValidationError('workspaceId query parameter is required for chat uploads')
}
const permission = await getUserEntityPermissions(sessionUserId, 'workspace', workspaceId)
if (permission !== 'write' && permission !== 'admin') {
return NextResponse.json(
{ error: 'Write or Admin access required for chat uploads' },
{ status: 403 }
)
}
const fileValidationError = validateAttachmentFileType(fileName, { allowArchives: true })
if (fileValidationError) {
throw new ValidationError(fileValidationError.message)
}
const customKey = generateWorkspaceFileKey(workspaceId, fileName)
presignedUrlResponse = await generatePresignedUploadUrl({
fileName,
contentType,
fileSize,
context: 'mothership',
userId: sessionUserId,
customKey,
expirationSeconds: 3600,
metadata: { workspaceId },
})
uploadToken = createDirectUploadToken({
response: presignedUrlResponse,
userId: sessionUserId,
workspaceId,
context: 'mothership',
fileName,
contentType,
fileSize,
})
await insertImmutableFileMetadata({
key: presignedUrlResponse.key,
userId: sessionUserId,
workspaceId,
context: 'mothership',
originalName: fileName,
contentType,
size: fileSize,
})
} else if (uploadType === 'execution') {
const workflowId = request.nextUrl.searchParams.get('workflowId')
const executionId = request.nextUrl.searchParams.get('executionId')
const workspaceId = request.nextUrl.searchParams.get('workspaceId')
if (!workflowId?.trim() || !executionId?.trim() || !workspaceId?.trim()) {
throw new ValidationError(
'workflowId, executionId, and workspaceId query parameters are required for execution uploads'
)
}
const permission = await getUserEntityPermissions(sessionUserId, 'workspace', workspaceId)
if (permission !== 'write' && permission !== 'admin') {
return NextResponse.json(
{ error: 'Write or Admin access required for execution uploads' },
{ status: 403 }
)
}
const fileValidationError = validateAttachmentFileType(fileName)
if (fileValidationError) {
throw new ValidationError(fileValidationError.message)
}
const customKey = generateUniqueExecutionFileKey(
{ workspaceId, workflowId, executionId },
fileName
)
presignedUrlResponse = await generatePresignedUploadUrl({
fileName,
contentType,
fileSize,
context: 'execution',
userId: sessionUserId,
customKey,
expirationSeconds: 3600,
metadata: { workspaceId, workflowId, executionId },
})
uploadToken = createDirectUploadToken({
response: presignedUrlResponse,
userId: sessionUserId,
workspaceId,
context: 'execution',
fileName,
contentType,
fileSize,
})
await insertImmutableFileMetadata({
key: presignedUrlResponse.key,
userId: sessionUserId,
workspaceId,
context: 'execution',
originalName: fileName,
contentType,
size: fileSize,
})
} else if (uploadType === 'workspace-logos') {
const workspaceId = request.nextUrl.searchParams.get('workspaceId')
if (!workspaceId?.trim()) {
throw new ValidationError(
'workspaceId query parameter is required for workspace-logos uploads'
)
}
const permission = await getUserEntityPermissions(sessionUserId, 'workspace', workspaceId)
if (permission !== 'admin') {
return NextResponse.json(
{ error: 'Admin access required for workspace logo uploads' },
{ status: 403 }
)
}
if (!isImageFileType(contentType)) {
throw new ValidationError(
'Only image files (JPEG, PNG, GIF, WebP, SVG) are allowed for workspace logo uploads'
)
}
presignedUrlResponse = await generatePresignedUploadUrl({
fileName,
contentType,
fileSize,
context: 'workspace-logos',
userId: sessionUserId,
expirationSeconds: 3600,
metadata: { workspaceId },
})
uploadToken = createDirectUploadToken({
response: presignedUrlResponse,
userId: sessionUserId,
workspaceId,
context: 'workspace-logos',
fileName,
contentType,
fileSize,
})
await insertImmutableFileMetadata({
key: presignedUrlResponse.key,
userId: sessionUserId,
workspaceId,
context: 'workspace-logos',
originalName: fileName,
contentType,
size: fileSize,
})
} else if (uploadType === 'knowledge-base') {
const workspaceId = request.nextUrl.searchParams.get('workspaceId')
if (!workspaceId?.trim()) {
throw new ValidationError(
'workspaceId query parameter is required for knowledge-base uploads'
)
}
const permission = await getUserEntityPermissions(sessionUserId, 'workspace', workspaceId)
if (permission !== 'write' && permission !== 'admin') {
return NextResponse.json(
{ error: 'Write or Admin access required for knowledge-base uploads' },
{ status: 403 }
)
}
const customKey = generateKnowledgeBaseFileKey(fileName)
presignedUrlResponse = await generatePresignedUploadUrl({
fileName,
contentType,
fileSize,
context: 'knowledge-base',
userId: sessionUserId,
customKey,
expirationSeconds: 3600,
metadata: { workspaceId },
})
uploadToken = createDirectUploadToken({
response: presignedUrlResponse,
userId: sessionUserId,
workspaceId,
context: 'knowledge-base',
fileName,
contentType,
fileSize,
})
await recordKnowledgeBaseFileOwnership({
key: presignedUrlResponse.key,
userId: sessionUserId,
workspaceId,
originalName: fileName,
contentType,
size: fileSize,
})
} else {
if (!isImageFileType(contentType)) {
throw new ValidationError(
'Only image files (JPEG, PNG, GIF, WebP, SVG) are allowed for profile picture uploads'
)
}
presignedUrlResponse = await generatePresignedUploadUrl({
fileName,
contentType,
fileSize,
context: uploadType,
userId: sessionUserId,
expirationSeconds: 3600, // 1 hour
})
}
const finalPath = `/api/files/serve/${getServeStoragePrefix()}/${encodeURIComponent(presignedUrlResponse.key)}?context=${uploadType}`
return NextResponse.json({
fileName,
presignedUrl: presignedUrlResponse.url,
fileInfo: {
path: finalPath,
key: presignedUrlResponse.key,
name: fileName,
size: fileSize,
type: contentType,
},
uploadHeaders: presignedUrlResponse.uploadHeaders,
...(uploadToken ? { uploadToken } : {}),
directUploadSupported: true,
})
} catch (error) {
logger.error('Error generating presigned URL:', error)
if (error instanceof PresignedUrlError) {
return NextResponse.json(
{
error: error.message,
code: error.code,
directUploadSupported: false,
},
{ status: error.statusCode }
)
}
return createErrorResponse(
error instanceof Error ? error : new Error('Failed to generate presigned URL')
)
}
})
@@ -1,73 +0,0 @@
/**
* @vitest-environment node
*/
import { authMockFns, storageServiceMock, storageServiceMockFns } from '@sim/testing'
import { NextRequest } from 'next/server'
import { beforeEach, describe, expect, it, vi } from 'vitest'
const { mockVerifyUploadToken } = vi.hoisted(() => ({
mockVerifyUploadToken: vi.fn(),
}))
vi.mock('@/lib/uploads/core/storage-service', () => storageServiceMock)
vi.mock('@/lib/uploads/core/upload-token', () => ({
verifyUploadToken: mockVerifyUploadToken,
}))
import { POST } from '@/app/api/files/presigned/verify/route'
const request = () =>
new NextRequest('http://localhost/api/files/presigned/verify', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ uploadToken: 'signed-token' }),
})
describe('POST /api/files/presigned/verify', () => {
beforeEach(() => {
vi.clearAllMocks()
authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
mockVerifyUploadToken.mockReturnValue({
valid: true,
payload: {
uploadId: 'receipt-1',
key: 'knowledge-base/unique-file.pdf',
userId: 'user-1',
workspaceId: 'workspace-1',
context: 'knowledge-base',
uploadKind: 'direct',
},
})
storageServiceMockFns.mockVerifyPresignedUploadReceipt.mockResolvedValue(true)
})
it('verifies the signed key and receipt against object metadata', async () => {
const response = await POST(request())
expect(response.status).toBe(200)
await expect(response.json()).resolves.toEqual({ uploaded: true })
expect(storageServiceMockFns.mockVerifyPresignedUploadReceipt).toHaveBeenCalledWith({
key: 'knowledge-base/unique-file.pdf',
context: 'knowledge-base',
uploadId: 'receipt-1',
})
})
it('does not treat a multipart or user-mismatched token as a direct-upload receipt', async () => {
mockVerifyUploadToken.mockReturnValueOnce({
valid: true,
payload: {
uploadId: 'upload-1',
key: 'knowledge-base/file.pdf',
userId: 'different-user',
workspaceId: 'workspace-1',
context: 'knowledge-base',
},
})
const response = await POST(request())
expect(response.status).toBe(403)
expect(storageServiceMockFns.mockVerifyPresignedUploadReceipt).not.toHaveBeenCalled()
})
})

Some files were not shown because too many files have changed in this diff Show More