mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
d28e8d7daaf2e420491979555be9d5cd2074e570
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Theodore Li
|
d86fdc152f |
feat(api): publish agent tool input schema (#7109)
* feat(api): publish agent tool input schema * fix(api): bound agent tool inputs |
||
|
WaleedandTheodore Li
|
656840a1b1 |
feat(api): make the platform operable headless over v2 (#6912)
* feat(v2): download run output files by API key
Adds GET /api/v2/workflows/{id}/runs/{runId}/files/{fileId}, closing the
async-run loop for headless callers. A run's output carries UserFile URLs
pointing at /api/files/serve/..., which rejects x-api-key outright, so an
async run that produces a file previously had no byte path out for an API
key at all.
The file is addressed by the id the run reported and resolved against the
run's own recorded execution data, from which the storage key is read. The
request never supplies a storage key, so the endpoint cannot be aimed at
bytes the run did not produce. Resolution deliberately reads the
materialized-but-undisplayed recording, because the display projection
strips exactly the `key`/`context` fields a byte read needs.
Also hardens normalizeStartFile to derive a file's storage key only from a
validated internal serve URL, discarding any caller-supplied `key`/`context`.
A workspace API key has no human subject, so the executor resolves its actor
to the workspace billing owner (preprocessing.ts -> resolveSystemBillingAttribution);
verifyFileAccess then authorizes a workspace-context key as that owner, whose
reach is not bounded by the key's workspace. Accepting an attacker-authorable
key made that substitution exploitable as a confused deputy. Normalization is
all-or-nothing, so a forged file now drops the whole files input.
* feat(workflows): one graph-write door, principal-derived audit source, and v2 authoring endpoints
Extract replaceWorkflowNormalizedState as the single persistence primitive for a
workflow graph replace and route both the internal editor save and the Copilot
edit tool through it, so neither can skip state preparation, the row lock, the
lastSynced stamp, or custom-tool extraction by choosing a different entry point.
Derive the audit source from the acting principal instead of hardcoding
'copilot', then widen workflows.variables.apply_operations and
workflows.bulk.move to every principal kind.
Add GET/PUT /api/v2/workflows/{id}/state, POST /operations, /duplicate,
/restore, PATCH /variables, and POST /api/v2/workflows/move over surface-neutral
application use cases; move the edit engine to lib/workflows/editing.
* test(workflows): cover the graph-write primitive, audit source, and the v2 authoring surface
Pin the two-doors fix (preparation runs, the row is locked, custom-tool
extraction is post-commit and best-effort) and the false-audit fix (a session
principal writes source: 'session', a delegated one writes its service). Both
were verified to fail with the fix reverted.
Add the application matrix for replaceWorkflowState, applyWorkflowOperations,
readWorkflowGraph, and restoreWorkflow — role floor, principal-kind rejection
before canonical load, asserted-scope concealment, lock, validation, atomic
conflict, plan gate, and audit-then-notify ordering — plus route tests for
every new endpoint.
* test(workflows): pin the internal graph-write door and the v2 list scope
Characterize saveWorkflowNormalizedState's statuses, messages, and notification
after the persistence extraction, and cover the new scope filter on
GET /api/v2/workflows including a cursor replayed under a different scope.
* feat(api): add v2 block, tool, connector-type, and enrichment catalogs
Adds six read endpoints under /api/v2 that publish Sim's code-defined
catalogs: GET /blocks, GET /blocks/{blockId}, GET /tools,
GET /tools/{toolId}, GET /connector-types, and GET /enrichments.
These read like static reference data and are not. What a caller may
place is decided per workspace by its permission-group integration
allowlist, per organization by which unreleased blocks have been
revealed, per deployment by ALLOWED_INTEGRATIONS, and per workspace
again by the workflows it has deployed as blocks. So all six are plain
defineWorkspaceOperation reads at minimumRole 'read' with
workspaceApiKey 'allow' — the exact policy of credentials.providers.list
— and every response keeps Cache-Control: private, no-store, because an
unrevealed preview block's existence must not leak across organizations
through a shared cache.
Trigger blocks ride as ?capability=trigger rather than a second
endpoint, and workspace custom blocks ride inside /blocks discriminated
by `source`, so "what may I place?" stays a one-call question.
The block projection is extracted out of the Copilot get_blocks_metadata
tool and rewritten onto @/tools/metadata and @/tools/metadata-outputs.
That cuts the tool's own @/tools/registry edge as a side effect: its
module graph drops from 6,756 to 1,318, and the new routes land at
1,673-1,734, next to the shipped /v2/credentials/providers baseline of
1,668.
Supporting changes:
- scripts/sync-tool-metadata.ts derives hostedApiKey ('always' |
'conditional' | 'none') from each tool's `hosting`. The config itself
stays excluded because it holds closures, but "does Sim host the key"
is a first-order authoring question, so the answer is emitted.
- getCopilotToolDescription takes hostedApiKey as an option instead of
reading `hosting` off the tool, so both an executable ToolConfig and
the generated metadata can answer it through one shared derivation.
- principalUserId / allowedIntegrationTypes move out of
lib/credentials/application/provider-catalog.ts into
lib/integrations/principal-scope.server.ts. Two copies of the
workspace integration gate would diverge first on the workspace-key
path, which has no user for permission groups to key on.
- scripts/check-tool-registry-boundary.ts walked page.tsx/layout.tsx
under app/workspace only, so a route importing the executable registry
passed green. It now walks a list of entry sources, seeded with the
four catalog route subtrees and the shared projection barrel. Routes
are covered per subtree rather than wholesale because 122 of ~1,130
route files legitimately execute tools.
Registry sweeps parse every block, tool, connector type, and enrichment
through its published response schema and compare against the wire
round-trip. They caught a real drift while being written: an operation's
inputs were typed as a union of the tool-param and block-input shapes,
and the union resolved to whichever member matched first, silently
dropping a block input's `schema`.
* docs(api): stop publishing a 413 GET /workflows/{id}/state cannot emit
* feat(v2): read upload-session state
Adds GET /api/v2/files/uploads/{uploadId}. Only DELETE was exported, so a
caller that lost track of a transfer could abort it but could not ask
whether the session was still alive, already finalized, or failed — the
resume story was missing.
Runs on a new files.upload.read operation at minimumRole 'read' rather than
reusing uploadCancel, which is a 'write': asking about a session must not
require permission to destroy it. The GET is a control leg like every other,
so it carries the signed upload token and re-authorizes the caller's present
workspace permission through reauthorizeWorkspaceUploadPurpose instead of
resolving the session on its id alone.
* fix(api): reconcile v2 catalog and workflow-authoring integration
Merging the catalog and workflow-authoring branches surfaced four issues
that neither produced in isolation.
- Route and OpenAPI counters were bumped to the same value on both
branches, so git merged them as one change while the merged tree holds
the sum. Corrects the route ratchet to 1142 and the workflows document
to 29 operations (152 total), then regenerates the OpenAPI documents
and the CLI surface from the reconciled contracts.
- The seven new workflow operations were published in the spec but absent
from the workflow API reference groups, which `check:openapi` rejects.
- `route-policies.ts` reached `WorkflowOperationsNotAppliedError` through
`apply-workflow-operations`, dragging the edit engine — and its diff and
comparison dependencies, which reach a client OAuth hook — into every
route that uses the shared workflow error policies. The class moves to
its own leaf module, mirroring `WorkflowImportError`, and each importer
now takes it from there.
- The operations route test shadowed that class inside its module mock, so
`instanceof` matched a fake and the assertion pinned a message the
production class never emits. It now uses the real class and asserts the
real message.
* feat(v2): extract ZIP archives over the public API
Adds POST /api/v2/files/{fileId}/extract and widens files.extract_archive
from principalKinds ['session'] / workspaceApiKey 'deny' to admit personal
and workspace API keys at the unchanged 'write' role.
The widening is an authorization change, so the justification lives in the
operation's TSDoc: extraction grants no capability an API key lacks, since
every file it writes could be created one at a time through files.create and
files.upload.create, both already 'allow' at the same role. It only collapses
many calls into one. The previous ['session'] restriction read as an artifact
of the UI having been the only caller. Delegated services stay out — no
copilot or executor caller exists and admitting one is a separate decision.
The response is counts plus the destination folderPath, never the extracted
files: a large archive would otherwise materialize thousands of objects into
one body. Callers page GET /api/v2/files?folderPath=... instead. The use case
returns the internal display path and the adapter projects it to a v2 path,
keeping the use case surface-neutral.
* feat(v2): extract file text over the public API
Adds GET /api/v2/files/{fileId}/text. Text extraction previously sat behind
checkInternalAuth on /api/files/parse, a route that also mixes in external-URL
fetching, execution-file upload, and multi-file aggregation, so it could not be
reused. The parse call is lifted into a thin application use case instead.
Runs on the existing files.read_content operation unchanged — it is already
workspaceApiKey 'allow' at the read role, and turning bytes it already
authorizes into text grants no further reach.
`degraded` is a required, non-optional boolean on the response. The legacy doc
and ppt parsers deliberately return best-effort or placeholder content rather
than throwing, so an omittable flag would let a client that never checks it
treat guessed text as extracted text. It is reported honestly rather than
converted into an error, because the parsers' behaviour is deliberate and
characterization-tested.
The read is bounded on its input at 25 MiB before extraction rather than on its
output after, given the parsers' documented DoS history; a caller may lower the
ceiling but never raise it.
* feat(v2): restore archived folders and list the archived set
DELETE /api/v2/files/folders archives recursively, so a recursive delete was
unrecoverable over the API: the archived files stayed visible through
GET /api/v2/files?scope=archived, but nothing could rebuild the folder
structure.
Adds POST /api/v2/files/folders/restore, path-addressed like the rest of the
v2 folder family, and a `scope` selector on the folder list so a caller can
find the archived path to hand it.
`scope` extends the files folder-list query rather than the shared
v2ListFoldersQuerySchema: only workspace files have an archived folder set, so
adding it to the shared schema would give tables, workflows, and knowledge a
parameter they ignore. GET /api/v2/files/folders is a FULL_SET_LIST, not paged,
so no cursor binding changes — list-pagination.test.ts passes unchanged.
Restore resolves the archived folder from its path by scanning the archived
set rather than walking the live tree, which by definition does not contain
the folder being restored. The folder-restored analytics hook now reports the
folder actually restored rather than the requested selector, which carries no
id on a path-addressed surface.
* feat(v2): bulk-download a file selection as a zip
Adds GET /api/v2/files/bulk-download, an adapter over the existing
downloadWorkspaceFileItems use case and its internal binary route.
Path collision: a static segment beside [fileId] permanently shadows a file
whose id equals it, and workspaceFileIdSchema does accept [A-Za-z0-9_-]+.
Rather than invent a new shape, this follows the existing bulk-delete sibling:
the hyphenated form cannot be produced by either minted id shape (UUID v4 or
wf_<shortId>), so the shadowed id is unreachable in practice. Documented on
the contract so the reasoning is not lost.
Folders are addressed by path, matching the rest of the v2 file surface. The
paths resolve against the folder set the selection already loads, so it costs
no extra query, and a path matching no folder is rejected rather than silently
dropped — a misspelled folder must not yield a zip of whatever else was
selected. The empty-selection and folder-count guards now account for
folderPaths, which a path-only selection would otherwise have tripped.
Selections are comma-separated only: v2 rejects a query parameter sent more
than once, so a repeated-parameter form would never reach the schema. Pinned by
a test so the contract cannot advertise a form the boundary rejects.
* feat(v2): expose run output files and optional inline bytes on the runs read
GET /api/v2/workflows/{id}/runs/{runId} now reports the files a run produced,
each with the downloadPath that fetches its bytes, and can inline them as
base64 on request.
Gated by includeOutput, matching `output`'s nullability: a caller that did not
ask for output does not receive a file list it did not request. The async
execute request's rejection of includeFileBase64 is deliberately left alone —
at submit time the run has not happened, so there is nothing to inline; reading
a finished run is the first moment the question means anything.
Inlining is capped per file at the executor's 16 MiB inline ceiling, which a
caller may lower but never raise. A file above it answers 413 naming that
file's downloadPath, so the caller is told exactly how to get the bytes rather
than being left stuck.
The descriptor deliberately omits the storage key — files are addressed by id
and the key is re-derived from the run's recording — and omits an expiry, which
the recording does not carry and which would be fabricated if published.
The route becomes headSafe: false, since inlining reads object storage. The
builder enforces that this requires the use case to expose authorize(), so HEAD
still answers from a real authorization rather than from authentication alone.
* feat(v2): permanently delete an archived file
DELETE /api/v2/files/{fileId} only archives — the OpenAPI says its stored bytes
are never removed — so there was no way to actually destroy a file over the API.
Adds the repository primitive, application use case, operation, and
DELETE /api/v2/files/{fileId}/permanent.
A distinct path rather than a flag on the ordinary delete: a query parameter
that turns a recoverable archive into an irreversible destruction is set by
accident, and the two acts carry different minimum roles, which one route
declaration cannot express. The file must already be archived; a live file
answers 409 naming the archive step, so no single request can turn a live file
into lost bytes.
minimumRole 'admin', which forces workspaceApiKey 'deny' since the workspace-key
ceiling is 'write' — the desired policy anyway: unattended credentials should
not destroy bytes.
Row first, then object. The two legs commit independently, so one can survive a
crash between them: deleting the row first leaves at most an orphaned object for
the storage sweep, while the reverse would leave a live row pointing at bytes
that no longer exist — a file that lists and opens but can never be read. A
failed object delete is therefore reported as objectDeleted: false rather than
thrown, because the request has genuinely succeeded once the row is gone. Both
directions are pinned by failure-injection tests, verified to fail when the
order is reversed.
Audited as a distinct FILE_PERMANENTLY_DELETED action, not a reuse of
FILE_DELETED, which records the recoverable archive step.
* feat(api): v2 log analytics, itemized cost, filters, and sortable query
Adds the aggregate and rich-read halves of the public logs surface, and
fixes three defects the existing reads carry.
Aggregate analytics. `GET /api/v2/logs/stats` returns time-bucketed run
counts, success rate, error count, mean latency, and the window bounds,
per workflow and for the workspace. The first-party route was a raw
handler with inline SQL and inline aggregation, so it is split into a
repository (`lib/logs/stats-queries.ts`), a pure aggregator
(`lib/logs/stats.ts`), and an application use case. That route keeps its
legacy authorization — it answers a caller without workspace access with
a zeroed 200, where v2 conceals the workspace as a 404 — and consumes
only the two surface-neutral halves.
`segmentCount` had no `.int()`, `.min()`, or `.max()`, so `0` divided by
zero and `1e9` allocated two billion-element arrays: both caller-reachable
500s. Bounded on both contracts. `workflows` is capped, with the workspace
totals still computed from every workflow and the cut reported as
`workflowsTruncated`.
Detail reads gain the itemized `cost.items` ledger (`null` and `[]` are
distinct answers and both reachable) and `workflowInput`, restoring a
v1→v2 regression.
The list gains `workflowName` and `status` filters, and `includeJobRuns`,
which unions Chat and Sim-agent job runs into the sequence behind a new
`kind` discriminator — without it a job run is indistinguishable from a
run whose workflow was deleted. A filter no job row can answer drops the
branch outright rather than meaning two things across the union.
`POST /api/v2/logs/query` carries the additional sort columns. `GET /logs`
is untouched: its single `order` param rests on there being exactly one
sortable column, and both escapes from that are ruled out, so the rich
read gets its own endpoint — the split the table surface already ships.
It uses the shared keyset scheme with the two nullable sort columns read
through a sentinel, since a keyset cannot compare against null.
`folderPaths` now covers a folder's whole subtree on the public path, as
it already did everywhere else; it previously omitted every nested run
with no error. The path strings did not change, so a folder-scope version
is stamped into the cursor and in-flight tokens restart rather than
silently skipping rows.
Also fixes `folderName`, which ILIKEd `workflow.name` — a copy of the
clause above it — and so searched workflow names instead of folders.
`buildLogSortCursorCondition`'s `IS NULL` disjunct is documented and
pinned: under `NULLS LAST` the null block is only reachable through it,
so removing it as a duplicate-row fix makes those runs unpageable.
Ratchets: route count 1142 -> 1144; logs OpenAPI operations 2 -> 4; total
operations 152 -> 154.
* feat(api): v2 tables run state, dispatch polling, batch update, bulk, archive
Closes the headless gaps on the v2 tables surface.
- Per-cell run state is now readable through an opt-in `includeRunState` on
`GET /rows`, `POST /query`, and `GET /rows/{rowId}`. The default projection
is byte-identical; a page whose sidecar outgrows its byte budget is a 413
rather than a silent truncation.
- Run dispatches are addressable: `GET /tables/dispatches/{dispatchId}`
publishes the column's full four-state domain so polling a finished run is
not a 500, and `GET /tables/{tableId}/dispatches` lists what is in flight.
- `POST /rows/batch-update` takes one distinct patch per row. Its transaction
moved out of the Copilot-only module into a surface-neutral use case both
surfaces now call.
- `GET .../enrichment/{groupId}` publishes the provider cascade, cost, and
timing behind one enrichment cell.
- `POST /tables/bulk-move` and `/bulk-delete` reach the existing bulk use
cases, which now accept folders by canonical path and resolve them inside
the application layer.
- `DELETE` is recoverable: `scope=archived` on the table list plus
`POST /tables/{tableId}/restore`.
* feat(api): expose knowledge chunks, tag writes, archive/restore on v2
Closes the knowledge cluster's remaining public-surface gaps.
Chunks: list/read/create/update/delete/bulk under
`/api/v2/knowledge/{id}/documents/{documentId}/chunks`. `queryChunks` gains
an `id` tiebreaker on every sort so the list pages on a keyset rather than an
offset — `tokenCount` and `enabled` are both non-unique, so a page boundary
inside a run of equal values used to repeat or drop the tied rows. The
internal offset caller is unchanged; the two positioning schemes share one
read.
Tag definitions: create, update, delete, next-slot, usage, and the
document-scoped save and cleanup. Without them a caller could write a tag
value into a slot with no definition and then had no way to name it, so
tag-filtered retrieval was unbuildable end-to-end. `v2KnowledgeTagSchema`
gains `id`, without which PATCH and DELETE are unaddressable. The
document-scoped DELETE is pinned to `action: 'cleanup'`: the domain's `'all'`
deletes the whole knowledge base's tag vocabulary from a document path.
Archive/restore: `GET /api/v2/knowledge/archived` as a sibling route rather
than a `scope` param — the two reads bind different operations and a v2 route
declares one — plus `POST /api/v2/knowledge/{id}/restore`. `knowledge.restore`
is a new workspace operation carrying `delete`'s policy, since an operation's
inverse must not be harder to reach; the internal session route now delegates
its workspace branch to the shared use case and keeps only the legacy personal
one.
Also: `POST .../documents/from-workspace-files` surfaces `addWorkspaceFiles`,
so a file already in workspace storage no longer has to be re-uploaded
byte-for-byte to be indexed; the `chunkingConfig` write widens to the
first-party five-key schema with its refines and separator bounds, while the
response stays `.catchall` so a legacy JSONB row cannot 500; and
`CONNECTOR_MANAGED_RESOURCE_READ_ONLY` joins `FORBIDDEN_DETAIL_CODES` now that
the bare 403 on connector-managed chunk writes is wire-reachable.
Document upsert is deliberately not included.
* feat(api): add v2 credential rotation and a gate-exempt capabilities endpoint
PATCH /api/v2/credentials/{credentialId} rotates service-account secret
material or renames a credential in place, preserving the credential id so
existing workflow, deployment, paused-run, connector, and webhook references
keep working. Re-posting to POST /api/v2/credentials answers 409, and
delete-and-recreate mints a new id, so rotation previously had no door.
The route is adapter-only: updateWorkspaceCredentialUseCase already owned the
rotation, its audit projection, and credentials.update. It gains one additive
assertedWorkspaceId field for the v2 workspace assertion, and the per-principal
credential-type table that deleteCredentialUseCase already applied is lifted
into requireManageableCredentialType so both operations share it. Without it a
personal API key could rename an env_workspace row and toV2Credential's throw
would surface as a caller-reachable 500.
CredentialProviderOperationError now maps to 503 with Retry-After when the
provider is unreachable, instead of the 400 its OrchestrationError('validation')
base projected. A transient outage rendered as a permanent input error invites a
caller to revoke a working credential.
GET /api/v2/meta reports the calling key's rollout cohort, type, and expiry.
It is the one route declaring the new typed gate: 'exempt' option, because the
rollout gate and the unknown-path catch-all answer byte-identical 404s and a
gated /api/v2/meta could never resolve that ambiguity. Authentication still runs
first, so the only fact disclosed is one about the caller's own credential.
* feat(api): publish deployment lifecycle and workflow-MCP v2 surfaces
Adds the four deployment-lifecycle operations v2 was missing, and the
workflow-as-MCP publishing surface, both as adapters over application use
cases that already existed.
Deployment lifecycle:
- PATCH /api/v2/workflows/{id}/versions/{version} relabels a version.
Deliberately not the internal route's body-shape dispatch between
"rename" and "promote to live".
- POST .../versions/{version}/activate promotes a version. Same use case
as rollback under a different transition, on its own path because the
two mean opposite things to a caller.
- POST .../versions/{version}/revert overwrites the draft. Accepts the
literal `active` alongside a version number.
- PATCH /api/v2/workflows/{id}/deployment toggles unauthenticated public
execution.
`workflows.public_api.update` widens from session-only to session plus
personal API key: it is an admin-role change the same accountable human
may make from a script. Workspace keys stay denied. Its EE refusal now
carries PUBLIC_SHARING_NOT_ALLOWED instead of a bare forbidden.
Workflow MCP servers:
- /api/v2/workflow-mcp-servers list, create, update, delete, plus
publish and unpublish of a workflow as a tool. Named apart from
/api/v2/mcp-servers, which registers the external servers Sim calls.
- The six mcp_servers.workflow_deployments operations widen from
['delegated'] to admit sessions and personal API keys; roles and the
workspace-key denial are unchanged.
- The server list gains keyset pagination, matching its external
sibling, since nothing caps how many a workspace publishes.
- Server, tool, and workflow reads move out of the use case into
lib/mcp/queries.
Route ratchet 1150 -> 1160; OpenAPI operations 161 -> 171.
* feat(api): extract chat deployments and publish the v2 surface
Chat deployment was a shipped module with no public API and two
authorization systems: `lib/workflows/application/chat-deployments.ts`
had deploy/undeploy extracted, but only Copilot used them — the REST
routes reimplemented workflow authorization inline, and `PATCH
/api/chat/manage/[id]` additionally owned password encryption, the
auth-type field-clearing matrix, identifier uniqueness, the
redeploy-gating protocol with two 409s, a raw db.update, and a manual
recordAudit.
New `lib/chat-deployments` domain:
- `chat_deployments.list/read/update/delete`, keyed on the deployment
whose workspace is derived by joining its workflow. Creation stays
`workflows.chat.deploy`, which is keyed on the workflow.
- The PATCH extraction, including the field-clearing matrix and the
asynchronous-cutover invariant the route had hand-mirrored from
`performChatDeploy`.
- One `buildChatDeploymentUrl`, replacing three constructions that had
already drifted onto two different host helpers. There is no chat
subdomain, so nothing publishes a host.
- Repository reads moved out of the use cases into
`lib/chat-deployments/queries`.
Internal routes are now adapters over those use cases. `GET /api/chat`
is deliberately not migrated: it scopes by `chat.userId` while every
other chat operation authorizes by workspace admin, and reconciling the
two is a product decision. `PATCH` keeps its 400 for an identifier
collision through a typed `ChatIdentifierInUseError`; v2 reports the
409 the condition actually is.
v2 surface at `/api/v2/chat-deployments`: list, create, read, update,
delete. Workspace-scoped, keyset-paged, and a stored password is never
readable — reads carry `hasPassword` only, and the session-only reveal
endpoint deliberately has no v2 counterpart.
Also: an email- or SSO-gated chat with an empty allow-list is now
refused in the use case rather than only at the internal boundary, since
it is unenterable; and the doc comment on `processHostedKeyCost`
claiming a `usageLog` write is corrected — no such write exists.
Route ratchet 1160 -> 1165; OpenAPI operations 171 -> 176.
* fix(api): close three review findings, two of them caller-reachable
- Run output files are filtered to keys under the run's own execution
prefix. The recording they came from is not a trustworthy key source:
the start block copies every caller-supplied input field verbatim into
its output and `collectUserFilesById` accepts anything carrying the
`UserFile` shape, so a caller could name any storage key and have the
download and base64 paths — neither of which authorizes per file — serve
it back.
- `getBlock` reads own keys only. `BLOCK_REGISTRY` is an object literal,
so `constructor`, `toString` and friends returned inherited functions
that every consumer then treated as a block, turning a path segment into
a 500. `getToolMetadata` already guarded this way.
- A folder-scoped log page no longer unions in every job run in the
workspace. The guard read `filters.folderIds`, which the public surface
never sets — it carries the folder filter in `folderScope` — so the page
contradicted the contract's promise that job runs are dropped whenever a
filter they cannot answer is set.
Also: the log cursor stamps `includeJobRuns` only when it is on, so its
`.default(false)` no longer puts a constant in every fingerprint and
rejects cursors minted before it existed; and the `folderName` subquery is
scoped to the workspace and to workflow folders instead of scanning the
whole `folder` table.
* fix(api): close two more review findings, one an authorization bypass
- `workflows.operations.apply` no longer admits a workspace API key. The
use case authorizes against three per-user policies — the EE permission
config, block visibility, and credential reachability — and all three
take a human subject. An actorless key has none, and both substitutes
fail open: attributing to the workspace billing owner evaluates the
batch as the least-restricted account in the workspace, and passing no
user makes `getUserPermissionConfig` return `null`, which every caller
reads as unrestricted. Either way a workspace constrained by an
allowlist was edited as though it were not. Personal keys keep the
capability, so headless editing is unaffected for a credential that
names a human.
- `GET /workflows/{id}/state` reads its variables through
`parseWorkflowVariables`, and the stored variable response schema drops
the two assertions the column cannot honour. The column has carried a
JSON string and a legacy array as well as the current record, the
realtime `variable.add` op types `type` as `z.any()`, and the parser
writes `name` through verbatim — so the input bounds on the read turned
a stored workflow into a 500 on the endpoint that opens it. The write
schema keeps them, which is where they can still be honoured.
- `GET /workflows?scope=archived` projects folder paths tolerantly.
Archiving a folder cascades onto the workflows inside it but leaves
their `folderId` dangling — which is why restore has to null it — so the
strict projector threw a bare `Error` and took the whole page down with
no cursor able to step past the row.
* fix(files): bind Start-block file keys to the executing workspace
The Start block derived a file's storage key by parsing the caller's own
`url`, which `isInternalFileUrl` matches on any host and
`extractStorageKey` returns verbatim — so a request body could name any
tenant's bytes. The key is now accepted only when its own layout names
the workspace the execution runs in, and every file is dropped when the
execution carries no workspace.
Also bounds `includeFileBase64` with an aggregate response ceiling and a
worker pool instead of an unbounded `Promise.all`, scopes the bulk
download's authorization resource to the workspace when folder paths are
requested, makes the folder-restore selector mutually exclusive at the
type level, and names the bound in the `maxBytes` validation message.
* fix(api): close v2 log review findings
Cursor scope: `scope` on the workflow and table lists carries
`.default('active')`, so it entered every fingerprint as a constant and
refused every cursor minted before the param existed — with the
"cursor does not match the requested filters" 400, which is actively
misleading for a caller that changed nothing. Both now stamp the
default as absent, so only a caller who asked for `archived` gets a new
sequence.
Dashboard stats: `maxWorkflows` capped the response, not the
allocation. Segment series are now densified after the cut instead of
before, so returning 200 series no longer materializes one
`segmentCount`-length array per workflow in the window. The aggregate
still sums every workflow, now from the sparse per-workflow maps.
Cost keyset: `cost_total` is an unconstrained `numeric`, so its anchor
travelled through `Number()` and was compared back at full precision —
rows differing beyond float64 collapsed onto one anchor. Adds
`decimalKey`, which carries the digit string and binds it `::numeric`.
Run detail: `cost_total` is a backfilled projection, so a run predating
the backfill reported `cost: null` even with a real ledger, making
`items` unreachable for exactly the runs the ledger explains. Falls
back to the ledger total.
Also caps the log folder-path index reads at MAX_FOLDERS_PER_WORKSPACE
like every other reader, publishing the folder-tree 413 on the four log
operations; reverts a dead `status` widening in `v2CommaListSchema`;
drops an unread `executionData` select; corrects the segment-count and
searchLogs prose; and replaces the sort-cursor SQL-text assertions with
a two-page walk over a fixture with a null block.
* fix(api): close knowledge v2 review findings
- widen knowledge.list_archived to the delete/restore policy so a workspace
API key can discover what it may restore
- escape LIKE wildcards on the now-public chunk search
- derive tag slot capacity from TAG_SLOT_CONFIG per field type
- type updateKnowledgeBase's chunkingConfig as ChunkingConfig and project
every declared field explicitly
- attribute a restore to the calling surface instead of a literal 'api'
- gate 'knowledge chunks batch-update' behind --yes, since it can delete
- present the tag-cleanup action from the parsed request rather than
faulting on the domain result after the delete committed
- unbind asserted-scope workspaceId from the nested knowledge cursors,
matching the table-row lists
- add executed-SQL coverage for the chunk keyset
* fix(catalog): close the catalog and registry-boundary review findings
The module-graph ratchet treated an entry with no baseline row as
informational, so the six catalog routes and the projection barrel were
unratcheted while the summary still read "within their module-count
baseline". An unbaselined entry now fails --check, the summary counts only
what was actually compared, and the baseline is re-recorded.
The Copilot block-metadata tool — the reason the shared projection exists,
6,756 modules down to 1,321 — was in no guarded subtree. It is now an entry
source and a catalog boundary root.
Catalog behaviour:
- hostedApiKey is gated on the deployment, so a self-hosted install reports
none instead of promising 127 tools' keys it will never supply
- block detail resolves an unversioned base type to its newest version and
projects through the viewer's visibility, so it can no longer 404 a block
the list contains or name it differently
- offset-cursor ordering compares code units rather than the process locale
- projections copy every array they publish instead of handing out the
registries' own
- an options function returning a thenable throws rather than silently
widening the providers-store substitution across the event loop
- a throwing block projection costs the Copilot tool one block, not all of them
- the trigger-kind log returns to debug: chat/manual/api are entry-point
kinds, not authoring defects
Also sweeps the custom-block detail branch against its response schema,
guards each projection module rather than the dead barrel over them, and
drops a provably dead branch in processHostedKeyCost.
* fix(workflows): close v2 workflow-authoring review findings
- Read a blockless draft back as an empty graph. `PUT /state` of
`{ blocks: {}, edges: [] }` — the contract's own published example —
deletes every block row, and the loader answers `null` for a blockless
workflow, so the following `GET /state` answered 404 while the list
endpoint still showed the workflow. Existence is the workflow row's to
decide; the null is now projected as an empty graph.
- Rewrite the `readWorkflowGraph` authorization test so it can fail. It
called `authorize?.()` and asserted only a negative, so deleting
`authorize` or replacing it with a no-op both passed — the invariant the
head-safe `HEAD` path depends on.
- Route `setWorkflowBlockEnabled` through `replaceWorkflowNormalizedState`,
the same door the other two graph writes use, instead of writing the
normalized tables itself without state preparation or custom-tool
extraction.
- Count applied operations directly. Enablement refusals landed in the same
skipped-item array and were subtracted from the operation count, which
`Math.max(applied, 0)` then masked when it went negative.
- Give a `disabled_ancestor` refusal its own member of the published skip
enum instead of reporting it as `block_locked`.
- Refuse an `atomic` batch whose credential or hosted API key would be
stripped, and carry the dropped inputs in the 409 details.
- Publish the whole lint report — `sources`, `sinks`, `orphanBlocks`,
`emptyOutgoingPorts`, `invalidBranchPorts`, `invalidConnectionTargets`,
`fieldIssues`, and the `kind` discriminator on unresolved references —
rather than only free-text reference prose.
- Stop reporting unresolved lint references as `inputValidationErrors`.
`collectUnresolvedReferences` is read-only, so those values stay
persisted; they were double-reported, and falsely as dropped inputs.
- Replace two unfalsifiable negative-principal tests, which used a
principal kind `Exclude`d from `PrincipalKind`, with a reachable one.
- Nits: drop a stranded TSDoc block; assert the membership predicate in the
selector-validator admin test; make the HEAD test assert a representation;
assert the sanitized graph is what `replaceWorkflowState` writes; add a
route test rejecting `baseGraph` in a v2 body; carry
`principalAuditSource` on restore/duplicate/moveBulk audit; unify the two
`base64MaxBytes` ceilings on `MAX_INLINE_MATERIALIZATION_BYTES`.
* fix(api): close seven v2 review findings, one an authorization bypass
Raise mcp_servers.workflow_deployments.update_server to admin: its body
carries isPublic, and a public server executes with no Sim credential, so a
write member could remove authentication from every workflow it publishes.
create_server already grants the same visibility at admin.
Pin the widened operations in registry tests — the six workflow-MCP ones,
the four workflow widenings, and files.extract_archive.
Reject secret fields on a credential that has no rotatable secret instead of
dropping them behind a 200, classify a non-transient provider 4xx as caller
error rather than a retryable outage, and reconcile a provider outage to 503
with Retry-After on all three surfaces.
Give /v2/meta a declarative principal policy through a new defineOperation
factory, carry the key expiry on the auth context instead of reading the
api_key table from the application layer, and make the impossible principal
branch an invariant error rather than a codeless 403.
Enforce the rollout-gate exemption at definition time, against the one
contract path it is reserved for, and remove the gate parameter from the
exported admission helper so the builder is its only door.
* fix(tables): bound the run-state sidecar, and close six v2 review findings
Enforces the 2 MiB run-state ceiling INSIDE the sidecar drain rather than
over its materialized result, refuses the unbounded query form paired with
it, and normalizes the two stored blobs the v2 surface publishes from bare
`as` casts.
- The run-state budget now travels into `loadExecutionsByRow`, which drains
row ids in bounded chunks and refuses before fetching the next one. The
post-hoc `requireBoundedRunState` walk is gone: it measured a spike that
had already happened, and re-serialized every entry to do it.
- Both row reads that accept `includeRunState` cap the page at
`V2_MAX_RUN_STATE_ROW_LIMIT`, and `POST /tables/{id}/query` additionally
refuses the flag paired with `limit: 0`.
- `runState.status` and the enrichment cascade blob are projected onto the
published shape before presentation; both were caller-reachable 500s on a
well-formed read.
- `sim tables bulk-delete` now gates behind `--yes`, and the CLI sweep that
should have caught it covers destructive non-DELETE forms.
- `POST /tables/{id}/restore` is idempotent (200, no audit) like its
knowledge sibling, and bulk folder selection deduplicates after resolution.
- The batch-update backstop keeps the looser Copilot ceiling and says so in
TSDoc: it is a backstop no surface reaches, because the contracts stop a
v2 caller at 1000 and the Copilot tool stops itself at 5000. Each caller
sees the bound that actually applies to it; neither surface's cap moved.
* fix(chat-deployments): close v2 chat review findings
Fixes the chat-deployments slice of the v2 review, several of which are
regressions the application-operation extraction introduced.
- Stop a `500` on schemaless JSONB: the response now declares a stored
shape without bounds and `toV2ChatDeployment` projects
`customizations`, `outputConfigs`, and `allowedEmails` onto it. The
request schemas keep `.strict()` and their bounds.
- Restore the specific validation message on `POST /api/chat` and
`PATCH /api/chat/manage/[id]`, and the deleted test that pinned it.
- Restore `chat_deployments.read` to workspace `admin`; the detail read
serves the visitor gate.
- Narrow the list projection so `chat_deployments.list` can stay a
`read` operation reachable by a workspace API key: `allowedEmails`,
`hasPassword`, and `customizations` are gone from the list entry and
available only from the admin-gated detail read. Serialized field by
field so a field added to the detail shape cannot reach the list by
default.
- Classify create-path failures: `performChatDeploy` carries an
`errorCode`, so an in-flight deployment is a `409` and an invariant
failure a `500` instead of every refusal being a `400`.
- Delete the callerless `GET /api/chat`, which served the encrypted
password column with no response contract.
- Propagate undeploy infrastructure failures instead of concealing them
as `404`, and return `ChatDeploymentView` from both delete paths.
- Name `CHAT_AUTH_MODE_NOT_PERMITTED` on the create path.
- Guard `getBaseUrl` inside `buildChatDeploymentUrl`, which otherwise
throws on a self-host with no `NEXT_PUBLIC_APP_URL`.
- Correct the published allow-list claim: a replacement `allowedEmails`
is applied after the auth-type clear, so it does survive.
- Assert `workspaceId` on the v2 detail routes and reconcile the
concealment TSDoc with what the error policy actually renders.
- Move `resolveActiveWorkspaceApplicationContext` to the workspaces
domain so chat-deployments no longer imports workflow application code.
* test(credentials): pin the reconciled provider-outage status
The internal route alone answered 502 where the v2 surface, the shared
status helper and `PROVIDER_OUTAGE_CODES`' own TSDoc all say 503. The
test pinned the divergence; it now pins the reconciliation, including the
`Retry-After` a 503 carries. Corrects a stale comment that still named
502 as the value callers see.
* fix(executor): restore cloud-storage Start files, dropped by the key rule
The ownership check accepted a key only when it could be parsed out of an
internal `/api/files/serve/...` URL. But the server-side uploader for run
inputs returns a *presigned cloud* URL whenever object storage is
configured, whose path is the bucket key — so every chat-deployment
attachment, API `files[]` payload and generic-webhook file field resolved
no key, and because normalization is all-or-nothing the entire `files`
input was dropped with no error. It passed locally and under vitest only
because the uploader falls back to an internal URL when no object storage
is configured, which is exactly why no test caught it.
The test is ownership, not provenance: a key is accepted when its own
layout names the executing workspace, whether it arrives directly or is
parsed out of the URL. Neither field has to be trusted, since both are
caller-authored and both are held to the same check. A payload whose key
and URL disagree is refused rather than resolved in the caller's favour —
a genuine uploader writes the two consistently, so only a forged pairing
is turned away.
`context` is now derived from the accepted key rather than read from the
payload or the URL's `?context=`, so an owned key can no longer be
labelled with a bucket its bytes do not live in — the hardening the
previous comment claimed but did not perform.
* fix(api): close four defects the fix pass introduced
- `resolveLatest` built a `RegExp` from the caller's block id and read the
registry with a bare lookup, so the catalog detail route — moved onto it
by the version-alias fix — routed around the `ownBlock` guard added for
exactly this. `GET /api/v2/blocks/%5B` was a `SyntaxError` 500 and
`.../constructor` an inherited function. Matched by string comparison
now, the way `tools/tool-ids.ts` resolves the same convention, and read
through `ownBlock`.
- The run-state byte budget was applied inside `queryRows` rather than at
the callers that publish it, so the first-party table grid — which reads
run state at five times the row limit and publishes no ceiling — turned
a large page into a hard failure, with an error naming a parameter it
does not expose. The budget is now an explicit option the public reads
pass and internal callers omit.
- Three graph-write CLI commands shipped ungated because the sweep meant
to catch them matched only the names already enumerated, so it could
never fail. It now forces every non-`GET` operation into a destructive
or non-destructive list, and the three carry confirmations.
- Unbinding `workspaceId` from the knowledge-documents cursor was right on
the merits and wrong in effect: the value is constant per sequence, so
removing it changed the fingerprint and refused every cursor already in
flight. Restored there; the chunks list is new in the same change and
keeps the cleaner reading.
* fix(api): resolve a detail read to a version the viewer can see
`getLatestBlockForViewer` took the newest version and then hid it, which
inverted the contradiction it was written to close: `slack_v2` and
`table_v2` are preview-gated while their v1 deliberately stays in the
toolbar, so an unrevealed viewer got a `404` on a detail read for a type
`GET /api/v2/blocks` was listing in the same breath. It now walks versions
newest-first and answers with the first one visible to that viewer.
Also:
- The chat password guard ran after `performFullDeploy`, so a request that
could never succeed burned a real workflow deployment version and then
answered 400. Its two sibling gate guards already refuse ahead of the
deploy; this one now does too.
- The Copilot sub-block serializer published the registry's own `options`
and `dependsOn` arrays by reference. Pre-existing, but the catalog
projection this parallels copies every array it publishes precisely
because they are process-global and shared by every request.
* fix(api): restore the locked read-modify-write and the password validator
Two findings verified as real regressions against staging, out of ten
checked — the rest were pre-existing, latent, or false.
`setWorkflowBlockEnabled` read the graph outside the row lock and wrote it
back inside a later transaction. The editor's own save takes that same
lock, so an autosave committing in the window was silently discarded: this
operation writes a whole graph, not a delta. The persistence primitive now
accepts a reader that runs after the lock is taken, and the toggle
re-reads and re-decides there. Its lock predicate is also scoped to the
workspace and to a live row again, so a workflow archived mid-flight is
refused rather than written.
The v2 chat-deployment contracts inlined their own password rule twice
instead of using `chatDeploymentPasswordSchema`, losing the refusal of a
whitespace-only password — which the internal contract rejects precisely
because it strands the deployment behind a password the visitor form will
not submit. Both sites use the canonical validator now.
* fix(api): one folder projection, one dynamic-provider list, honest 413s
- `toV2Folder` existed twice, and the second copy had been written without
the name/path invariant — so a row the list read refuses loudly would
have been served with a mismatched pair by the restore read. One
definition, guard included.
- The catalog projection restated `DYNAMIC_MODEL_PROVIDERS` and had
drifted by one member. Derived from the canonical list instead.
- The tables reads documented a `413` for run state that they cannot emit
— the budget became opt-in, and the row limit is the bound now — so the
claim is removed rather than declared. The workflow run read has the
opposite problem: it genuinely emits one, on a single file *or* the
run's inlined total, and declared neither. Now declared, and the
sentence covers both.
- Reclassifies the operations staging added into the destructive sweep, so
the triage stays exhaustive.
* fix(v2): classify storage and uniqueness failures, drop permanent file delete
- remove the permanent file-delete endpoint; the platform offers no such
action in the UI, and its manager wrote outside a transaction with no
storage accounting
- extract a generated document's text from its compiled artifact rather than
its generation source, matching the download path; a `.pdf` source was a
500 and a `.docx` source returned generator JavaScript as clean content
- report a run file whose object retention has already swept as 404 rather
than 500, on both the inline base64 read and the download stream
- report a knowledge tag that loses at a unique index as 409, naming whether
the slot or the display name is taken
- gate `workflows versions revert` behind a CLI confirm; it overwrites the
draft graph and was classified non-destructive
* feat(v2): report lint from both graph writes and add dry-run previews
- `PUT /workflows/{id}/state` now returns the same `lint` report as
`POST /operations`; an agent authoring a graph from scratch needs the
findings at least as much as one editing incrementally
- extract the report into one shared builder so the two writes cannot drift,
and one shared presenter so the wire shape is identical
- skip the credential/tool reference pass when the caller has no human
subject, rather than resolving it against the workspace billing owner:
that would misreport what the workflow can reach and disclose another
person's grants. `lint.notes` says when it was skipped
- add `?dryRun=true` to both graph writes: validates and lints, persists
nothing, records no audit, notifies nobody. A query param, not a body
field, since the body of a PUT is the resource itself
- CLI: a dry run no longer demands `--yes`; requiring confirmation to preview
a change teaches callers to pass `--yes` reflexively
- CLI: name the graph commands for their verbs — `workflows state get`,
`workflows state replace`, `workflows operations apply` — instead of the
derived `state list` / `state update` / `operations create`
- document when to use `rollback` vs `versions/{version}/activate` on both
* chore(docs): sync generated docs manifest for the new CLI pages
* feat(v2): add the missing workflow-MCP reads and align bulk naming
- add `GET /workflow-mcp-servers/{serverId}` and
`GET /workflow-mcp-servers/{serverId}/tools`. The resource could be
PATCHed and DELETEd but never read, and its tools could be published and
unpublished but never listed — the server list reports tool names only, so
nothing published the `workflowId` that addresses a tool for deletion.
Both mirror `mcp-servers` beside them, and carry that family's
workspace-API-key denial rather than the wider `mcp_servers.read` policy
- rename `POST /tables/bulk-move` to `POST /tables/move`, so tables matches
the shipped `files` resource exactly (`move` + `bulk-delete`)
- name the CLI commands for their operations instead of the derived
`... create`: `tables move`, `workflows move`, `tables bulk-delete`, and
`tables rows update-each` for the per-row batch, which sits beside the
existing filter-based `tables rows batch-update`
`POST /tables/{id}/rows/batch-update` keeps its name: a distinct payload per
resource is precisely AIP-234 BatchUpdate, and `bulk-` would have collided
one word away from the filter form.
* fix(v2): correct documented statuses and a caller-reachable 500
- duplicating a workflow into a locked destination folder answered 500:
`FolderLockedError` is a plain Error carrying `status = 423`, which the v2
error policy does not classify. Converted to OrchestrationError('locked')
at the application boundary, matching the bulk-move path
- restore workflow promised a 413 for an oversized folder tree that its
response list never published; the cap is real, so the status now is too
- move workflows and apply variables documented 409/423 they cannot emit:
every per-item lock and conflict is reported in `failed`, not thrown
- bulk download and delete knowledge tag can both 409 and did not say so;
cleanup tag definitions cannot and did say so
- apply workflow operations denies workspace API keys but never documented it
- the dry-run responses are not byte-identical to a committed write:
`needsRedeployment` describes the pre-write state and persistence warnings
cannot appear. Reworded rather than overclaimed
- read file text and get file upload are head-safe, so the "HEAD skips the
effect" sentence did not apply to them
* fix(v2): guard tag field-type changes and publish the 415 every body route can return
- `PATCH /knowledge/{id}/tags/{tagId}` accepted a `fieldType` incompatible
with the slot the tag already occupies. Slots are enumerated per field
type, so a text tag could be relabelled `number` and every later read
would interpret its values as the wrong type. Create checked this; update
now runs the same two checks
- derive `415` from the contract the way `413` already is: the JSON builder
answers UNSUPPORTED_MEDIA_TYPE for any body under a content type it cannot
read, so all 100 body routes could return a status none of them published
- give version activation its own result component instead of publishing it
as `RollbackResult`; the shipped rollback keeps that name
- correct descriptions that promised behaviour the code does not have: a
`processingStatus` field never returned, a `gmail_send` resolution example
that short-circuits, bucket widths that overflow the window, a bulk tag
save that relocates rather than overwrites, and per-server tool names
actually gathered under a page-wide budget
- drop 409 from three knowledge and upload reads that cannot emit it
* docs(v2): correct the upload transfer contract and 16 other published claims
The upload transfer step was documented as Sim's own data plane on every
deployment: "success is 204" and "a failure is the v2 error envelope". That
holds only when Sim stores objects itself. With object storage configured the
URL is the provider's presigned URL, so S3 and GCS answer 200 and Azure 201,
and a failure is the provider's XML — a client written to the old text reads a
successful cloud upload as a failure. Also states that part ETags do not need
retaining: completion takes no body because Sim lists the parts from the
provider itself.
Other corrections, all to shipped descriptions rather than behaviour:
- DELETE table and bulk-delete files archive rather than erase, and neither
said so; bulk delete also cannot emit the 409 it declared
- complete knowledge upload published a 402 only the create leg can raise
- billing status conceals a foreign workspace id as 404, not the 403 its
TSDoc and description both claimed
- audit entries null a folder's resourceId and strip folder ids from metadata
at every level; neither redaction was documented
- details=full adds the workflow summary to workflow runs only, never to job
runs; GET /logs folderPaths covers a subtree like its two siblings; getLog
now carries the retention sentence
- list secrets returns description too, and the logs and resources documents
described only part of what they serve
* improvement(api): consolidate the v2 surface and close seven defects
Endpoint consolidation:
- Fold POST /logs/query into GET /logs; add sortBy/sortOrder, cap the
comma lists, and move the list onto the shared keyset codec
- Fold GET /knowledge/archived into GET /knowledge?scope=archived,
matching files, tables, and workflows
- Re-home chat deployments as a singleton under the workflow they
belong to; keep the workspace-scoped discovery list
- Move the tag-definition writes off the document path onto
/knowledge/{id}/tags, where they already acted
- Nest the table export and dispatch reads under their parent table
Defects:
- Publish isPublicApi on the deployment read; it was write-only, so a
workflow could be opened to unauthenticated execution unauditably
- Stop publishing raw storage keys and an unusable URL in log files
- Classify a chat-identifier unique violation as 409 rather than 500
- Fall back to the root path instead of throwing when a knowledge
base's folder is archived
- Cap bulk-download at the ceiling it actually enforces
- Normalize variables through one helper on both graph write paths
- Fix a folder-name log filter that matched workflow names
Naming and gaps:
- Rename /files/{id}/extract to /unarchive, /rows/find to /rows/search,
/rows/batch-update to /rows/bulk-update, /columns/run to /dispatches
- Type the last six generic [id] path segments
- Add table folder restore and id-addressed dispatch cancel
* chore(audits): record the v2 catalog routes in the boundary baseline
* fix(api): accept a null chat password and correct three published claims
- performChatDeploy validated `password: null` as a password, so the
replace-shaped chat PUT answered 400 for every mode that owns no
password — public (the default), email, and sso. The declared payload
type has always allowed null, and the stored value is cleared by
authType regardless, so null needs no validation of its own. The route
test could not catch it: it mocks the orchestration module and pinned
the exact null the real guard refused.
- Redirect the two docs slugs this branch retired that were genuinely
published: findTableRows and runTableColumns.
- The table folder restore described an idempotent no-op for an already
active folder; it answers 404. Say so, and say where the path comes
from, since the tables folder list cannot yet report archived folders.
- Name the customizations exception to the chat PUT's replace semantics.
- A cursor-binding case used status=error, which is a level and not a
status, so it failed contract validation and never reached the cursor
check. Use an accepted value and pin the reason, not just the status.
* chore(cli): classify the new v2 chat operation as non-destructive
* feat(cli): expose canonical resource URLs
* fix(api): close three caller-reachable failures found by the final probe
- GET /files/{fileId}/text called parseBuffer unguarded, and parseBuffer
signals every failure as a bare Error that no v2 policy classifies. A
zero-byte upload or a mislabelled archive was an unhandled 500. Empty
bytes now answer empty text — a zero-length file has no text — and
unparseable bytes answer 409, matching the rendered-artifact resolver.
- GET /workflows/{workflowId}/state asserted write-side bounds over
stored data. workflow_blocks.name and .type are bare text() and the
realtime rename op accepts z.string(), so a block renamed past 255
characters made the workflow unreadable, and unrepairable, over v2.
The read shape now takes the same input/stored split the variable
schema already had. Stored subflow conditions are coerced in the
loader beside the existing numeric guards.
- GET /knowledge stamped scope into the cursor fingerprint
unconditionally. scope defaults to active and is new on that list, so
every cursor the deployed build handed out would have been refused
with a message saying the caller changed a filter they never sent.
Its siblings already carry the guard and the comment.
Also: POST /workflow-mcp-servers answers 201 like every other v2 create;
GET /logs/stats reuses the log list's entry ceilings; the execute and
resume routes install the media-type-aware 415 they publish; and a
rationale citing an endpoint that never reached the wire is corrected.
* fix(tables): carry the dispatch terminal timestamps through the stale sweep
Staging's abandoned-dispatch recovery builds its own `DispatchRow`, and
this branch had added `completedAt`/`cancelledAt` to that shape for the
id-addressed dispatch read and cancel. The merge was textually clean and
left the new mapping short two fields.
* fix(workflows): deny workspace API keys on the graph replace
`PUT /workflows/{workflowId}/state` stores blocks and their tool wiring
wholesale, but the policies deciding which of those a member may add —
the EE permission config and block visibility — take a human subject.
A workspace API key has none, and both substitutes fail open: the
billing owner is a different, typically less-constrained person, and
passing no user makes the permission lookup return null, which every
caller reads as unrestricted.
That made the replace a second graph-write door storing what its sibling
`POST /workflows/{workflowId}/operations` refuses, which denies workspace
keys for exactly this reason. Both doors now agree. Personal keys keep
the capability, so headless authoring is unaffected for a credential
that names a human.
* chore(api): drop the enrichment catalog endpoint
GET /api/v2/enrichments listed the code-defined table enrichments. The
per-row enrichment run detail stays; only the catalog read goes.
Removes the route, contract, response and query schemas, the semantic
operation, the use case, the projection module, its registry-boundary
entry, and the CLI command. The "not found" and "blank search" cases it
covered are repointed at the connector-type sibling so the shared
behaviour stays tested rather than deleted with it.
* improvement(api): make the workflow operations endpoint self-describing
Two things stood between this endpoint and a caller who has only the
published spec.
The accepted `params` keys existed only in the edit engine's source. The
spec said "the accepted keys depend on the target block type", so a
caller reading it could create a nameless empty block and nothing more —
while the Copilot tool catalog, over the same engine, has always spelled
out the envelope. That guidance now lives in the contract, shared by the
add, edit, and insert_into_subflow parameter schemas so the two surfaces
cannot describe one engine differently: `inputs` keyed by sub-block id,
`retry`/`triggerMode`/`advancedMode` beside it rather than inside it,
`connections` keyed by source handle, and `removeEdges` for dropping one
edge without restating the rest.
A `block_id` that is not already a UUID is replaced with a minted one,
and the mapping was computed and then dropped. A caller could not
reference the block it had just created except by re-reading the graph
and matching on name. The engine now returns it and the response
publishes it as `mintedBlockIds`, with the in-batch versus cross-request
rule stated in the operation description.
* fix(api): close the pre-merge scan's blocking findings
Docs and public wire, all of it permanent surface once released.
- Two published tag groups, Catalog and Meta, had no sidebar entry in any
locale, so six operations shipped unbrowsable. Added to all six, and a
test now fails when a published tag has no entry.
- deleteWorkflowChatDeployment pointed callers at DELETE on /deployment;
the undeploy verb is on /deploy.
- PUT /state still promised workspace API keys a degraded lint pass after
the operation began rejecting them outright. It also lived in a
single-quoted string, so the shared clause would not have interpolated.
- POST /tables/move took targetFolderPath as nullable-but-required, which
rendered the CLI flag as a required `<json|@file>`: `--to /Archive`
failed to parse and omitting it failed outright, while the docs said
"omit for root". Now optional and a plain string, matching
POST /files/move; the route supplies the null the use case wants.
- Table dispatch status and row run state published `cancelled` beside
imports, exports, and job state publishing `canceled`, and the note
explaining the split was wrong about its own sibling. Both new schemas
now publish `canceled`; the stored column is unchanged and mapped at
the presenter. The shipped `cancelled` count field is left alone.
- applyWorkflowVariables can answer 423 and both workflow-MCP deletes can
answer 409; none declared it. Hand-assembled error lists replaced with
the shared sets.
- /files/{fileId}/unarchive became /unzip. `extract` reads as "extract
text" and `unarchive` reads as the inverse of restore on a resource
where archived means soft-deleted; unzip collides with neither and is
what the implementation calls itself.
* improvement(cli,docs): finish the naming and clear the enrichment leftovers
- Four single-record GETs derived to `list` while returning one thing:
meta, a workflow's chat deployment, log stats, and file text. Renamed to
`meta status`, `workflows chat status`, `logs stats`, `files read`,
matching the `workflows deployment status` correction already in this
branch. None of the old spellings shipped.
- Dropped five `renamedFrom` aliases pointing at spellings that never
existed, each of which built a hidden command and a permanent
deprecation warning for argv nobody could have typed. `tables rows find`
keeps its alias — that one really shipped.
- Removed the enrichment catalog from three prose sites left behind when
the endpoint went: the resources spec description, its Catalog tag, and
the contract and pagination-test comments.
- Documented the six new command groups in the CLI index table. That page
is a guide page, so the docs staleness check cannot flag it.
- Published the `customizations` exception to the chat replace semantics.
It was in the route TSDoc and invisible to every caller reading the
spec, which is where the claim "Replace, not merge" is made.
* improvement(cli): use batch- for the tables bulk delete, matching its siblings
The CLI renames a bulk form only when it would collide with its singular
sibling, and uses AWS's `batch-` prefix when it does — `files
batch-delete`, `tables rows batch-delete`, `knowledge chunks
batch-update`. `tables delete` exists, so `tables bulk-delete` was that
same rename reaching for the other word, and the only `bulk-` command on
the surface. No `bulk-` CLI command has ever shipped, so this costs
nothing now and would be permanent later.
`files bulk-download` keeps its name: there is no `files download` to
collide with, and it is one archive rather than N operations. Its config
block now says why it exists at all, since the command is never built —
the builder skips non-JSON response modes, but the contract sweeps still
read the entry and require the folder-path field to be marked.
* fix(workflows): allow operations on blockless drafts
* feat(workflows): add manual and run-from-block execution
* fix(cli): update workflow run description test
* fix(tests): align fixtures with current contracts
* fix(api): derive chat activity from workflow deployment
---------
Co-authored-by: Theodore Li <theo@sim.ai>
|
||
|
Theodore Li
|
a99f61bee9 |
feat(api): add v2 resource management endpoints (#6900)
* feat(api): add v2 resource management endpoints * fix(cli): gate destructive v2 commands * fix(test): update v2 request-slice count * feat(cli): add shared workspace profiles |
||
|
Waleed
|
4f9d5f33b0 |
improvement(search): search every folder, and document real API error bodies (#6861)
* improvement(search): search every folder, and document real API error bodies Search on Files, Tables, and Knowledge was ANDed with the open folder, so a query only ever matched that folder's direct children — and the query was not cleared when you entered a folder, filtering the folder you just opened down to the same matches. A non-empty query now searches the whole workspace, a Location column names each result's folder, and opening a folder ends the search. Also gives GET /api/v2/files a `recursive` flag, and replaces the single shared OpenAPI error example — which showed `BAD_REQUEST` under every status tab — with one real body per status. * fix(search): discard the search term on clear instead of masking it `useSearchFilterValue` returned the debounced term whenever the input was non-empty, so clearing only hid the settled needle. The mask lifted on the next keystroke while the debounce still held the pre-clear term — opening a folder and typing within the window searched the whole workspace for the query the user had just abandoned. A clear now resets the settled term rather than hiding it, adjusted during render so the reset is visible to the render that follows the clear. The initial state is seeded from the first value so a deep-linked `?search=` still filters on the first render. |
||
|
Theodore Li
|
6006870f02 |
feat(credentials): add v2 credential lifecycle APIs (#6664)
* feat(credentials): add v2 OAuth connection APIs * fix(credentials): preserve active OAuth connection links * fix(credentials): bind OAuth links to connection intent * feat(credentials): complete v2 credential lifecycle * fix(credentials): make disconnect idempotent * fix(credentials): stabilize oauth draft retries * fix(credentials): bind oauth callbacks to drafts * fix(credentials): fail closed on oauth completion * fix(credentials): bind shopify completion to oauth state * fix(credentials): align custom oauth reconnects * fix(credentials): centralize application authorization * fix(credentials): keep OAuth draft intent immutable * fix(credentials): allow renamed reconnect targets * fix(credentials): close OAuth draft edge cases * fix(credentials): fail closed without breaking auth * fix(credentials): preserve migrated route behavior * feat(credentials): add provider search * fix(credentials): prevent stale secrets and drafts |
||
|
Waleed
|
6de8ba2504 |
fix(v2): close the correctness gaps an end-to-end audit found (#6655)
* fix(v2): stop a third-party tool description from 500ing MCP discovery
`v2McpToolInputSchema` declared `description: z.string().optional()` inside a
`.catchall(z.unknown())` object, and a declared key beats the catchall. The MCP
SDK's own `ToolSchema.inputSchema` does not declare `description` at all, so any
value — including the JSON `null` a Python server emits for an absent one —
passes its validation and reaches Sim unchecked. The builder's outbound `.parse()`
then threw, and the discovery error policy correctly declines to classify a
Sim-side schema defect, so the endpoint that completes MCP onboarding answered a
bare 500. The key is dropped and left to the catchall; `type`, `properties`, and
`required` stay pinned because the SDK enforces those at least as tightly.
Also in the v2 resources family:
- The single-resource query schemas for MCP servers, skills, custom tools, and
secrets are now `.strict()`, matching every list in the same family. A mistyped
flag was silently ignored behind a 200.
- `openapi/resources.ts` re-derived `RESOURCE_ERRORS` and
`RESOURCE_CONFLICT_ERRORS` inline in 21 of 22 operations. They now import the
shared constants; the generated spec is unchanged, which is the point.
- The internal MCP refresh route stamped `updatedAt` alongside `lastToolsRefresh`.
`updatedAt` means "configuration last changed" and is a public keyset sort, so
a refresh moved rows out from under an in-flight page. `updateServerStatus`
already held that invariant; the route now matches it.
- The discovery cooldown is a typed `McpServerCooldownError` rather than a
substring search for `cooldown`. `McpConnectionError` interpolates the server's
display name into its message, so a server named after the word was reported as
a transient cooldown when its connection had genuinely failed.
* fix(v2): close correctness gaps in the workflows deployment surface
Deploy and rollback bodies were plain objects, so a misspelled key was
stripped rather than rejected. On rollback that is silent misbehavior:
an omitted `version` legitimately means "reactivate the preceding
version", so `{"versoin": 5}` rolled back somewhere else and answered
200. Both v2 bodies, the run-read query, and the versions cursor are now
strict.
Deployment versions are an `integer` column, but the path param, the
versions cursor, and the v1 body each bounded it differently or not at
all — an out-of-range value overflowed the comparison into an
unclassifiable 500. One exported bound now covers all three.
Resume admission raised bare `Error`s for a stale contextId or an
already-resumed run, which the resume surfaces could not classify and
reported as 500. They now use the sibling `ResumeAdmissionError` already
in that file, carrying 404/409/400 and whether an automatic retry can
clear the refusal.
Docs corrections: rollback publishes the 409 its webhook-path conflict
already produces; deploy/undeploy/rollback reject a workspace key with
403, not the concealed 404 they documented; the workflows OpenAPI module
imports the shared error sets instead of re-deriving them; import and
the folder ops explain their folder-tree 413. The export route is marked
`headSafe: false` so a HEAD probe stops filing a WORKFLOW_EXPORTED audit
event for an export that never happened. `runId` is one bounded schema
across the run and log resources.
* fix(v2): conceal knowledge upload existence, tighten knowledge/files bounds
Security: the four knowledge document-upload routes rendered a bare upload
error policy with no resource concealment, while every sibling knowledge route
uses one. Because the use case resolves the knowledge-base context before
workspace authorization, the unconcealed 403 told any valid API-key holder that
a knowledge base exists in a workspace it cannot reach — the exact signal
GET /api/v2/knowledge/{id} withholds by answering 404 either way. All four now
use the composed concealing policy, which also renders the 415/402/413 the
route-local renderer already handled; that duplicate renderer is deleted.
Contracts:
- POST /knowledge/search is strict. It was the only non-strict v2 request body,
so a mis-cased rerankerEnabled or topK returned 200 with the key stripped,
changing what the caller was billed and silently disabling reranking.
- The document list takes limit, cursor, and search from the shared v2 schemas.
search was an unbounded, empty-accepting v1 string, so ?search= answered 200
with a full page here and 400 on GET /knowledge, and the term reached an
unindexed filename LIKE scan with no ceiling.
- The 16 non-strict single-field workspace query slices across both families are
strict, matching GET /knowledge/{id}/tags.
- GET /audit-logs takes workspaceIdSchema instead of a bare string (?workspaceId=
was forwarded as a filter and returned zero rows) and the shared run-window
bounds for startDate/endDate.
Documentation:
- listAuditLogs drops the 404 it has no code path to emit.
- upsertFileShare describes its workspace-key refusal as the 403 it renders;
the operation denies the key by principal kind, which the concealment policy
does not rewrite.
- The 12 body-reading knowledge and files operations publish the 413 their
pre-validation body read raises, and the file list publishes the folder-tree
413 its now-capped path index raises.
Correctness: queryWorkspaceFilePage loads its folder path index under
MAX_FOLDERS_PER_WORKSPACE like the workflow, table, and knowledge lists. An
uncapped index does not fail on truncation, so a real folder outside the read
rows resolved to undefined and answered "Folder not found".
* fix(v2): publish the reachable 413 on body-carrying resources ops
`parseRequest` buffers a JSON body through `parseJsonBody` under
`DEFAULT_MAX_JSON_BODY_BYTES` before any schema runs, and the v2 builders supply
`V2_PARSE_DEFAULTS.payloadTooLargeResponse`, so every operation whose contract
declares a body already answers 413 above the cap. The resources family
published it on none of them. A status a caller cannot see in the spec is a
status they will not handle.
Adds `RESOURCE_BODY_ERRORS` and `RESOURCE_CONFLICT_BODY_ERRORS` to the shared
sets and applies them to the seven affected operations: createMcpServer,
updateMcpServer, createSkill, updateSkill, createCustomTool, updateCustomTool,
and setSecret. All seven are `defineV2JsonRoute` handlers on non-GET methods
with no `parseOptions` override, so the 413 is genuinely reachable on each. The
new sets are opt-in rather than folded into the base sets precisely because
reachability is not automatic — an operation with no body, or one whose payload
reaches it through an uncapped path, would be publishing a response that can
never arrive.
A sweep test pins the invariant across the resources, billing, and logs
documents. It is one-directional by construction: several bodyless operations
publish 413 for their own folder-tree and render ceilings, so the converse would
flag correct documentation.
Also completes the shared-constant consolidation started in
|
||
|
Waleed
|
1fa40b8118 |
feat(v2): complete and align the v2 API surface (#6643)
* fix(v2): close four validation holes in the logs and billing surfaces
Each of these answered a caller-supplied value with a 500 or a silently
wrong result instead of a 400.
- `GET /api/v2/logs` accepted any string as `startDate`/`endDate`. The
route constructs a `Date` from it, so `?startDate=abc` reached the
driver's timestamp mapper as an `Invalid Date` and 500'd. Both bounds
now carry `.datetime()`, matching the sibling run list so one timestamp
works on both collections. This narrows the accepted set: a date without
a time and an offset-bearing timestamp are now rejected, and the field
descriptions say "UTC ISO 8601" rather than overpromising "ISO 8601".
- `v2BillingStatusQuerySchema` was the only non-strict query schema in its
family, so a mis-cased `workspaceID` was stripped and the caller got
account-scope billing in place of the workspace scope it asked for — a
wrong answer about money, served as a 200.
- An unresolvable `cursor` on `/api/v2/billing/logs` applied no cursor
condition and restarted the sequence at page 1 while still reporting
`hasMore`, so a pager holding a cursor across a deploy loops over the
first page and counts the same credits on every lap. It is now a 400.
The message does not reuse `INVALID_CURSOR_MESSAGE`, which names
`sortBy`/`sortOrder` params this collection does not accept.
- The logs `status` field disagrees with the run resources for the same
run: the run projection overlays `paused` from `paused_executions`,
so an ordinary human-in-the-loop pause reads `paused` there and
`pending` here. Reconciling would mean joining `paused_executions` in
this read and silently moving live runs between two buckets of a
shipped field, so the divergence is documented on the contract instead.
* feat(v2): expose the MCP tool plane and page the MCP server list
Registering an MCP server through v2 dead-ended: nothing on the public
surface ever ran tool discovery, so connectionStatus, toolCount, lastError,
and lastToolsRefresh stayed at their registration defaults and there was no
way to read a server's tools without opening the UI.
Adds GET /api/v2/mcp-servers/{id}/tools over a thin use case composed from
the existing mcp_servers.tools.discover operation, resolveServerContext, and
mcpService.discoverServerTools. It is personal-API-key-only — discovery
resolves the acting user's own OAuth credentials, which a workspace key
cannot supply — and the contract says so rather than letting callers meet an
unexplained 403. Discovery failures are classified instead of collapsing
into a 500: an unreachable or cooling-down server is a retryable 503, a
stale OAuth grant is a 401.
Also pages GET /api/v2/mcp-servers. It was the one unbounded list on the v2
surface, classified full-set on a bounded-by-construction rationale that
only holds for folder lists; nothing caps how many servers a workspace
registers.
* feat(v2/tables): strict row bodies, a filtered row count, and round-trippable required columns
Three tables gaps from the v2 capability evaluation.
Strictness. Every v2 tables request body is now `.strict()`. The row family
was the whole hole: `POST /query` sent v1's `filter` key answered 200 with a
fully unfiltered page, because Zod strips unknown keys unless told not to. The
same laxity covered the row create/update/delete/upsert/find bodies, the
run and cancel-runs bodies, the enrichment body, and — outside the row family
but the same class — the column delete, view create/update, and export bodies.
A contract sweep now walks every body-bearing tables contract and fails if one
of them stops rejecting an unrecognized key.
Filtered row count. `POST /api/v2/tables/{tableId}/query/count` answers the
question v1's `includeTotal`/`totalCount` answered and the `{data, nextCursor}`
envelope has nowhere to put: how many rows a predicate matches. It binds the
existing `queryTableRows` use case with `includeTotal: true, limit: 1` — no new
domain logic and the same `tables.rows.query` read policy. The use case types
`totalCount` as nullable because paged callers can decline it; this route always
asks for it, so a null is treated as a broken invariant rather than presented as
a fabricated zero.
Required columns. `required` is accepted on create-table, add-column, and
update-column, matching v1. v2 emitted the flag on every read while stripping it
from every write, so a column could not round-trip. Enforcement was already
complete: turning it on over rows with null, missing, or empty cells is rejected
by the domain.
* test(skills): pin the workspace-API-key split as structural, not accidental
A workspace API key can create a skill it can then never update or delete,
which no sibling resource does — so the asymmetry reads like an oversight
worth widening. It is not. Skill edits are authorized by the per-skill
editor row belonging to the acting user, which is why update/upsert/delete
declare a 'read' floor rather than 'write': workspace role is not the
authority. A workspace key carries no user subject, so allowing one replaces
a 403 with an unclassified PrincipalSubjectUserRequiredError that the v2
surface renders as a caller-reachable 500.
Records the reason on the registry and pins it, so the next reader finds the
argument instead of flipping the flag.
* feat(v2): read deployment state, and undo a file delete
Two v2 reads that existed only as a side effect of a mutation.
`GET /api/v2/workflows/{id}/deployment` publishes the state the deploy,
undeploy, and rollback responses carry, plus `needsRedeployment` — which
those responses structurally cannot carry, because they answer at the
moment the draft and the live version are equal. A caller that lost the
mutation response, or that polls from another process, had no way to ask.
Reuses `readWorkflowDeploymentStatus` behind `workflows.read`, the same
use case the internal status and deploy GETs already adapt.
`DELETE /api/v2/files/{fileId}` was a soft delete with no way to see what
it archived and no way to reverse it. `GET /api/v2/files?scope=archived`
pages the archived set and `deletedAt` on the file resource dates each
one; `POST /api/v2/files/{fileId}/restore` reverses the delete through
the existing `files.restore` operation. Restore is not a pure undo — it
returns the file to the root and renames it on a collision — so the use
case now reads the file back and both the response and the OpenAPI
description say what actually came back rather than what was deleted.
`scope=all` is rejected on the list for the reason the internal contract
already gives: it drops the `deleted_at` predicate and cannot use the
partial index. `scope=archived` combined with `folderPath` 404s when the
containing folder was archived too, which the contract documents.
* fix(v2): keep the unresolvable-cursor rejection a 400 on every surface
The cursor rejection lived in shared billing core but was an OrchestrationError
only, which the session-only GET /api/users/me/usage-logs cannot project: that
route is raw withRouteHandler and readTypedError matches instanceof HttpError,
so any signed-in caller typing ?cursor=x got a 500. UnknownUsageCursorError is
an HttpError carrying the OrchestrationError as its cause, so the v2 route still
renders BAD_REQUEST off the cause chain and the internal route answers 400.
Also closes the other half of the run-list parity: an inverted window on
GET /api/v2/logs is now a 400 instead of a silently empty page.
* fix(v2/tables): sweep union bodies per member and name the shapes on a rows 400
Review follow-ups on the strictness work.
The sweep was vacuous on the one union body it covers. Parsing
`{ notAContractField: true }` against `v2CreateTableRowsBodySchema` and looking
for `unrecognized_keys` anywhere in the issue tree is satisfied by either member
alone, so dropping `.strict()` from the single-row branch shipped green —
reproduced, 36/36 passing with the regression in place. The sweep now flattens a
union body into its members and asserts each one separately; removing `.strict()`
from either branch now fails a case that names it.
`POST /rows` answered an unknown key with `Invalid input`, the exact message the
v2 conventions name as failing the actionable-error rule, because a union
surfaces `invalid_union` first. The union now carries a message naming both
accepted shapes; the per-member failures still ride along in `details`.
Two TSDoc corrections. The `required` docstring claimed the domain rejects
turning the flag on over rows with empty cells — true of the update path, false
of add-column, which applies the flag as given (the same shape `unique` already
had here). And `.strict()` binds the top level only, so the view `config` object
and the shared sort-spec elements still strip unknown keys; both docstrings now
say so instead of implying full coverage.
* fix(v2): classify MCP discovery failures by type, not by substring
The tool-discovery error policy consumed categorizeError's status, whose
fallback is a substring match on the upstream message. Three consequences,
all caller-visible:
- A ZodError from the builder's own response `.parse` contains `invalid_type`,
so a Sim-side response-schema defect answered 400 "Invalid request
parameters" and suppressed the builder's 500 and its unhandled-error log.
- An upstream `Invalid params` or `not found` became the caller's 400/404 on a
request the contract had already validated.
- A stale OAuth grant to the third-party server answered 401, the status this
surface reserves for a missing or invalid Sim API key, so a client would
rotate a credential that was never the problem.
The policy now dispatches on the MCP error families and returns null for
anything else. Reauthorization is a 409 carrying
`details.code: MCP_SERVER_REAUTHORIZATION_REQUIRED`; an unreachable, slow, or
cooling-down server is a 503 with a constant message.
Also: widen the shared server path-param description now that it covers tool
listing, map the list query explicitly so no undeclared `cursor` reaches the
use-case input, and document the endpoint's write side effects.
* merge: bring in the MCP tool plane workstream
* feat(v2): make knowledge tags usable and let documents be updated
v2 accepted tag slots on upload and filtered search by tag display name,
but no response ever returned a tag value and nothing listed the
vocabulary, so a shipped feature dead-ended in the public API. A document
that failed processing could only be deleted and re-uploaded, and
retiring 500 documents cost 500 requests.
- GET /api/v2/knowledge/{id}/tags returns the vocabulary (display name,
slot, field type) as a full-set list.
- Document list and detail responses carry `tags`, keyed by display name
exactly as search keys its result metadata. Writes stay slot-keyed; the
tags endpoint is the mapping and the contract documents the split.
- PATCH /api/v2/knowledge/{id}/documents/{documentId} renames, enables,
disables, retags, or requeues processing. Derived indexing state is not
writable: asserting `processingStatus` on an unindexed document would
corrupt search. A retry may not ride along with field updates.
- PATCH /api/v2/knowledge/{id}/documents bulk-enables or bulk-disables.
Bulk delete is deliberately absent — that operation records no semantic
audit, and a public bulk delete would empty a knowledge base leaving no
DOCUMENT_DELETED entries.
- The document list accepts the same name-based `tagFilters` as search;
the name-to-slot resolver moves out of search into a shared helper, and
the filters are stamped into the offset cursor scope so a replayed
cursor cannot cross a filter change.
- Search accepts `rerankerEnabled`, `rerankerModel`, `rerankerInputCount`
and returns `rerankerScore`; `rerankerApiKey` and `skipUsageBilling`
stay unexposed. Every result now names its `knowledgeBaseId`.
knowledge.tags.list flips from workspaceApiKey 'deny' to 'allow' (and
gains the workspace_api_key principal kind) so it matches the sibling
reads knowledge.documents.list / read / search. The vocabulary is
required input for two operations a workspace key can already perform.
Every tag write stays human-delegated.
* fix(v2): name every 403 cause, unfork boolean params, close nested strictness holes
Four cross-cutting consistency gaps on the v2 public surface.
**403s now carry a machine-readable cause.** The conventions skill mandated
`error.details.code` on 403 and nothing emitted one, so a client had to
string-match prose to tell "raise this member's role" from "this workspace
refuses personal keys" from "buy an enterprise plan" — four different
remedies behind one status, and every message reword a silent break. The
vocabulary is a closed set, `FORBIDDEN_DETAIL_CODES`, with a `Record` of
descriptions beside it that the generated OpenAPI 403 description is built
from, so a code cannot reach the wire unpublished. Refusals throw
`ForbiddenOperationError` in the domain and `v2CaughtOrchestrationError` —
the function every v2 error policy falls through to — attaches the code, so a
route cannot forget it. The audit-log resolver distinguished four causes and
collapsed them into one; it now names each.
Cross-tenant refusals deliberately get no code: they are concealed as 404 and
naming their cause would hand back the existence signal the concealment
withholds.
**Two boolean query params rejoin the majority.** `?includeDeparted` and
`?includeOutput` were `'true'`/`'false'` string enums inherited from the
internal shapes they reused, while four sibling params were real booleans.
Both move to `booleanQueryFlagSchema`, which still coerces both strings — a
strict widening, so an existing caller is unaffected, and the spec stops
telling callers to send a string.
**Two nested strictness holes close.** `.strict()` binds the top level only,
so `sort: [{ field, direction, nulls: 'last' }]` was answered 200 with the
null-ordering request dropped, and an unknown key inside a saved view's
`config` was accepted and discarded — the headline `filter` bug one level
down. `sortSpecSchema`'s element and both view-config schemas are now strict.
Safe on the read side because `normalizeStoredViewConfig` projects the
schemaless stored blob onto the declared keys first, so a legacy row cannot
turn into a 500.
The two sort dialects stay as they are. `/logs` and `/workflows/{id}/runs`
have one sortable column, so there is no `sortBy` to pair with; renaming
`order` breaks every caller and an alias is a second spelling of one thing
with undefined precedence. Both contracts and the skill now state the rule.
* style: format the files the workspace-scoped lint gate does not reach
`turbo run lint:check` runs `biome check .` per workspace, so `scripts/` at the
repo root is outside the graph and four changed files were unformatted — one of
them a merge artifact from reconciling the route baseline across branches.
* fix(v2): collapse the four knowledge document projections onto one null-tolerant summary
Extracts toV2DocumentSummary in app/api/v2/knowledge/utils.ts and composes the
list, upload-acknowledgement and detail presenters from it. toV2TaggedDocument
serialized uploadedAt with a bare .toISOString(), so a document with no upload
timestamp threw where every sibling returned null and the contract declares the
field nullable.
Also consolidates the two Zod strictness walkers onto one shared introspection
helper that unwraps wrappers and expands unions, closing the hole where a
union-shaped schema answered null and was skipped by the pagination sweep.
* fix(v2): stop HEAD driving MCP discovery, and unbreak the updatedAt keyset page
B1: Next aliases HEAD onto GET, which RFC 9110 permits only because GET is safe.
The MCP tool-discovery GET is not: it opens a live connection to the registered
endpoint and writes the outcome onto the server row. The v2 JSON builder gains a
headSafe option, default true, and the discovery route declares itself unsafe —
a HEAD is authenticated and rate-limited, then answered bodiless.
B2: a discovery status write stamped updatedAt, which this branch added as a
keyset sort, so any concurrent discovery duplicated and skipped servers across a
caller's pages. Discovery liveness already has lastConnected, lastToolsRefresh,
lastError and statusConfig.
B4: a public refresh now skips the positive cache but keeps the failure cooldown,
so it cannot be used to drive a connection attempt per request at a failing
endpoint. An explicit user action on their own server keeps the full bypass.
B6: the consecutive-failure counter is incremented SQL-side rather than read,
incremented and written back, and the success branch carries the same workspace,
liveness and staleness guard the failure branch already had.
* fix(v2): bound the bulk update echo, close the search leak, and make the docs true
B3: a selectAll bulk document update echoed every changed identifier, which the
request does not bound — a 100k-document knowledge base produced a multi-megabyte
array, materialized and then element-wise validated. The use case now reports
whether the selection was unbounded and the presenter omits the echo.
A1: the knowledge search presenter spread the whole use-case result, which also
carries userId, workspaceId, a cost breakdown and a live secret-trace registry.
Only Zod's default key-stripping kept them off the wire. Projected explicitly.
P1-a: GET /knowledge/{id}/tags advertised all 17 slots while the document PATCH
accepted only the seven text ones. The writer already coerces every slot type,
so the PATCH now takes all 17 in their declared types, with a 400 where a
malformed value used to silently clear the tag.
P1-b: both new PATCHes deny workspace API keys and now say so.
P1-c: the two table query reads declare maxBodyBytes and now document the 413.
P1-d: getWorkflowDeploymentV2 loses its legacy suffix.
C3: deletes two orchestration error mappers with no callers that mapped
'forbidden' with no details.
D2: a stored null in table_views.config survived the pick and failed the
response schema.
Also folds the six 'bounded set' paraphrases onto one FULL_SET_LIST constant,
shares the run-window date bound between the logs and runs lists so their
documented parity is enforced rather than asserted, adds the missing barrel
export for FORBIDDEN_DETAIL_CODE_DESCRIPTIONS, and strictens two response
schemas whose peers were already strict.
Migrates 40 v2 route tests onto the shared @sim/testing harness: 26 asserted a
rateLimitSubjectIds shape v2 auth never returns, 26 asserted the wrong
refillRate, 33 could not exercise their 401 path at all, and 6 hard-wired the
rollout gate to null.
* fix(mcp): bound the connect handshake, and stop the 403 description over-claiming
B5: the connect clamp was getMaxExecutionTimeout(), the workflow ceiling of
seven days, so the real bound became the server row's own timeout — which the
registration contract permits up to 300s — times the connect retries. A slow
server could hold a Node request for roughly twenty minutes. Connecting is not a
workflow run, so the handshake now shares the one-minute ceiling tools/list
already applies to itself.
C2: the generated 403 description asserted that error.details.code names the
cause on every 403. Nine domain refusals still throw a bare forbidden
OrchestrationError and reach the wire codeless, so the wording now says 'where
the cause is one a caller can act on'. Reparenting those throws is left as a
deliberate change: one of them is a cross-tenant refusal that belongs in the
codeless class and would change its status.
* chore: reconcile the route ratchet with staging
* style: sort imports and format the three files biome flagged
* fix(openapi): import the forbidden-code constants from their module, not the application barrel
The barrel also re-exports the authorized use-case layer, which loads
@sim/db at import time. That pulled a database connection into the
OpenAPI spec check, so check:audits failed wherever DATABASE_URL is
absent, including CI.
|
||
|
Waleed
|
47f143016e |
fix(docs): restore api-reference URL continuity and fix translated SDK bodies (#6617)
Two docs-only defects introduced by #5273 (`263e3ca67e`), which re-founded the public API reference on the v2 surface. 1. Ten translated SDK snippets produce a deterministic 400. The streaming example in the five translated `api-reference/typescript.mdx` and `python.mdx` pages was repointed from `/api/workflows/{id}/execute` to `/api/v2/workflows/{id}/execute` and nothing else was changed — fr/ja/zh typescript.mdx are literally one-line diffs. `message` stayed at the body root. That was correct against v1, whose route treats the whole non-control body as workflow input, but `v2ExecuteWorkflowBodySchema` ends in `.strict()` and the route parses before executing, so every copied snippet returns `400 Unrecognized key: "message"`. The same commit fixed the English bodies to `input: { … }`, so this is an oversight, not a decision. The ten fences now match `en/api-reference/typescript.mdx:959` and `python.mdx:681`. Not relaxing `.strict()`: it is deliberate house style across the v2 contract and is what makes a typo'd option fail loudly instead of silently. 2. Thirty-two published operation pages 404 with no redirect. Replacing the single v1 `openapi.json` with seven v2-only specs changes page identity, because fumadocs derives every generated page as `slugify(tag)/operationId` from the specs at build time. Re-deriving both sets gives 52 old slugs and 128 new ones: 32 disappear and 20 keep their URL while silently retargeting v1 -> v2 (`knowledge-bases/updateKnowledgeBase` also flips PUT -> PATCH). All 52 are in the live sitemap — parsing `<loc>` from docs.sim.ai/sitemap.xml gives 458 URLs of which 56 are `/api-reference/`: the four static pages plus all 52 generated ones by name, including every one of the 32 that die. They are 200 today under an allow-all robots.txt. The spec swap itself is deliberate and CI-enforced (`check-openapi-specs.ts` requires every published operation under `/api/v2/`), so restoring the v1 operations is not an option. The missing piece is the redirect map, in a file that already carried 56 such rules from earlier doc moves. `permanent: true` (308) is used only for a true 1:1 successor — same operation, renamed. A 308 is cached indefinitely and effectively unrecallable, so anything that collapses two pages onto one, changes the identifier model, or lands on a merely adjacent operation is `permanent: false` (307). That splits 21/11. Four destinations differ from the mapping proposed in review, each on evidence from the specs rather than from the operation names: - `workflows/getJobStatus` is not destination-less. The v2 queued-execution receipt (`QueuedWorkflowRun`) returns `statusUrl` `/api/v2/workflows/{id}/runs/{runId}`, so `workflow-runs/getWorkflowRunV2` is the successor poll target — far better than a generic landing page. - The three HITL read operations go to `getWorkflowRunV2`, not to the resume page: `WorkflowRunStatus` carries a `paused` object with `contextId`, `pausedAt`, and `pauseKind`. Pointing a GET doc at a POST doc would be wrong. - `human-in-the-loop/listPausedExecutions` goes to `listWorkflowRunsV2`, whose `status` filter includes `paused`. - `tables/batchUpdateRows` is 307, not 308. v2 `updateTableRows` is "Update Rows by Filter" — the successor of v1 `updateRows` (PUT, predicate-based), which keeps its 308. v2 has no by-id batch update at all, so batchUpdateRows lands on a genuinely different operation. 3. A guard, so the map cannot rot silently. `scripts/openapi/docs-redirects.test.ts` recomputes the generated slug set the way fumadocs does and asserts no `/api-reference/` source shadows a live page and every destination resolves. Nothing else in the repo reads docs URLs, so a future spec regeneration would otherwise break the map with no signal. It needs no wiring: `check-openapi.ts` already runs this vitest config. The redirect array moves to `apps/docs/lib/redirects.ts` because the guard cannot import `next.config.ts` — `createMDX()` runs the fumadocs-mdx generator at import time, which made vitest emit an unhandled build error and warn about false positives. The 56 pre-existing rules are byte-identical to before, verified programmatically; `next.config.ts` keeps the same public shape and Next's own `checkCustomRoutes` accepts all 88 rules. Open question for the owner, larger than the redirects: all 82 `/api/v1` route files survive on staging, so a live public API now ships with zero reference docs, while the documented `/api/v2` surface returns 404 for any caller outside the off-by-default `v2-api` flag cohort. Is that the intended end state or transitional? |
||
|
Waleed
|
9b9f4ee596 |
fix(v2-api): close three secret disclosures, make the surface consistent, and align docs with signatures (#6560)
* fix(v2-api): close two secret disclosures and align docs with signatures
Two P0 disclosures, five correctness bugs, and the standardization and
guard work that came out of auditing them.
**Secret disclosure — workflow version state.** `GET /api/v2/workflows/{id}/
versions/{version}` served the deployed graph unsanitized, so a read-role
workspace API key received plaintext block-password values and OAuth
credential ids. The sibling export route has always sanitized. Every other v2
response is protected structurally because the builder re-parses it, but this
field is `z.custom<WorkflowState>()` — a predicate that validates nothing —
which is why it survived earlier audits. Sanitization now lives in the use
case, secure by default, with a named `includeCredentialValues` opt-in that
only the session-authed deploy-preview route sets.
**Secret disclosure — MCP headers.** The internal list and update routes
returned custom `Authorization` headers verbatim to any read-role member;
headers are stored unencrypted. Values are now gated on write permission and
projected through one shared helper. The settings UI genuinely prefills from
them, so blanking outright would wipe headers on unrelated edits — write-only
headers plus encryption at rest are the follow-up.
Correctness:
- v2 execute ignored `X-Sim-Via`, resetting the call chain on every hop and
defeating the recursion guard. Wired on both the keyed and anonymous paths.
- v2 knowledge search accepted `searchMode` and dropped it, silently serving
vector-only results for a hybrid request, and allowed 50MB bodies where
internal caps at 2MiB.
- v2 run cancel never released the plan concurrency slot and half-cancelled
group runs; a group conflict now returns 409 instead of reporting success.
- v2 table row writes stamped no secret provenance, so the next internal read
reported the whole page incomplete. `secretProvenance` is now required on
the primitives, making the next omission a compile error.
- Folder conflicts and malformed paths returned 500; they are 409/404/400 now.
`FolderPathError` splits from `FolderHierarchyError` so a corrupt stored
tree stays a 500 and stays in 5xx alerting.
Standardization and documentation:
- `PUT /files/{id}/share` -> PATCH. The resource is not round-trippable
(`hasPassword`, never the password), so merge-on-omission is the only
implementable semantics.
- ~40 spec truthfulness fixes: a 410 the API cannot emit, eight 423s with no
lock guard, ~30 reachable-but-undocumented 404/400/413s, and six inverted
field claims. Eleven operations that always reject a workspace key now say
so — four of them answer 404, so a workspace key was told the resource did
not exist.
- `NAME_PATTERN` lost its `/i` through `z.toJSONSchema`, publishing 15
patterns that reject names the runtime accepts. Every generated client
rejected any capitalized table or column name, and two of the spec's own
examples failed the spec's own schema.
Guards, so these classes cannot recur:
- `check:route-verbs` (new) cross-checks all 212 builder routes' exported verb
and path against their contract. The builders only compare at runtime, so a
half-done rename previously passed CI and 500'd in production.
- Example validation now runs against the published JSON Schema with formats
on, covering 225 nodes instead of 100 — this is what caught the regex bug.
- The list-pagination sweep is union-aware and fails loudly on a schema it
cannot introspect, rather than counting it compliant.
* refactor(v2-api)!: flatten the single-resource response envelope
BREAKING: 31 endpoints that returned `{ data: { <resource>: T } }` now return
`{ data: T }`.
This corrects drift, not a design decision. PR #5273 added skills, custom
tools, MCP servers, secrets, and knowledge nested while adding workflows,
files, and logs flat — and in the same commit wrote the `v2/shared.ts`
docblock declaring `single resource: { data: T }` is the standard. The nested
half appears to have been modelled on the v2 tables surface (#6067), which
landed twelve days earlier. Lists were already `{ data: T[], nextCursor }`, so
flat single-resource is what actually matches them; nesting made every client
destructure a layer that carries nothing.
Doing it now because the cost only grows: `v2-api` is still dark-launched, so
today this breaks no one. After GA it needs a deprecation window.
Payloads that carry real information were deliberately left alone — this was a
classification exercise, not a mechanical sweep. Unchanged: delete
acknowledgements (`{ id, deleted }`, `{ path, deleted, deletedItems }`), the
knowledge search envelope (which echoes query, knowledgeBaseIds, topK and
totalResults alongside hits), upload payloads carrying signed tokens and
transfer instructions, bulk-operation counts, `{ row, operation }` upserts,
named acknowledgement scalars (`{ dispatchId }`, `{ cancelled }`), and
`{ columns: [...] }` — a collection, where a bare `{ data: T[] }` would be
indistinguishable from the list envelope but without `nextCursor`.
Also flattened the two file-share responses, which were not in the original
survey: leaving them would have put one resource in two shapes on one path.
`GET /files/{id}/share` now returns `{ "data": null }` when a file has never
been shared.
No consumer is affected. Both SDKs touch exactly two v2 endpoints — execute
and run status — and both were already flat. No docs MDX, client hook, or
internal caller reads a changed response; Copilot table tools call the
application use cases directly rather than the HTTP surface.
The shared `v2FolderSchema` is untouched: every folder flatten was achievable
at the response site, which is itself evidence flat was the intended shape.
* fix(v2-api): close a third secret disclosure and make concealment coherent
**Secret disclosure — run snapshot.** `GET /api/v2/logs/{runId}` returned
`workflowState` straight from `workflowExecutionSnapshots.stateData`, which is
the workflow graph: `blocks[].subBlocks[].value` holds `password: true` field
values and `oauth-input` credential ids. Nothing on that path sanitized it, and
the field was typed `z.unknown()`, so the builder's response parse stripped
nothing. A read-role workspace API key could read plaintext credentials.
This is the third instance of one pattern, and the pattern is the finding: the
builder protects every response by re-parsing it, so the only fields that can
leak are the ones typed `z.unknown()` or `z.custom()`. Both prior disclosures
sat behind exactly such a field. The snapshot is now sanitized in the use case
and the field is typed object-or-null. An inventory of every remaining
`z.unknown()` in the v2 contracts is in the PR description; two carry data with
no projection behind them and are named there as follow-ups.
**Concealment was bypassable.** `createV2ResourceConcealmentPolicy` rewrites
resource-authorization failures to 404 so a caller cannot probe for existence.
Workflows and files applied it on every verb; tables and knowledge applied it
only on reads. A caller could therefore probe with PATCH, read the 403, and
learn the resource exists — the read-side concealment bought nothing. Nine
mutation sites now conceal, plus the three table-column verbs, which were
inconsistent with their own sibling sub-resources.
`lib/logs/api/route-policies.ts` was a second, divergent implementation that
sniffed `response.status === 403` and so also swallowed workspace-policy
denials the canonical helper deliberately preserves. It now uses the helper. A
third such sniff survives in the upload-control helper and is noted as a
follow-up.
Also:
- `DELETE /tables/{tableId}/rows/{rowId}` returned the bulk `{deletedCount,
deletedRowIds}` shape while nine sibling single-resource deletes return
`{id, deleted}`. It now matches them.
- Nine operations can 404 on an unknown folder path and did not document it;
`createWorkflow` could 413 on an oversized folder tree and did not; getting a
run can 409 when trace data was truncated and did not.
- `queryTableRows` documented a 413 it cannot emit and `resumeWorkflowRun` a
423 with no lock guard anywhere in its path — the same un-producible-status
class already cleared for 410 elsewhere.
- Execute's 409 description covered only the run-id case after the
recursion-guard fix added a second cause, and named a code the route does not
emit: the wire carries `error.code: CONFLICT` with the specific cause in
`error.details.code`. `x-sim-via` is now a declared request header.
- Deploy and rollback published examples that were impossible: `isDeployed:
true` beside `activeDeployment: null`, where the route computes the former
from the latter.
- `afterRowId`/`beforeRowId` were published on row insert and silently dropped
by the route, so a positional insert became a tail append.
- A generated document whose script fails permanently answered "still being
generated, try again" forever; the underlying cause is now preserved.
* docs(v2-api): correct eleven false or misleading spec claims
Structural parity between contracts and specs is CI-enforced; semantic truth is
not. These are claims the spec made that the code does not honour.
Outright false:
- `DELETE /files/{fileId}` said it deletes "the stored bytes". It archives:
the row is retained with a deletion timestamp and the bytes are never
removed. Restore exists, but only on the internal API, so the description now
says so rather than implying v2 offers it.
- Execute documented `409 EXECUTION_ID_CONFLICT` in three places. The wire
carries `error.code: CONFLICT` with `error.details.code: RUN_ID_CONFLICT`;
only v1 ever emitted the documented string.
- The files spec claimed every endpoint uses the canonical envelopes while
`GET /files/{fileId}` returns octet-stream.
- The shared timestamp rule justified itself with a rendering claim that is
false — 29 bare-form sites publish `format: date-time` identically. The real
difference is runtime validation, so the rule now says that. It was softened
rather than enforced: responses are re-parsed, so adding `.datetime()` to a
field whose producer can emit a non-ISO string turns a working read into a
500, and that could not be proven for all 29 without a much larger audit.
Misleading:
- The billing ledger silently defaults to a 30-day window, so a client
paginating to `nextCursor: null` believes it has the whole ledger.
- Deleting a connector-backed knowledge document does not delete its chunks —
the row survives as excluded and the embeddings remain.
- `listTables` said "all tables"; it is keyset-paged with a default limit.
- `GET /files/{id}/share` omitted the `data: null` never-shared case its own
schema and example already declare.
- The share PATCH matrix omitted two hard 400s, so following it literally
against a never-shared file fails.
- Five knowledge operations render a canonical folder path back and can 413 on
an oversized tree without carrying the sentence that says so.
Also: the upload-control helper was a third implementation of concealment by
sniffing `response.status === 403`, which masks workspace-policy denials the
canonical helper deliberately preserves. It now uses the shared policy, so
those denials keep their 403. And the shared docblock's search-field
enumeration was presented as exhaustive while omitting two lists, and its
error-envelope claim omitted the two upload data-plane routes that emit a bare
`{error: string}` — both now carry the carve-out the CI allowlist already had.
* test(v2-api): align upload concealment test with cross-tenant-only semantics
#6557 narrowed `createV2ResourceConcealmentPolicy` to conceal only the three
cross-tenant authorization classes, deliberately letting a same-workspace
policy denial keep its 403 so the caller learns why. My test predated that and
asserted a workspace-key denial was concealed as 404.
Split into two cases that pin the distinction rather than paper over it: a
cross-tenant reach conceals, a workspace-key policy denial does not.
* fix(v2-api): accept the redacting log status and envelope the knowledge-search 413
The v2 log presenters parsed status against a five-value enum, but the
execution logger persists a sixth, redacting, while a finished run's output
is scrubbed. Any such row failed the response parse; on the list route one
row 500'd the whole page. The enum is now derived from
PersistedWorkflowExecutionStatus with a compile-time exhaustiveness
assertion, so a future status is a type error rather than a production 500.
POST /api/v2/knowledge/search declared maxBodyBytes without
payloadTooLargeResponse, so its 413 returned a bare string instead of the v2
error envelope. It now matches the sibling deploy/rollback routes.
* fix(uploads): restore archive extraction folder parity
Archive extraction into workspace files/ was rewritten onto the authorized
application-operation boundary, and three behavioral regressions came with
that move. Together they broke every archive containing a subdirectory, and
100% of copilot extract() calls (materialize-file always passes
rootFolderSegments: [baseName], and its catch only handles ArchiveError).
1. Non-canonical folder path. The extractor joined the folder segments with
"/" and passed the result as `path` to createWorkspaceFileFolderOperation.
That path reaches requireNonRootFolderPath -> parseFolderPath, which
requires a leading "/" and byte-for-byte canonical per-segment encoding,
so "bundle/data" threw FolderPathError before anything was written — and
a folder name containing a space or a reserved character would still have
thrown after merely prefixing a slash.
2. exactName: true. createWorkspaceFileFromBuffer was told to demand the
exact leaf name, which sets maxAttempts = 1 and raises FileConflictError
when the name already exists. The extractor's rollback then deleted every
file written so far, so one colliding name destroyed the whole
extraction. Reachable today for flat archives through the unzip action of
POST /api/tools/file/manage. Restored to auto-suffixing via
allocateUniqueWorkspaceFileName.
3. Wrong folder primitive. createWorkspaceFileFolderAtPath creates exactly
one leaf, conflicts on an existing path, and requires the parent to exist
already. The extractor never creates intermediates and caches by full
path, so the first nested entry asked for a folder whose parent was never
created. The correct semantics are ensureWorkspaceFileFolderPath: walk
every segment, reuse what exists, create only what is missing.
Rather than bypass the operation boundary by calling the manager primitive
directly, this adds ensureWorkspaceFileFolderPathOperation — an authorized
application use case under files.folders.create that expresses "ensure this
whole chain exists" — and routes the extractor through it with raw decoded
segments, so no path string is built and no encoding can be malformed.
archive.test.ts previously mocked the folder operation and asserted the
broken shape (path: 'bundle'), which is why this shipped. The suite now
fakes the workspace-file store in memory while enforcing the real rules:
folder paths run through the production parseFolderPath family, the
create-one-leaf operation conflicts and requires a parent, and exactName
governs conflict vs auto-suffix. Nested, reuse, encoded-name, and collision
cases are covered and each fails against the pre-fix code.
* chore(files): tidy archive extraction cleanup
* fix(uploads): roll back folders archive extraction created
Extraction now materializes folders before uploading files, but the failure
path only deleted the extracted files — every folder the call created was left
behind. That is not cosmetic: `materialize_file` guards re-extraction by looking
up the root folder path and refusing when it has any child, so a half-extracted
nested archive turned every retry into "already extracted — delete that folder
first" until a human cleaned up the tree by hand.
The rollback must delete only folders this call actually inserted, never one it
reused: extracting into an existing path is normal (a sibling entry, an earlier
successful extraction), and deleting a pre-existing folder would destroy
unrelated user data. `ensureWorkspaceFileFolderPath` already distinguishes the
two while walking the segment chain, so it (and its application operation) now
reports `createdFolderIds` alongside the leaf id. The extractor accumulates
those ids in creation order and, on failure, deletes them in reverse — parents
are recorded before their children, so reverse order is deepest-first and a
parent is never removed out from under a child. Folder cleanup is best-effort
like the existing file cleanup, so a cleanup failure never masks the original
error.
* fix(billing): withhold the payer credit pool from v2 status readers
`GET /api/v2/billing/status` resolved the workspace's payer and projected
that payer's pooled allowances — credits used, credit limit, credits
remaining, and the payer entity's storage usage and quota — to any caller
holding only `read` on the workspace, including a personal API key. The
payer pool is shared across every workspace that payer funds, and the
platform already treats it as privileged: the workspace credit-availability
surface computes `canViewPayerPool` from `canManageWorkspaceBilling` and
substitutes member-scoped or null figures for everyone else. The new
versioned endpoint had no equivalent gate.
`credits` and `storage` are now projected only to a caller who may manage
the resolved payer's billing: the billed account holder of a personally
hosted workspace, an admin of the hosting organization, or a workspace API
key, which only a workspace admin can provision. The endpoint stays at
`read` so a plain member keeps the plan, period, and standing the workspace
UI already shows them, and an exceeded pooled limit still reports as
`limit_exceeded` without disclosing the numbers behind it. Both fields are
nullable on the wire and in the regenerated OpenAPI spec.
The decision lives in the application use case, resolved from canonical
workspace state, not in the route: billing authority is payer identity and
organization role, which the workspace permission ladder cannot express —
a plain workspace `admin` is deliberately not enough.
* chore(api): remove the unused public API route builder and dead endpoint labels
`withPublicApiRouteHandler` and 27 `ApiEndpoint` union members landed together
in #5273, but the v2 surface shipped on `defineV2JsonRoute` + `v2RateLimits`
instead. The builder had no production caller — only its own test — and the v2
rate limiter never reads an `ApiEndpoint` label, so those members were never
emitted to telemetry by symbol or by string literal.
Remaining members are exactly the labels a v1 route passes to `checkRateLimit`
or `authenticateRequest`. Drops the now-unreachable `hasZodUsage` branch from
the API validation audit; no ratchet metric moves (route total stays 1093).
* fix(billing): deny the payer pool to actor-less workspace API keys
The first pass gated `credits` and `storage` on billing authority for
personal API keys but let a `workspace_api_key` principal through
unconditionally, which left the excluded role a way back in. Any workspace
`admin` may mint a workspace API key, and a workspace `admin` is
deliberately not a billing manager, so an admin who reads `null` as
themselves could mint a key and read the full pool with it. On an
organization-hosted workspace that pool is the organization's, spanning
workspaces the admin has no standing in.
Billing authority is payer identity or an organization admin role — a
property of a person. A workspace API key is deliberately actor-less, so it
can never satisfy it and now reads both fields as `null`. Attributing the
key to its creator was rejected: it would launder the same workspace-admin
role, it breaks when the creator's authority is revoked while the key lives
on, and substituting a key's owner for the acting principal is what the
application operation boundary forbids. The reasoning sits in TSDoc at the
decision point.
The key keeps the plan, period, and standing it needs to monitor a
workspace, including `limit_exceeded` and `billing_blocked`. No in-repo
caller reads `credits` or `storage` from this endpoint. The payer storage
pool is now read only once disclosure is authorized, so a caller who may
not see it no longer triggers the query at all.
* fix(folders): bound the workflow folderId-branch path index reads
`createWorkflow` and `updateWorkflow` each resolve a folder two ways inside one
function. The folderPath branch goes through `resolveWorkflowFolderPath`, which
loads the path index with `maxRows: MAX_FOLDERS_PER_WORKSPACE`; the folderId
branch loaded it with no bound at all, issuing a `SELECT` over every active
folder row in the workspace. In `updateWorkflow` the unbounded read and the
bounded fallback sit thirty lines apart in the same function.
Passes the cap at both sites, matching the read sites that already opt in.
Exceeding it throws `FolderCollectionLimitExceededError` rather than truncating,
because a partial path index resolves real folder paths to `undefined` and
re-roots resources at the workspace root.
`maxRows` deliberately stays opt-in rather than becoming the default. Folder
creation does not refuse at the same ceiling on every path — `POST /api/folders`
goes through the `createFolder` name/parentId variant, which passes no
`maxFolderRows`, so the count guard in `executeCreateFolderAtPath` never runs
and a workspace can already hold more than `MAX_FOLDERS_PER_WORKSPACE` folders.
Defaulting the bound would make every path-index consumer throw for a state the
product allows to exist. Reconciling reader and writer is a separate change with
a user-facing limit, not a chore.
* chore(billing): tidy payer-pool concealment cleanup
* fix(api): reject an undecodable offset cursor on v2 table rows
GET /api/v2/tables/{tableId}/rows coerced an undecodable pagination cursor to
offset 0 and re-served page one. A client paging forward reads that as a fresh
first page and can loop over it forever. Every sibling v2 cursor list — logs,
files, workflows, workflow runs, workflow versions, workspace members, tables,
knowledge documents — already rejects with a validation error instead.
Extracts the offset-cursor decode both offset-paginated v2 routes had inlined
into `decodeOffsetCursor`, next to the existing `decodeSortedCursor`, so the
reject-don't-restart rule has one home.
* fix(api): restore v1 table error-response parity and stop internal message leak
The v1 table routes were rewritten to consume `lib/table/orchestration`
results, and two response behaviors drifted from what the live API returned.
Information disclosure: an unclassified failure's `outcome.error` carries
whatever text the fault happened to have. Drizzle wraps a throw raised inside
a transaction in an error whose own message is the failed statement and its
bound parameters, so `DELETE /api/v1/tables/{tableId}` and
`DELETE /api/v1/tables/{tableId}/rows/{rowId}` returned that verbatim in the
500 body to any API-key holder. Previously these returned a fixed generic
string.
Lost `lock` field: the 423 body used to be `{ error, lock }`. The delete,
row-delete, and column-update routes (v1 and internal) dropped the lock kind
the orchestration result already computes, leaving clients unable to tell
which lock to clear.
Both are fixed at one altitude: `orchestrationOutcomeErrorResponse` in
`app/api/table/utils.ts` is now the only way a table route projects an
orchestration failure onto the wire. It renders the route's fallback for an
unclassified failure and the real message for a classified one (validation,
not-found, conflict, locked keep their specific text), and carries `lock` on a
423. A future route cannot reintroduce either bug by hand-spelling the body.
Duplicate table names on `POST /api/v1/tables` keep answering 409 rather than
reverting to the previous 400. 409 is the correct semantic, and every other v1
duplicate-name surface (knowledge, files, workflow import) already answers 409;
the tables 400 was the outlier. v1 tables appears in no published OpenAPI
document and no in-repo client branches on the status, so the compatibility
cost is limited to a caller matching 400 specifically for a name collision.
* fix(skills): only reject a built-in name collision on an actual rename
The built-in-name guard ran on every update that carried a `name`, without
comparing it to the skill's current persisted name. Skills created before the
guard existed can legitimately carry a built-in's name (they simply shadowed
the built-in at read time), and the skill modal always submits the full object
including the unchanged name — so every save of such a skill returned 400 with
"The skill name ... is reserved by a built-in skill", with no way to fix it
short of renaming.
Move the guard in `updateSkill` to after the canonical row is loaded and run it
only when the submitted name differs from the current one. Creating a skill
with a built-in name, and renaming an existing skill into one, are still
rejected. The check stays in the shared orchestration primitive because that is
the only layer both the internal `/api/skills` adapter (via `performUpdateSkill`)
and `updateSkillUseCase` (v2 + Copilot) pass through, and it is where the
current name is in hand.
* chore(tables): tidy v1 error projection cleanup
* chore(skills): tidy collision guard cleanup
|
||
|
|
263e3ca67e |
improvement(external-endpoints): v2 versions with clean signatures + updated docs based on openapi spec (#5273)
* v0.6.29: login improvements, posthog telemetry (#4026) * feat(posthog): Add tracking on mothership abort (#4023) Co-authored-by: Theodore Li <theo@sim.ai> * fix(login): fix captcha headers for manual login (#4025) * fix(signup): fix turnstile key loading * fix(login): fix captcha header passing * Catch user already exists, remove login form captcha * improvement(external-endpoints): v2 versions with clean signatures + updated docs * feat(usage): accept X-API-Key on usage-logs list + export /api/users/me/usage-logs and /export now use checkHybridAuth — the same auth /api/users/me/usage-limits already accepts — so external monitors can read summary.bySourceCredits (the source breakdown of usage-limits' aggregate currentPeriodCost) instead of estimating Copilot spend by subtraction. Workspace-scoped keys are pinned to their own workspace's slice of the ledger: the filter defaults to the key's workspace and an explicit mismatch 403s. Both endpoints documented in openapi-core.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(billing): dedicated v2 usage endpoints; keep internal usage routes session-only Replaces the earlier X-API-Key enablement on /api/users/me/usage-logs with a dedicated public surface, so the internal Billing-settings endpoints can evolve with the UI while external monitors get a stable versioned contract: - GET /api/v2/billing/usage — current-billing-period summary with bySourceCredits (the source breakdown external monitors need to watch e.g. Copilot consumption without estimating by subtraction), plus limitCredits and plan - GET /api/v2/billing/usage/logs — cursor-paged credit ledger in the v2 envelope - workspace-scoped keys are pinned to their own workspace's slice; personal keys read the account ledger The public wire is credits-only: usage-logs rows now carry a hasCost boolean instead of dollarCost (the Billing UI only needed the >0 signal), and the rateLimit block is removed from the usage-limits response and docs (deploy-modal tab relabeled accordingly). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(docs): validate OpenAPI specs against the Zod contracts in CI The specs in apps/docs are hand-authored because they carry what Zod never defines — error envelopes, status codes, prose, examples — so they can't be generated; check:openapi validates them instead: - spec integrity: $refs resolve, operationIds unique, 2xx documented, no orphaned component schemas - v2 conventions: every /api/v2 operation documents 401 + 429 and every 4xx/5xx resolves to the canonical { error: { code, message } } envelope - contract cross-check: contracts are auto-discovered from lib/api/contracts/v2 (each carries its method + path); doc<->contract coverage both ways, query/body/response field diffs via z.toJSONSchema - examples: documented request/response examples must parse with the matching contract's actual Zod schemas First run caught real drift, fixed here: 16 stale orphaned schemas in the core spec, the v2 billing ops referencing v1-shaped error components, deploy/rollback examples missing the required nullable lifecycle keys, CreateTableBody missing folderId, a legacy-grammar delete-rows example, and four knowledge document ops missing their required workspaceId query param. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(docs): recursive field diff in check:openapi + the deep drift it found A mutation test showed the doc<->contract field diff only compared top-level properties, so a typo inside the { data } envelope passed. The diff now descends through matching object properties and array items (both sides must expose a property set — passthrough contracts and prose-only docs end the descent instead of false-positive), with the Zod JSON-schema root doubling as the $defs context. Deep drift it immediately caught, fixed here: select-column config (options/multiple) missing from every tables column schema, AddColumnBody hand-rolling a third column shape (now composed from ColumnInput, with position/workflowGroupId as the per-op extensions the contracts actually admit), chunking strategyOptions undocumented, and the deployment lifecycle fields (activeDeployment/latestDeploymentAttempt) missing from DeploymentState. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(security): close the triggerType rate-limit bypass on workflow execute Caller-supplied triggerType flowed unchecked into preprocessExecution, whose checkRateLimit default turns OFF for 'manual'/'chat' — so any API-key caller, and any anonymous public-API caller billed to the workspace owner, could execute unthrottled by sending {"triggerType":"manual"} (async runs also skipped the worker-side check via admissionCompleted). External callers may now only send the redundant 'api' value; internal JWT callers ('workflow'/'mcp') are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * refactor(execution): extract enqueue/status/cancel into shared libs Prepares the v2 execution surface: handleAsyncExecution's queue logic moves to lib/workflows/executor/enqueue-execution.ts (slot/claim semantics encoded in a discriminated outcome, not HTTP statuses), the execution-status read to execution-status.ts, and the order-sensitive cancel machinery to lib/execution/cancel-workflow-execution.ts. The v1 routes re-render identically — their suites pass unmodified. Also: preprocessExecution gains rateLimitCounter ('sync'|'async') and its 429 now carries code RATE_LIMIT_EXCEEDED + retryAfterMs (previously indistinguishable from the concurrency 429 and Retry-After was discarded); and the duplicate cancel contract in contracts/logs.ts is unified on the full 5-value reason enum — its narrower copy made requestJson throw a client ZodError when cancelling a paused HITL run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(execution): callable execution service + structured error classifier executeWorkflowService composes the same libs the v1 route holds inline (call-chain guard, execution-id claim, LoggingSession, preprocessing, deployed-state load + file-field processing, timeout-bound executeWorkflowCore, output hydration/compaction) for the deployed-state caller class — the seam the v2 execute route and in-process internal callers share, making the HTTP endpoint syntactic sugar. classifyExecutionError stops discarding the block context that buildBlockExecutionError already attaches at throw sites: failed runs now yield {message, code, blockId, blockName, blockType} with a stable append-only code enum (TIMEOUT/CANCELLED/USAGE_LIMIT_EXCEEDED/ INVALID_INPUT/BLOCK_EXECUTION_FAILED/CHILD_WORKFLOW_FAILED/ OUTPUT_TOO_LARGE/EXECUTION_FAILED), so callers route on error class instead of substring-matching messages — the single place raw errors are interpreted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): POST /api/v2/workflows/[id]/execute Thin route over executeWorkflowService: X-API-Key or anonymous public-API auth (sync/stream only for anonymous), strict body with body-flag async (no mode headers on v2), SSE passthrough for stream, and the execution resource response — executionId always present, in-band run failures are status:'failed' with the structured {message, code, blockId, blockName, blockType} error, sync timeout is status:'failed' + TIMEOUT instead of v1's 408, and a Response block's payload stays inside output (authors never control response status/headers on this origin). Async debits the async bucket and the 202 statusUrl points at the v2 executions resource. Adds CLIENT_CLOSED_REQUEST/SERVICE_UNAVAILABLE to the v2 error codes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): v2 executions status + cancel with queued backfill GET /api/v2/workflows/[id]/executions/[executionId] is the single status URL for sync and async runs: before the async worker writes the durable log row, status is backfilled from the job queue (deterministic job id) as 'queued'/'running' — closing v1's 202-to-pickup 404 window — and failed runs carry the structured error object. POST .../cancel renders the shared cancellation lib in the v2 envelope with the tightened 5-value reason enum. Both authenticate via the shared resolveV2WorkflowAccess (X-API-Key, authz masked as 404, allowPersonalApiKeys honored). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(execution): workflow tool + MCP bridge run in-process workflow_executor (workflow-as-agent-tool) short-circuits in executeTool through WorkflowBlockHandler — the same invocation boundary canvas child workflows use — mirroring the deployed_block_executor precedent. The MCP serve bridge calls executeWorkflowService directly instead of fetching its own execute endpoint; deployment-version pinning, MCP response-size rejection, and the actor override become typed options instead of header sniffing. Both callers drop the double admission slot and duplicate top-level log row the HTTP hop cost, and failed child runs now surface the structured error + child executionId so parents and MCP clients can route on error class and hand providers a reproducible handle. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(infra): CORS + CSP coverage for the v2 execute path /api/v2/workflows/:id/execute gets the same wildcard-origin, credential-free CORS policy as v1 (the default credentialed policy would block browser API-key calls and open a cookie CSRF surface) with X-Sim-Stream-Protocol allowed and no X-Execution-Mode (async is body-selected on v2), plus the COEP/COOP/CSP header block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(ui): deploy modal + copilot advertise the v2 execute surface All 20 API-tab snippets move to POST /api/v2/workflows/{id}/execute with the nested {"input": ...} body, async as the "async": true body flag (X-Execution-Mode gone), status polling against the v2 executions resource, the third tab renamed Usage and pointed at /api/v2/billing/usage, and {data} envelope unwraps in the printed responses. Fixes the latent baseUrl derivation (endpoint.split('/api/workflows/')) that would have silently built garbage URLs under a v2 endpoint, and deletes dead code (exampleCommand across 3 sites, getAsyncExampleTitle). Copilot deploy/manage/serializer endpoint builders and the api_trigger bestPractices example follow (the latter also drops its hardcoded staging host). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * docs(api): document the v2 execution surface Adds execute, execution status, and cancel to openapi-v2-workflows.json with the structured ExecutionError schema (append-only code enum + block attribution) and the ExecutionResource contract, documenting the rules that differ from v1: modes are body-selected, a failed run is HTTP 200 with status 'failed', an executionId always means data (never the error envelope), queued status is visible immediately, and Response-block payloads stay inside output. Registers the three pages in the generated workflows meta.json and bumps the route-count baseline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): gate the whole /api/v2 surface behind one flag; UI stays on v1 Every v2 route now runs exactly one check immediately after auth — v2ApiGateError — and answers 404 when the `v2-api` flag is off, so the surface is invisible until it is deliberately rolled out. The gate is keyed on userId only: a workspace/org-keyed check would have to read membership for a caller-supplied id before authorization runs, and its 404-vs-403 split would leak cohort membership (the trap the per-domain table gate worked around by running late). The two executions routes inherit it from the shared access resolver; the tables-specific gate is removed so no route checks twice. `tables-v2-api` stays, now gating only the internal predicate-grammar route /api/table/[tableId]/query — note v2 tables routes move to the unified flag, so enabling them is a `v2-api` decision now. Reverts the deploy modal, copilot handlers, and api_trigger example to the v1 execute endpoint: v1 works unchanged, and the UI must not advertise a surface most users would get a 404 from. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(executor): restore child-cost aggregation dropped by the staging merge Staging's custom-block rewrite deleted `aggregateChildCost` from workflow-handler.ts, and git merged that file cleanly — but this branch's workflow-tool-runner.ts, added for the v2 execute migration, still imports it. A silent semantic conflict: no marker, broken build. Taking staging's rewrite is correct, so the helper is defined locally in its one remaining consumer rather than resurrected in the file staging just rewrote. Same four lines over the still-exported `calculateCostSummary`, so a failed child workflow keeps billing the hosted-key spend it consumed instead of reporting $0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): make lib/table/orchestration the single implementation (#6134) * refactor(orchestration): move the shared error contract out of lib/workflows OrchestrationErrorCode and statusForOrchestrationError are the contract every lib/[resource]/orchestration module returns against, but they lived inside the workflows module, so resource-neutral code (lib/folders) already had to import from a workflow path. Moved to lib/core/orchestration/types. Adds a 'locked' class mapping to 423. Both tables and workflows have a lock that forbids a mutation, and each caller was translating that to a status itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): make lib/table/orchestration the single implementation Column update was implemented four times — the UI route, v1, v2, and the copilot table tool — each calling the same column services but owning its own guards, error mapping, and audit. The copies had drifted, and the drift was the bug: v2 was missing both guards, only the copilot copy minted stable option ids, and only v1/v2 audited. performUpdateTableColumn, performDeleteTable, and performDeleteTableRow now own that logic; all ten call sites reduce to auth, parse, call, render. The guards are asserted once in lib/table/orchestration rather than four times against four routes. Behavior this consolidates, previously true on only some paths: - The typeChanging guard. updateColumnType early-returns on an unchanged type and drops any options sent with it, so restating the current type alongside new options silently discarded them. v2 had no guard at all and, since its contract shares v1's body schema, accepted options and ignored them. - The select-unique guard. Each write is its own locked transaction, so a rename or type change paired with a constraint write that is going to fail commits first and then throws, half-applying the schema change. - Stable select-option ids. Cells reference the option id, so an edit that re-sends an option by name has to reuse it or every cell holding it is orphaned. Only the copilot path did this; normalizeSelectOptionsInput moves to lib/table/select-options and now covers every caller. It preserves a supplied id, so it is a no-op for the fully-formed options the HTTP contracts accept. - required forwarded into the type and options writes, so a conversion validates against the constraint the same request is setting. - An audit on every successful update. The UI route and the copilot tool emitted none. - Single-row delete through the row service. v2 did a raw db.delete, skipping assertRowDelete and deleteOrderedRow, so a delete-locked table returned 200 and the row-count bookkeeping never ran. - The delete actor handed to deleteTable, which audits only when a row was actually archived. v1 and v2 omitted it and audited themselves outside that check, emitting TABLE_DELETED for a no-op delete of an archived table. Failure classes come back as OrchestrationErrorCode; v2 renders them through a new v2ErrorForOrchestration, mirroring statusForOrchestrationError on the v1 and UI surfaces, so a given failure maps to the same status everywhere. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(tables): bind the column-update tests to the orchestration function The base's route tests assert which column service each payload reaches — the behavior that now lives in performUpdateTableColumn. They mocked the `@/lib/table` barrel; the orchestration module imports the service directly, so they mock that too and keep asserting the same thing through the extracted implementation. The orchestration tests move onto the base's semantics: writes address the stable column id, a rename rides inside the write it accompanies rather than running first, and the currency guards replace the non-select options guard the service now owns. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(copilot): drop the column-type import the delegation made dead Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): move the audit log out of the table service `lib/table/service.ts` wrote its own audit rows, so whether an operation was audited depended on which function a caller reached for rather than on a user having performed it. That is what let v1 and v2 audit a no-op delete, and what made `deleteTable`'s optional `actingUserId` double as an audit opt-out flag. Worse, most sites fell back to `actingUserId ?? createdBy`, so an unattributed call was logged against the table's *creator*. The copilot `mv` path passed no actor at all: renaming someone else's table recorded them as the renamer. Audit now lives in the orchestration functions — performDeleteTable, performRenameTable, performMoveTableToFolder, performUpdateTableLocks — and the services just write. Internal callers (folder cascade, import rollback) keep calling the service and are silent by construction rather than by remembering to omit an argument. Two services now return what the audit needs: `deleteTable` reports whether it actually archived a row, so a repeat delete logs nothing; `updateTableLocks` returns the before/after locks, since only the locked write can observe the transition its description names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(tables): restore audit provenance and conflict status in orchestration Moving the audits into the orchestration functions dropped three things the routes had been carrying, and added one the orchestration now owns twice. - The v1 and v2 column-update routes passed `request` to `recordAudit`, so their audit rows recorded the caller's IP and user-agent. The orchestration function had no way to receive it. Every table orchestration function now takes an optional `OrchestrationRequestContext` and every HTTP route forwards it; the copilot and VFS callers, which have no request, omit it. - `classifyTableMutation` matched `TableConflictError` on "already exists" appearing in the message and reported it as `validation`, turning the UI route's 409 on a duplicate table rename into a 400. It now matches the type, the way `performRestoreTable` already did. - `captureServerEvent` ran on every delete while the audit was gated on a row actually being archived, so a repeat delete of an archived table still reported `table_deleted`. Both now hang off the same evidence. - The copilot delete path kept its own `captureServerEvent` from when the service did not emit one, double-counting every copilot table delete. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a * fix(tables): say which type a no-op column update restated A copilot `update_column` payload whose only content was the column's current type used to return success with the live schema, while the v1, v2, and UI routes rejected the same payload with "No updates specified". Delegating to `performUpdateTableColumn` unified them onto the routes' rejection — correct, but the message tells the caller its request was empty when it named a type. The orchestration function now reports the same thing `updateColumnType` reports when it loses this race concurrently: the column is already that type, re-issue without the type change. An empty payload still reads "No updates specified". Drops the copilot's `outcome.table ?? tableForUpdate` fallback with it — the comment described the no-op that can no longer reach that line, and a success always carries a table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a * refactor(tables): classify failures by type instead of by message text The table module decided HTTP statuses by searching error messages for phrases. `VALIDATION_MESSAGE_FRAGMENTS` and `ROW_WRITE_ERROR_PATTERNS` held 32 substrings between them, and fifteen more lists were inlined in routes — 83 matchers over 17 files, each its own copy of the guesswork and already drifted apart. It made message wording load-bearing: `TableRowLimitError`'s own doc comment noted that its text had to contain "row limit" for a route to answer 400, and adding "already exists" to a rename message silently demoted a 409 to a 400 (the bug fixed one commit ago, by adding another special case). Services now throw `OrchestrationError`, which carries the transport-neutral `OrchestrationErrorCode` the layers above already speak. Classification is one `instanceof` in `orchestrationErrorResponse` (UI + v1) and `v2CaughtOrchestrationError` (v2). Every pattern list is gone. Wording is free to change; an unclassified error still becomes a generic 500, which is what an unexpected fault should be. `asOrchestrationError` walks the `cause` chain rather than testing the caught value directly: drizzle wraps a throw raised inside a transaction callback in a `DrizzleQueryError` whose own message is the failed SQL, so a bare `instanceof` would drop every failure raised inside `withLockedTable`. That is the same reason `rootErrorMessage` had to dig for a root cause before. Three throws stay bare `Error` deliberately — `Table ID mismatch`, `Workspace ID mismatch`, and `Failed to build upsert conflict predicate` are internal invariants no consumer classified, and they keep falling through to a 500. `Insufficient capacity` was in the pattern list with no producer anywhere in the codebase. Status changes, all deliberate: - `'forbidden'` joins the code union so the table-row-limit ceiling keeps its 403; without it this refactor would have flattened it to 400. - import-async's table-limit rejection: 400 -> 403, matching the two other create routes it had drifted from. - Renaming a table to an invalid name: 500 -> 400. `validateTableName` messages don't contain "Invalid", so no matcher ever caught them. - Restoring a table that isn't archived, or into an archived workspace: 500 -> 400. - A duplicate *column* name stays `validation`/400 rather than becoming a 409 like a duplicate table name. Both v1 and the orchestration have always answered 400 for it; changing a published status is not this refactor's job. The twelve tests that changed were asserting the substring mechanism itself, constructing plain `Error`s with magic strings. They now assert the real contract, plus new cases pinning that identical wording carrying no classification stays internal and keeps its message off the wire. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials (#6150) * feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials * fix(api): correct credential role, skill permission bar, MCP url identity, and custom-tool conflict mapping * fix(api): align credential mutation gating, provider-outage status, and unique-violation conflicts * fix(api): close unique-violation, revival, orphan-write, and env-rename gaps * fix(api): treat every provider-outage code as unavailable on create and update * fix(credentials): use the shared outage predicate on the session update path * fix(contracts): anchor the predicate double-cast annotation to the cast `check:api-validation:strict` counted 9 unannotated double-casts against a baseline of 8, failing CI. The predicate leaf schema was annotated, but the annotation sat above the declaration while the checker anchors on the line carrying the cast — five lines below, at the close of the object literal. The scanner walks back at most three lines and stops at the first non-comment one, so it hit `value: z.unknown().optional(),` and never saw the reason. Splitting the object schema from the cast puts them adjacent, so the existing reason binds. No behavior change — the cast, the schema, and the reasoning are unchanged. Also lowers the rawJsonReads ratchet 6 -> 5 to match the current count, which had drifted down; leaving it high lets a removed raw read silently come back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(skills): point the orchestration error contract at its moved module #6150 branched before #6134, so skill-lifecycle.ts imports @/lib/workflows/orchestration/types — the module #6134 moved to @/lib/core/orchestration/types. Git merged a file deletion on one side with a new file referencing it on the other: no textual conflict, broken build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(knowledge): make lib/knowledge/orchestration the single implementation (#6154) * refactor(knowledge): make lib/knowledge/orchestration the single implementation Knowledge base create was implemented four times — the internal route, v1, v2, and the copilot tool — and the orchestration around the shared write had drifted. Extract it the same way lib/table/orchestration was: services write, orchestration decides which writes run, guards them, audits them, and returns a transport-neutral failure. Behavior converged, not preserved: - One chunking default (DEFAULT_CHUNKING_CONFIG). The agent defaulted minSize to 1 against the API's 100, so identical input produced differently-chunked knowledge bases depending on who created it. The agent path now chunks at 100. - Every successful mutation is audited inside the orchestration function. The copilot tool called recordAudit zero times, so agent-created knowledge bases, document uploads, updates and deletes left no audit trail at all. - Failures classify by class, not by message text. The knowledge service errors are OrchestrationError subclasses and storage-quota rejections throw a shared StorageLimitExceededError, replacing four separate message greps for "already exists" / "does not have permission" / "storage limit". delete_connector reported the opposite of what happened. It reached the route through an internal HTTP self-call that sent no query string, so the route's keep-documents default always applied while the agent told the user the documents had been removed. The self-call is gone — all four connector operations run in-process — and the orchestration returns the real counts. Also: - OrchestrationErrorCode gains 'payload_too_large' (413 / PAYLOAD_TOO_LARGE). Without it, dropping the storage-limit message match would have regressed the documented 413 on knowledge base create and document upload to a 500. - messageForOrchestrationError renders a route's own wording for an unclassified fault, so a driver's message no longer reaches the client on a 500. - v1 and v2 knowledge base update now forward actorUserId, which the service requires for a workspace move; both omitted it. - The connector DELETE route reads deleteDocuments through parseRequest. Its contract declared z.boolean(), which would have rejected the string a query param actually is. - Drop the 409 from POST /api/v2/knowledge/{id}/documents in the OpenAPI spec. Nothing on the upload path throws a conflict; it was only ever reachable by the message match this change removes. Behavior change worth noting: a v1/v2 PUT carrying only the workspaceId scope field and no actual updates now returns 400 rather than 200 with the unchanged knowledge base. Deliberately deferred: document update remains internal-only. Extracting performUpdateKnowledgeDocument makes exposing it on v1/v2 a contract and a route away, but that is a new public surface rather than part of this consolidation. * fix(knowledge): make connector create atomic and stop flattening failures Review round 1 on #6154. - Resolve the billing payer before the connector is committed, not after. A malformed attribution header rejected post-commit left a live connector behind a 500, and a retry created a duplicate plus duplicate sync work. Manual sync resolves before writing its audit for the same reason. - Let the source-config validator carry its own failure class. Collapsing every rejection to `validation` flattened the connector PATCH route's 401 (stale stored credential) and 409 (missing workspace context) into a 400. - Add `unauthorized` to OrchestrationErrorCode. It is the class that 401 was already expressing on this route, and the v2 vocabulary already had UNAUTHORIZED; only the shared union was missing it. - Report a knowledge base that exists but failed to archive as failed, with the reason, rather than as not found. The copilot delete loop folded every non-not-found failure into `notFound`, telling the user it was never there. - Route copilot failures through the same message helper the HTTP surfaces use, so an unclassified fault's raw text (a driver's failed SQL) no longer reaches the agent verbatim while the UI and public APIs get the generic wording. * feat(api): expand the public v2 files surface (#6160) * feat(api): expand the public v2 files surface Adds folder support, rename/restore, move, bulk archive, share, and content replace to /api/v2/files, so managing files by API no longer stops at upload + download + archive-one. Routes are thin: auth -> parse -> perform* -> serialize. Share and content replace get their orchestration extracted first so the session routes and the public ones cannot diverge on the effective-authType resolution, the EE public-sharing gate, or the storage-quota classification. Presigned upload stays session-only: presign does an advisory quota check and the real debit happens in the separate register step, so a caller that never registers leaves unaccounted bytes with no reaper. The buffered multipart path debits inside uploadWorkspaceFile's own transaction. * fix(files): classify folder and content failures instead of 500ing them Bugbot round 1. The v2 routes map errorCode straight to a status, so every manager failure that arrived unclassified became a 500 for what is really a caller-fixable 400 or 404. - Folder manager throws OrchestrationError: missing target/folder -> not_found, reparent cycle / self-parent / restore-into-archived-workspace -> validation. - File manager does the same for the in-transaction 'File not found' paths that the earlier pass missed. - updateWorkspaceFileContent's outer catch re-wrapped everything in a bare Error, which stripped the class off StorageLimitExceededError and the new not_found alike. It now rethrows a classified failure untouched and attaches cause to the generic wrap, so asOrchestrationError can still walk the chain. - Every remaining perform* gained the asOrchestrationError branch. - renameWorkspaceFile returned the pre-update read, so the v2 PATCH reported a stale updatedAt; it now returns the timestamp it actually wrote. Docs: upload auto-suffixes a duplicate name rather than rejecting it, matching the in-app uploader. The description claimed 409 and was simply wrong. * fix(files): surface a failed upload read-back as the real error getWorkspaceFile swallows a query failure and returns null unless throwOnError is set, so a transient blip on the post-upload read reported as 'file could not be read back'. Distinguish the two: a real null after a just-committed write is an invariant break, a query failure is itself. * revert(api): drop the dedicated v2 file-folder routes File folders already live in the shared folder table as resourceType 'file' (#6045 cut them over, #6051 dropped workspace_file_folders), and the remaining file-specific folder machinery is being folded into the generic folder engine. Publishing /api/v2/files/folders/** would pin that transitional split into a public contract we'd then have to keep or break. Files stay folder-aware — folderId/folderPath on the projection, folderId on upload, and the move route — because a folder id is a folder.id and survives the unification untouched. Folder management belongs on /api/v2/folders once that surface serves resourceType 'file'; until then there is no v2 way to enumerate file folders, which is the deliberate gap. The orchestration classification fixes stay: the internal routes and the copilot file-folder tools still call those perform* functions. * fix(files): classify upload failures instead of matching their wording Bugbot round 2. uploadWorkspaceFile had the same outer-catch rewrap that updateWorkspaceFileContent did, so a blown storage quota reached the route as a bare Error and the v2 handler recovered the status by substring-matching the message. Any rewording silently demoted a 413 to a 500. - uploadWorkspaceFile rethrows a classified failure untouched and attaches cause to the generic wrap. - FileConflictError is now an OrchestrationError('conflict'), so a duplicate name classifies like every other conflict. Its 'FILE_EXISTS' discriminator had no readers and is gone; the instanceof checks elsewhere still hold. - The v2 upload handler uses v2CaughtOrchestrationError, dropping all three string matches. Also documents that bulk-archive is best-effort: unknown or already-archived ids are skipped rather than failing the call, and deletedItems is what actually happened. That asymmetry with the single-id DELETE was undocumented. * feat(api): add search, filtering, and sorting to the v2 list endpoints (#6189) * feat(api): add search, filtering, and sorting to the v2 list endpoints One convention across every v2 list, documented on lib/api/contracts/v2/shared.ts: `search` (case-insensitive substring on the resource's natural name field), `sortBy` + `sortOrder` (per-resource enum, never a free string), and enumerated resource-specific filters. Reuses the sortBy/sortOrder pair v2 logs and v2 knowledge-documents already ship rather than inventing a third dialect alongside the Logs filters and the Tables predicate grammar. Every filter and sort is pushed into SQL. GET /api/v2/files previously read the whole scope and sorted/sliced it in JS; it now goes through a new queryWorkspaceFiles that filters, orders, and bounds the page in one query. Cursors are stamped with the sort they were minted under, so replaying one under a different sort is a 400 instead of silently duplicated or skipped rows. * fix(api): validate v2 cursor key values and compare timestamps at ms precision Two review findings, fixed at the root by making a keyset key own its cursor codec instead of hand-writing a decoder per sort. Cursor key values are caller-controlled, and matching the sort stamp and key count was not enough: an unparseable timestamp or a non-numeric size reached the query as an Invalid Date or NaN and surfaced as a 500. Each key now type- checks its own value and rejects a cursor it cannot hold, which both routes render as the documented 400. Timestamp keys now order and compare on date_trunc('milliseconds', col). Postgres keeps microseconds and defaultNow() populates them, but a cursor value round-trips through a millisecond-only JS Date — comparing the raw column against the truncated value re-admitted the page's own last row, duplicating it and stalling pagination outright at a page size of one. Reachable today via workspace_files.updated_at, which insertFileMetadata leaves to defaultNow(). * feat(api): complete the v2 workflows resource with versions and CRUD (#6184) * feat(api): complete the v2 workflows resource with versions and CRUD Adds version listing/detail plus create, update, and delete to the v2 workflows surface, which previously covered only execution and deployment. - GET /api/v2/workflows/[id]/versions — cursor-paginated, newest first - GET /api/v2/workflows/[id]/versions/[version] — version + pinned state - POST /api/v2/workflows, PATCH and DELETE /api/v2/workflows/[id] All six delegate to the existing orchestration and persistence helpers; no new domain logic. * fix(api): check folder containment before lock state; reject malformed version cursors assertFolderMutable walks a folder's ancestor chain without filtering on workspace, so inspecting it before containment let a caller tell a locked folder in someone else's workspace (423) from a nonexistent one (400). Create and update now assert containment first, matching the ordering import-workflow.ts already uses. A version cursor that decodes to JSON without a numeric version filtered every row out and returned an empty page with nextCursor null, which reads as a clean end-of-list. Malformed cursors are now a 400. * refactor(api): page workflow versions in the persistence helper listWorkflowVersions read every version row and the route filtered and sliced the result in memory, so the response was bounded but the query was not. It now takes optional limit/afterVersion, turning the cursor into a real keyset query; the route asks for limit + 1 and only trims the has-more probe. Both params are optional, so the internal, v1 admin, and copilot callers are unchanged. Also restores the untouched GET handler in [id]/route.ts to its original formatting — collapsing its signature had re-indented the whole body and buried the actual additions in whitespace churn. * feat(api): expand v2 tables with stateless multipart transfers (#6188) * feat(api): expand the public v2 tables surface Adds 16 operations so a v2 caller can do what the internal surface can: rename/move/lock a table, restore it, manage saved views, run enrichment columns, look up rows, and import/export with observable job control. Extracts lib/table/orchestration/import.ts (performTableCsvImport, performCreateTableFromCsv) and lib/table/export-stream.ts from the first-party routes, then repoints those routes at them, so v1 and v2 cannot drift on what an import or export actually does. events/stream, metadata and dispatches stay internal — they are editor state, not public API. * fix(api): make v2 table PATCH all-or-nothing and name the lock in every 423 Greptile P1: PATCH applied locks, rename and move as three sequential transactions, so a folder rejected mid-request left the earlier writes persisted while the response reported failure — and the schema-changed signal was skipped, leaving open clients on stale state. Every rejectable condition now runs before the first write, and the signal fires whenever anything did land. Cursor: v2TableLockError dropped the lock kind, so async import, column run, enrichment and table mutations returned a bare LOCKED. A table has four independent locks, so the caller could not tell which to clear. * fix(api): report the lock kind on classified 423s too, not just thrown ones The previous commit named the lock only where the rejection was thrown and caught at the route boundary. Where it instead arrives as a classified `errorCode: 'locked'` outcome — delete table, delete row, update column, and the table mutations — the kind was dropped, so those 423s stayed unactionable while their neighbours improved. The orchestration results now carry `lock`, and a shared `v2TableOrchestrationError` renders both arrival paths into the same `{ code, message, details: { lock } }` body. `details` is omitted rather than sent null when the kind is unknown, so a caller branching on it sees absence instead of a phantom value. * fix(api): make async table imports observable, not just startable `POST /import-async` pointed callers at `GET /api/v2/tables/jobs` to track progress, but that endpoint filters to `type = 'export'` — imports are derived onto the table itself, one write job at a time, and exports get a separate list precisely because they are excluded from that derivation. The public Table shape omitted those derived fields, so an async import could be started and cancelled but never observed to completion, failure, or progress. That is the gap the import/export/job-control set was meant to close. Table now carries `job` — id, type, status, rowsProcessed, error, or null when idle — and the import-async docs point at the table rather than the export list. * feat(api): make v2 table PATCH state which operations landed on failure Greptile held the PR at 4/5 on the residual non-atomicity and named two acceptable resolutions: make PATCH atomic, or have the contract adopt and expose partial-success explicitly. Atomicity would mean threading one transaction through renameTable, moveTableToFolder and updateTableLocks — three shared service functions with four non-test callers including the first-party route and two copilot tools — and deferring their per-operation audits to commit time. That is a refactor of shared write paths well outside this PR. So the contract states it instead. Every rejectable condition is already pre-validated, so a failure here is a genuine fault; when one follows a successful operation the error now carries `details.applied` listing what is live. Absent when nothing applied, so its presence always means "these changes took effect despite the error". Documented on the operation. `v2ErrorForOrchestration` gained the optional `details` this needs. * fix(api): make table lock flags read-only on the public v2 surface The new PATCH /api/v2/tables/[tableId] accepted a `locks` object, gated on workspace admin plus the table-locks feature. That still lets an API key clear the guard placed there to stop it: `write` is the floor for the endpoint, and admin keys are ordinary API keys, so a lock is no longer a boundary the key cannot cross. Locks stay readable on the table resource and enforcement is unchanged (a locked verb still returns 423). Changing one is now a first-party admin action only. The v2 body is declared here rather than reusing the first-party updateTableBodySchema, which keeps its `locks` field so the UI can still toggle them. It is .strict(), so a request carrying `locks` is rejected with a 400 naming the field instead of silently succeeding without applying it. * fix(api): keep reporting applied operations when the PATCH re-read fails The composite table PATCH promises that `error.details.applied` names the operations that are live despite an error, but `applied` was scoped inside the try. A rename or move that committed and was then followed by a throw in the final re-read — or a re-read finding the table archived — returned a bare 500/404 with no details, telling the caller nothing had landed. It would then retry into a duplicate-name conflict or repeat the move. `applied` is now function-scoped so every post-write exit carries it: the 404 on a missing re-read, a thrown lock error, a classified orchestration error, and the generic 500. `v2TableLockError` gains the same `extraDetails` parameter `v2TableOrchestrationError` already had. * feat(api): add workflow group writes to the v2 tables surface v2 exposed GET /groups but none of the writes, so the public API could run an enrichment or workflow column and read its binding, but never create one. A caller could add a plain data column and trigger the machine; wiring the two together still required the UI. Adds POST/PATCH/DELETE on /api/v2/tables/[tableId]/groups. The group is the unit that fills columns — one group feeds several — so creating one creates its output columns in the same call, matching the first-party shape rather than inverting it onto the column endpoint. Four departures from the first-party body, all public-surface concerns: - group.id is optional and server-generated. The UI mints an id to render optimistically; a public caller has no such need and a client-chosen id is a collision waiting to happen. - outputColumns[].workflowGroupId is dropped from the body and stamped from the resolved group, so it cannot disagree with it. - autoRun defaults to false. First-party defaults true so a UI add fills cells immediately; here it would make one POST fan out a metered run across every existing row. - A group naming neither a workflowId (type manual) nor an enrichmentId (type enrichment) is a 400 rather than a half-specified group the route has to guess about. Also rejects an outputColumns entry no group output feeds — the two arrays are joined by column name, and the first-party client builds both from one picker so it cannot desync, but a public caller can. Workspace containment on workflowId is asserted before it is persisted, on create and on any update that re-points the group; without it a table becomes a way to invoke workflows the key cannot otherwise reach. * improvement(api): make v2 table import and export async-only Drops the three synchronous entry points: POST /tables/[tableId]/import, POST /tables/import-csv, and GET /tables/[tableId]/export. Sync import tied a write to the lifetime of an HTTP request. The body *was* the data, so it carried a 10 MB cap that Next silently truncates past — a partial import reporting success. It also had no job, so a timeout mid-write left rows in place with nothing to poll and nothing to cancel. The async path reads the file from storage instead: upload via POST /api/v2/files for a key, start with POST /import-async, watch GET /tables/[tableId] -> job, stop with POST /job/cancel. Sync export carried no such hazard, but one shape per operation beats two: with both removed the surface has exactly one way to move a table in or out, and the CLI wraps the extra calls. This also removes the last multipart handling in v2 tables. Those were the only routes bypassing parseRequest — form fields were parsed by hand against separate form schemas, outside the contract system every other v2 write goes through. Create-a-table-from-CSV is now two calls: POST /tables, then /import-async with createColumns. csvImportModeSchema is append|replace, so there is no single-call create. Route baseline 1064 -> 1061. * docs(api): correct the import-async note about upload size limits The docstring claimed there is no synchronous upload endpoint and so no request-body size cliff. Both are wrong: POST /api/v2/files is a synchronous multipart upload with a 100 MB cap, and it is the only v2 upload path (presigned is deliberately absent). What async-only actually bought: the cap went 10 MB -> 100 MB, it fails on an explicit size check and a bounded body read rather than a proxy cap that silently truncates, authorization completes before any body is buffered, and the table write is a job that can be watched and cancelled. * feat(api): unify file and table transfers * improvement(api): make multipart transfers stateless * fix(api): make table import completion retries idempotent * feat(v2-tables): paginate the table list `GET /api/v2/tables` returned every table in the workspace in one response — it used the cursor envelope but hardcoded `nextCursor: null`, and had no `limit`. That was defensible when tables were only created through the UI; `POST /api/v2/tables` is public now, so a script can create them in bulk and the list has no way to ask for less. Adds `queryTables` alongside `listTables` rather than changing it, so the internal callers that genuinely want the whole scope are untouched — the same split `queryWorkspaceFiles` / `listWorkspaceFiles` already uses. Filter, order and slice all run in the query, so a `search` never costs a full-workspace read. A cursor whose values don't bind raises a validation error instead of being coerced to "no filter", which would have silently served page 1 under a resumed cursor. The keyset closes on `id` so a page boundary inside a run of equal names or timestamps stays stable. The shared `LimitQuery` doc component said "Maximum rows to return"; it now serves the table list too, so the wording is resource-neutral. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(api): add multipart knowledge document uploads * fix(api): keep usage admission at knowledge upload session creation * feat(knowledge): wire knowledge base uploads to multipart sessions * fix(knowledge): refuse to abort an upload once a document is bound * fix(uploads): prevent multipart cleanup races * Unify file creation and signed upload sessions (#6264) * feat(uploads): unify signed upload sessions * fix(uploads): preserve attachment storage semantics * feat(files): add authored file creation * fix(uploads): omit hoisted S3 metadata headers * feat(api): add file metadata endpoint * improvement(api): scope folders to resource paths (#6284) * improvement(api): scope folders to resource paths * fix(files): serialize folder resolution with uploads * fix(files): release folder lock before upload setup * fix(api): normalize folder paths and unblock resource mutations * fix(api): make resource cleanup and metadata consistent * improvement(uploads): persist multipart sessions in postgres * fix(db): store table row trigger timestamps in UTC * improvement(api): default folder deletion to non-recursive * fix(billing): unify chat usage source * improvement(logs): expose trace spans on log detail * fix(logs): parse list trace spans * improvement(api): replace workflow jobs with execution resources (#6294) * improvement(api): replace workflow jobs with execution resources * fix(api): preserve legacy jobs while preferring v2 executions * fix(api): make execution polling resume-aware * fix(ui): hide async examples for public workflows * fix(api): bridge resume queue visibility lag * feat(api): add v2 workflow resume endpoint * fix(api): project pending resume attempts * fix(api): prefer terminal logs over stale resumes * improvement(api): unify v2 resource query layers (#6319) * improvement(api): unify v2 resource query layers * fix(api): address v2 review findings * fix(api): preserve cancelled queue status * fix(api): guard cancelled job transitions * fix(api): close v2 resume and log gaps * feat(api): rename v2 executions to runs * feat(api): split credentials and secrets * feat(api): add workspace metadata and email attribution * improvement(api): consolidate public v2 route handling * improvement(files): centralize operations across APIs and Copilot (#6392) * improvement(files): unify rename authorization * chore(skills): add file operation migration guide * improvement(files): consolidate file operation authorization * improvement(files): extract shared operation foundation * improvement(api): simplify internal route declarations * improvement(files): centralize application authorization * refactor(api): share workspace file name validation * refactor(files): centralize copilot application calls * docs(skills): generalize application operation migration * improvement(api): centralize remaining v2 resource operations (#6412) * improvement(api): centralize v2 resource operations * fix(api): preserve custom tool conflict errors * improvement(api): migrate policy-sensitive v2 reads (#6410) * improvement(workflows): centralize v2 application operations (#6411) * refactor(api): migrate v2 knowledge operations (#6413) * refactor(api): migrate v2 knowledge operations * fix(knowledge): fail upload completion on dispatch errors * fix(knowledge): preserve upload retry and VFS errors * improvement(tables): centralize v2 application operations (#6414) * improvement(tables): centralize v2 application operations * fix(tables): preserve run validation and signals * feat(auth): add scoped internal executor delegation (#6459) * feat(auth): add scoped internal executor delegation * fix(auth): derive delegation lifetime from one timestamp * Include share status in file metadata * feat(auth): centralize delegated identity policy (#6462) * improvement(copilot): consolidate application adapters (#6450) * improvement(api): harden application route boundaries (#6451) * improvement(api): harden application route boundaries * fix(folders): reject creates at workspace cap * fix(knowledge): enforce trusted workspace scope (#6452) * fix(knowledge): enforce trusted workspace scope * refactor(knowledge): declare v2 body lifecycle * finish knowledge application migration * refactor(knowledge): compose copilot batch commands * fix(knowledge): parse connector query flags * fix(knowledge): finalize partial batch effects * fix(knowledge): align merged application boundaries * fix(knowledge): close application boundary review gaps * style(knowledge): satisfy branch biome checks * fix(knowledge): page connector documents in editor * refactor: enforce Copilot table application boundary (#6453) * refactor: enforce copilot table application boundary * fix(tables): finish application boundary migration * fix(tables): restore scoped copilot imports * fix(tables): compose copilot commands atomically * fix(tables): preserve workflow group scheduling * fix(tables): complete fixed copilot composition * fix(tables): reject enrichment output mutation * fix(tables): complete authorized application boundary * fix(workflows): migrate Copilot application boundary (#6455) * fix(workflows): migrate Copilot application boundary * fix(workflows): finish delegated application migration * fix(workflows): encode VFS folder aliases * fix(workflows): close application composition gaps * fix(workflows): preserve VFS validation errors * fix(workflows): complete application boundary migration * test(workflows): format canonical binding coverage * fix(workflows): scope executor metadata reads * fix(workflows): bind executor metadata targets * improvement(skills): align application operation guidance (#6532) * feat(api): expose v2 resource owners * fix(api): distinguish visible resource authorization failures (#6537) * feat(api): generate v2 OpenAPI from contracts (#6509) * feat(api): generate v2 OpenAPI from contracts * fix(api): preserve string boolean wire defaults * fix(api): document file download headers * fix(docs): use TypeScript CLI with Next.js * fix(docs): avoid client-rendered theme script * fix(api): document departed audit default * feat(api): replace legacy core docs with v2 * feat(api): generate v2 OpenAPI from contracts * feat(api): refine generated v2 OpenAPI docs * fix(docs): align localized v2 execution examples * fix(ci): restore Helm diff and sync audit mock * fix CI regressions after staging merge --------- Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Theodore Li <theodoreqili@gmail.com> Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com> Co-authored-by: Theodore Li <theo@sim.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |