12 Commits

Include renames

Author	SHA1	Message	Date
WaleedandTheodore Li	656840a1b1	feat(api): make the platform operable headless over v2 (#6912) ... * feat(v2): download run output files by API key Adds GET /api/v2/workflows/{id}/runs/{runId}/files/{fileId}, closing the async-run loop for headless callers. A run's output carries UserFile URLs pointing at /api/files/serve/..., which rejects x-api-key outright, so an async run that produces a file previously had no byte path out for an API key at all. The file is addressed by the id the run reported and resolved against the run's own recorded execution data, from which the storage key is read. The request never supplies a storage key, so the endpoint cannot be aimed at bytes the run did not produce. Resolution deliberately reads the materialized-but-undisplayed recording, because the display projection strips exactly the `key`/`context` fields a byte read needs. Also hardens normalizeStartFile to derive a file's storage key only from a validated internal serve URL, discarding any caller-supplied `key`/`context`. A workspace API key has no human subject, so the executor resolves its actor to the workspace billing owner (preprocessing.ts -> resolveSystemBillingAttribution); verifyFileAccess then authorizes a workspace-context key as that owner, whose reach is not bounded by the key's workspace. Accepting an attacker-authorable key made that substitution exploitable as a confused deputy. Normalization is all-or-nothing, so a forged file now drops the whole files input. * feat(workflows): one graph-write door, principal-derived audit source, and v2 authoring endpoints Extract replaceWorkflowNormalizedState as the single persistence primitive for a workflow graph replace and route both the internal editor save and the Copilot edit tool through it, so neither can skip state preparation, the row lock, the lastSynced stamp, or custom-tool extraction by choosing a different entry point. Derive the audit source from the acting principal instead of hardcoding 'copilot', then widen workflows.variables.apply_operations and workflows.bulk.move to every principal kind. Add GET/PUT /api/v2/workflows/{id}/state, POST /operations, /duplicate, /restore, PATCH /variables, and POST /api/v2/workflows/move over surface-neutral application use cases; move the edit engine to lib/workflows/editing. * test(workflows): cover the graph-write primitive, audit source, and the v2 authoring surface Pin the two-doors fix (preparation runs, the row is locked, custom-tool extraction is post-commit and best-effort) and the false-audit fix (a session principal writes source: 'session', a delegated one writes its service). Both were verified to fail with the fix reverted. Add the application matrix for replaceWorkflowState, applyWorkflowOperations, readWorkflowGraph, and restoreWorkflow — role floor, principal-kind rejection before canonical load, asserted-scope concealment, lock, validation, atomic conflict, plan gate, and audit-then-notify ordering — plus route tests for every new endpoint. * test(workflows): pin the internal graph-write door and the v2 list scope Characterize saveWorkflowNormalizedState's statuses, messages, and notification after the persistence extraction, and cover the new scope filter on GET /api/v2/workflows including a cursor replayed under a different scope. * feat(api): add v2 block, tool, connector-type, and enrichment catalogs Adds six read endpoints under /api/v2 that publish Sim's code-defined catalogs: GET /blocks, GET /blocks/{blockId}, GET /tools, GET /tools/{toolId}, GET /connector-types, and GET /enrichments. These read like static reference data and are not. What a caller may place is decided per workspace by its permission-group integration allowlist, per organization by which unreleased blocks have been revealed, per deployment by ALLOWED_INTEGRATIONS, and per workspace again by the workflows it has deployed as blocks. So all six are plain defineWorkspaceOperation reads at minimumRole 'read' with workspaceApiKey 'allow' — the exact policy of credentials.providers.list — and every response keeps Cache-Control: private, no-store, because an unrevealed preview block's existence must not leak across organizations through a shared cache. Trigger blocks ride as ?capability=trigger rather than a second endpoint, and workspace custom blocks ride inside /blocks discriminated by `source`, so "what may I place?" stays a one-call question. The block projection is extracted out of the Copilot get_blocks_metadata tool and rewritten onto @/tools/metadata and @/tools/metadata-outputs. That cuts the tool's own @/tools/registry edge as a side effect: its module graph drops from 6,756 to 1,318, and the new routes land at 1,673-1,734, next to the shipped /v2/credentials/providers baseline of 1,668. Supporting changes: - scripts/sync-tool-metadata.ts derives hostedApiKey ('always' | 'conditional' | 'none') from each tool's `hosting`. The config itself stays excluded because it holds closures, but "does Sim host the key" is a first-order authoring question, so the answer is emitted. - getCopilotToolDescription takes hostedApiKey as an option instead of reading `hosting` off the tool, so both an executable ToolConfig and the generated metadata can answer it through one shared derivation. - principalUserId / allowedIntegrationTypes move out of lib/credentials/application/provider-catalog.ts into lib/integrations/principal-scope.server.ts. Two copies of the workspace integration gate would diverge first on the workspace-key path, which has no user for permission groups to key on. - scripts/check-tool-registry-boundary.ts walked page.tsx/layout.tsx under app/workspace only, so a route importing the executable registry passed green. It now walks a list of entry sources, seeded with the four catalog route subtrees and the shared projection barrel. Routes are covered per subtree rather than wholesale because 122 of ~1,130 route files legitimately execute tools. Registry sweeps parse every block, tool, connector type, and enrichment through its published response schema and compare against the wire round-trip. They caught a real drift while being written: an operation's inputs were typed as a union of the tool-param and block-input shapes, and the union resolved to whichever member matched first, silently dropping a block input's `schema`. * docs(api): stop publishing a 413 GET /workflows/{id}/state cannot emit * feat(v2): read upload-session state Adds GET /api/v2/files/uploads/{uploadId}. Only DELETE was exported, so a caller that lost track of a transfer could abort it but could not ask whether the session was still alive, already finalized, or failed — the resume story was missing. Runs on a new files.upload.read operation at minimumRole 'read' rather than reusing uploadCancel, which is a 'write': asking about a session must not require permission to destroy it. The GET is a control leg like every other, so it carries the signed upload token and re-authorizes the caller's present workspace permission through reauthorizeWorkspaceUploadPurpose instead of resolving the session on its id alone. * fix(api): reconcile v2 catalog and workflow-authoring integration Merging the catalog and workflow-authoring branches surfaced four issues that neither produced in isolation. - Route and OpenAPI counters were bumped to the same value on both branches, so git merged them as one change while the merged tree holds the sum. Corrects the route ratchet to 1142 and the workflows document to 29 operations (152 total), then regenerates the OpenAPI documents and the CLI surface from the reconciled contracts. - The seven new workflow operations were published in the spec but absent from the workflow API reference groups, which `check:openapi` rejects. - `route-policies.ts` reached `WorkflowOperationsNotAppliedError` through `apply-workflow-operations`, dragging the edit engine — and its diff and comparison dependencies, which reach a client OAuth hook — into every route that uses the shared workflow error policies. The class moves to its own leaf module, mirroring `WorkflowImportError`, and each importer now takes it from there. - The operations route test shadowed that class inside its module mock, so `instanceof` matched a fake and the assertion pinned a message the production class never emits. It now uses the real class and asserts the real message. * feat(v2): extract ZIP archives over the public API Adds POST /api/v2/files/{fileId}/extract and widens files.extract_archive from principalKinds ['session'] / workspaceApiKey 'deny' to admit personal and workspace API keys at the unchanged 'write' role. The widening is an authorization change, so the justification lives in the operation's TSDoc: extraction grants no capability an API key lacks, since every file it writes could be created one at a time through files.create and files.upload.create, both already 'allow' at the same role. It only collapses many calls into one. The previous ['session'] restriction read as an artifact of the UI having been the only caller. Delegated services stay out — no copilot or executor caller exists and admitting one is a separate decision. The response is counts plus the destination folderPath, never the extracted files: a large archive would otherwise materialize thousands of objects into one body. Callers page GET /api/v2/files?folderPath=... instead. The use case returns the internal display path and the adapter projects it to a v2 path, keeping the use case surface-neutral. * feat(v2): extract file text over the public API Adds GET /api/v2/files/{fileId}/text. Text extraction previously sat behind checkInternalAuth on /api/files/parse, a route that also mixes in external-URL fetching, execution-file upload, and multi-file aggregation, so it could not be reused. The parse call is lifted into a thin application use case instead. Runs on the existing files.read_content operation unchanged — it is already workspaceApiKey 'allow' at the read role, and turning bytes it already authorizes into text grants no further reach. `degraded` is a required, non-optional boolean on the response. The legacy doc and ppt parsers deliberately return best-effort or placeholder content rather than throwing, so an omittable flag would let a client that never checks it treat guessed text as extracted text. It is reported honestly rather than converted into an error, because the parsers' behaviour is deliberate and characterization-tested. The read is bounded on its input at 25 MiB before extraction rather than on its output after, given the parsers' documented DoS history; a caller may lower the ceiling but never raise it. * feat(v2): restore archived folders and list the archived set DELETE /api/v2/files/folders archives recursively, so a recursive delete was unrecoverable over the API: the archived files stayed visible through GET /api/v2/files?scope=archived, but nothing could rebuild the folder structure. Adds POST /api/v2/files/folders/restore, path-addressed like the rest of the v2 folder family, and a `scope` selector on the folder list so a caller can find the archived path to hand it. `scope` extends the files folder-list query rather than the shared v2ListFoldersQuerySchema: only workspace files have an archived folder set, so adding it to the shared schema would give tables, workflows, and knowledge a parameter they ignore. GET /api/v2/files/folders is a FULL_SET_LIST, not paged, so no cursor binding changes — list-pagination.test.ts passes unchanged. Restore resolves the archived folder from its path by scanning the archived set rather than walking the live tree, which by definition does not contain the folder being restored. The folder-restored analytics hook now reports the folder actually restored rather than the requested selector, which carries no id on a path-addressed surface. * feat(v2): bulk-download a file selection as a zip Adds GET /api/v2/files/bulk-download, an adapter over the existing downloadWorkspaceFileItems use case and its internal binary route. Path collision: a static segment beside [fileId] permanently shadows a file whose id equals it, and workspaceFileIdSchema does accept [A-Za-z0-9_-]+. Rather than invent a new shape, this follows the existing bulk-delete sibling: the hyphenated form cannot be produced by either minted id shape (UUID v4 or wf_<shortId>), so the shadowed id is unreachable in practice. Documented on the contract so the reasoning is not lost. Folders are addressed by path, matching the rest of the v2 file surface. The paths resolve against the folder set the selection already loads, so it costs no extra query, and a path matching no folder is rejected rather than silently dropped — a misspelled folder must not yield a zip of whatever else was selected. The empty-selection and folder-count guards now account for folderPaths, which a path-only selection would otherwise have tripped. Selections are comma-separated only: v2 rejects a query parameter sent more than once, so a repeated-parameter form would never reach the schema. Pinned by a test so the contract cannot advertise a form the boundary rejects. * feat(v2): expose run output files and optional inline bytes on the runs read GET /api/v2/workflows/{id}/runs/{runId} now reports the files a run produced, each with the downloadPath that fetches its bytes, and can inline them as base64 on request. Gated by includeOutput, matching `output`'s nullability: a caller that did not ask for output does not receive a file list it did not request. The async execute request's rejection of includeFileBase64 is deliberately left alone — at submit time the run has not happened, so there is nothing to inline; reading a finished run is the first moment the question means anything. Inlining is capped per file at the executor's 16 MiB inline ceiling, which a caller may lower but never raise. A file above it answers 413 naming that file's downloadPath, so the caller is told exactly how to get the bytes rather than being left stuck. The descriptor deliberately omits the storage key — files are addressed by id and the key is re-derived from the run's recording — and omits an expiry, which the recording does not carry and which would be fabricated if published. The route becomes headSafe: false, since inlining reads object storage. The builder enforces that this requires the use case to expose authorize(), so HEAD still answers from a real authorization rather than from authentication alone. * feat(v2): permanently delete an archived file DELETE /api/v2/files/{fileId} only archives — the OpenAPI says its stored bytes are never removed — so there was no way to actually destroy a file over the API. Adds the repository primitive, application use case, operation, and DELETE /api/v2/files/{fileId}/permanent. A distinct path rather than a flag on the ordinary delete: a query parameter that turns a recoverable archive into an irreversible destruction is set by accident, and the two acts carry different minimum roles, which one route declaration cannot express. The file must already be archived; a live file answers 409 naming the archive step, so no single request can turn a live file into lost bytes. minimumRole 'admin', which forces workspaceApiKey 'deny' since the workspace-key ceiling is 'write' — the desired policy anyway: unattended credentials should not destroy bytes. Row first, then object. The two legs commit independently, so one can survive a crash between them: deleting the row first leaves at most an orphaned object for the storage sweep, while the reverse would leave a live row pointing at bytes that no longer exist — a file that lists and opens but can never be read. A failed object delete is therefore reported as objectDeleted: false rather than thrown, because the request has genuinely succeeded once the row is gone. Both directions are pinned by failure-injection tests, verified to fail when the order is reversed. Audited as a distinct FILE_PERMANENTLY_DELETED action, not a reuse of FILE_DELETED, which records the recoverable archive step. * feat(api): v2 log analytics, itemized cost, filters, and sortable query Adds the aggregate and rich-read halves of the public logs surface, and fixes three defects the existing reads carry. Aggregate analytics. `GET /api/v2/logs/stats` returns time-bucketed run counts, success rate, error count, mean latency, and the window bounds, per workflow and for the workspace. The first-party route was a raw handler with inline SQL and inline aggregation, so it is split into a repository (`lib/logs/stats-queries.ts`), a pure aggregator (`lib/logs/stats.ts`), and an application use case. That route keeps its legacy authorization — it answers a caller without workspace access with a zeroed 200, where v2 conceals the workspace as a 404 — and consumes only the two surface-neutral halves. `segmentCount` had no `.int()`, `.min()`, or `.max()`, so `0` divided by zero and `1e9` allocated two billion-element arrays: both caller-reachable 500s. Bounded on both contracts. `workflows` is capped, with the workspace totals still computed from every workflow and the cut reported as `workflowsTruncated`. Detail reads gain the itemized `cost.items` ledger (`null` and `[]` are distinct answers and both reachable) and `workflowInput`, restoring a v1→v2 regression. The list gains `workflowName` and `status` filters, and `includeJobRuns`, which unions Chat and Sim-agent job runs into the sequence behind a new `kind` discriminator — without it a job run is indistinguishable from a run whose workflow was deleted. A filter no job row can answer drops the branch outright rather than meaning two things across the union. `POST /api/v2/logs/query` carries the additional sort columns. `GET /logs` is untouched: its single `order` param rests on there being exactly one sortable column, and both escapes from that are ruled out, so the rich read gets its own endpoint — the split the table surface already ships. It uses the shared keyset scheme with the two nullable sort columns read through a sentinel, since a keyset cannot compare against null. `folderPaths` now covers a folder's whole subtree on the public path, as it already did everywhere else; it previously omitted every nested run with no error. The path strings did not change, so a folder-scope version is stamped into the cursor and in-flight tokens restart rather than silently skipping rows. Also fixes `folderName`, which ILIKEd `workflow.name` — a copy of the clause above it — and so searched workflow names instead of folders. `buildLogSortCursorCondition`'s `IS NULL` disjunct is documented and pinned: under `NULLS LAST` the null block is only reachable through it, so removing it as a duplicate-row fix makes those runs unpageable. Ratchets: route count 1142 -> 1144; logs OpenAPI operations 2 -> 4; total operations 152 -> 154. * feat(api): v2 tables run state, dispatch polling, batch update, bulk, archive Closes the headless gaps on the v2 tables surface. - Per-cell run state is now readable through an opt-in `includeRunState` on `GET /rows`, `POST /query`, and `GET /rows/{rowId}`. The default projection is byte-identical; a page whose sidecar outgrows its byte budget is a 413 rather than a silent truncation. - Run dispatches are addressable: `GET /tables/dispatches/{dispatchId}` publishes the column's full four-state domain so polling a finished run is not a 500, and `GET /tables/{tableId}/dispatches` lists what is in flight. - `POST /rows/batch-update` takes one distinct patch per row. Its transaction moved out of the Copilot-only module into a surface-neutral use case both surfaces now call. - `GET .../enrichment/{groupId}` publishes the provider cascade, cost, and timing behind one enrichment cell. - `POST /tables/bulk-move` and `/bulk-delete` reach the existing bulk use cases, which now accept folders by canonical path and resolve them inside the application layer. - `DELETE` is recoverable: `scope=archived` on the table list plus `POST /tables/{tableId}/restore`. * feat(api): expose knowledge chunks, tag writes, archive/restore on v2 Closes the knowledge cluster's remaining public-surface gaps. Chunks: list/read/create/update/delete/bulk under `/api/v2/knowledge/{id}/documents/{documentId}/chunks`. `queryChunks` gains an `id` tiebreaker on every sort so the list pages on a keyset rather than an offset — `tokenCount` and `enabled` are both non-unique, so a page boundary inside a run of equal values used to repeat or drop the tied rows. The internal offset caller is unchanged; the two positioning schemes share one read. Tag definitions: create, update, delete, next-slot, usage, and the document-scoped save and cleanup. Without them a caller could write a tag value into a slot with no definition and then had no way to name it, so tag-filtered retrieval was unbuildable end-to-end. `v2KnowledgeTagSchema` gains `id`, without which PATCH and DELETE are unaddressable. The document-scoped DELETE is pinned to `action: 'cleanup'`: the domain's `'all'` deletes the whole knowledge base's tag vocabulary from a document path. Archive/restore: `GET /api/v2/knowledge/archived` as a sibling route rather than a `scope` param — the two reads bind different operations and a v2 route declares one — plus `POST /api/v2/knowledge/{id}/restore`. `knowledge.restore` is a new workspace operation carrying `delete`'s policy, since an operation's inverse must not be harder to reach; the internal session route now delegates its workspace branch to the shared use case and keeps only the legacy personal one. Also: `POST .../documents/from-workspace-files` surfaces `addWorkspaceFiles`, so a file already in workspace storage no longer has to be re-uploaded byte-for-byte to be indexed; the `chunkingConfig` write widens to the first-party five-key schema with its refines and separator bounds, while the response stays `.catchall` so a legacy JSONB row cannot 500; and `CONNECTOR_MANAGED_RESOURCE_READ_ONLY` joins `FORBIDDEN_DETAIL_CODES` now that the bare 403 on connector-managed chunk writes is wire-reachable. Document upsert is deliberately not included. * feat(api): add v2 credential rotation and a gate-exempt capabilities endpoint PATCH /api/v2/credentials/{credentialId} rotates service-account secret material or renames a credential in place, preserving the credential id so existing workflow, deployment, paused-run, connector, and webhook references keep working. Re-posting to POST /api/v2/credentials answers 409, and delete-and-recreate mints a new id, so rotation previously had no door. The route is adapter-only: updateWorkspaceCredentialUseCase already owned the rotation, its audit projection, and credentials.update. It gains one additive assertedWorkspaceId field for the v2 workspace assertion, and the per-principal credential-type table that deleteCredentialUseCase already applied is lifted into requireManageableCredentialType so both operations share it. Without it a personal API key could rename an env_workspace row and toV2Credential's throw would surface as a caller-reachable 500. CredentialProviderOperationError now maps to 503 with Retry-After when the provider is unreachable, instead of the 400 its OrchestrationError('validation') base projected. A transient outage rendered as a permanent input error invites a caller to revoke a working credential. GET /api/v2/meta reports the calling key's rollout cohort, type, and expiry. It is the one route declaring the new typed gate: 'exempt' option, because the rollout gate and the unknown-path catch-all answer byte-identical 404s and a gated /api/v2/meta could never resolve that ambiguity. Authentication still runs first, so the only fact disclosed is one about the caller's own credential. * feat(api): publish deployment lifecycle and workflow-MCP v2 surfaces Adds the four deployment-lifecycle operations v2 was missing, and the workflow-as-MCP publishing surface, both as adapters over application use cases that already existed. Deployment lifecycle: - PATCH /api/v2/workflows/{id}/versions/{version} relabels a version. Deliberately not the internal route's body-shape dispatch between "rename" and "promote to live". - POST .../versions/{version}/activate promotes a version. Same use case as rollback under a different transition, on its own path because the two mean opposite things to a caller. - POST .../versions/{version}/revert overwrites the draft. Accepts the literal `active` alongside a version number. - PATCH /api/v2/workflows/{id}/deployment toggles unauthenticated public execution. `workflows.public_api.update` widens from session-only to session plus personal API key: it is an admin-role change the same accountable human may make from a script. Workspace keys stay denied. Its EE refusal now carries PUBLIC_SHARING_NOT_ALLOWED instead of a bare forbidden. Workflow MCP servers: - /api/v2/workflow-mcp-servers list, create, update, delete, plus publish and unpublish of a workflow as a tool. Named apart from /api/v2/mcp-servers, which registers the external servers Sim calls. - The six mcp_servers.workflow_deployments operations widen from ['delegated'] to admit sessions and personal API keys; roles and the workspace-key denial are unchanged. - The server list gains keyset pagination, matching its external sibling, since nothing caps how many a workspace publishes. - Server, tool, and workflow reads move out of the use case into lib/mcp/queries. Route ratchet 1150 -> 1160; OpenAPI operations 161 -> 171. * feat(api): extract chat deployments and publish the v2 surface Chat deployment was a shipped module with no public API and two authorization systems: `lib/workflows/application/chat-deployments.ts` had deploy/undeploy extracted, but only Copilot used them — the REST routes reimplemented workflow authorization inline, and `PATCH /api/chat/manage/[id]` additionally owned password encryption, the auth-type field-clearing matrix, identifier uniqueness, the redeploy-gating protocol with two 409s, a raw db.update, and a manual recordAudit. New `lib/chat-deployments` domain: - `chat_deployments.list/read/update/delete`, keyed on the deployment whose workspace is derived by joining its workflow. Creation stays `workflows.chat.deploy`, which is keyed on the workflow. - The PATCH extraction, including the field-clearing matrix and the asynchronous-cutover invariant the route had hand-mirrored from `performChatDeploy`. - One `buildChatDeploymentUrl`, replacing three constructions that had already drifted onto two different host helpers. There is no chat subdomain, so nothing publishes a host. - Repository reads moved out of the use cases into `lib/chat-deployments/queries`. Internal routes are now adapters over those use cases. `GET /api/chat` is deliberately not migrated: it scopes by `chat.userId` while every other chat operation authorizes by workspace admin, and reconciling the two is a product decision. `PATCH` keeps its 400 for an identifier collision through a typed `ChatIdentifierInUseError`; v2 reports the 409 the condition actually is. v2 surface at `/api/v2/chat-deployments`: list, create, read, update, delete. Workspace-scoped, keyset-paged, and a stored password is never readable — reads carry `hasPassword` only, and the session-only reveal endpoint deliberately has no v2 counterpart. Also: an email- or SSO-gated chat with an empty allow-list is now refused in the use case rather than only at the internal boundary, since it is unenterable; and the doc comment on `processHostedKeyCost` claiming a `usageLog` write is corrected — no such write exists. Route ratchet 1160 -> 1165; OpenAPI operations 171 -> 176. * fix(api): close three review findings, two of them caller-reachable - Run output files are filtered to keys under the run's own execution prefix. The recording they came from is not a trustworthy key source: the start block copies every caller-supplied input field verbatim into its output and `collectUserFilesById` accepts anything carrying the `UserFile` shape, so a caller could name any storage key and have the download and base64 paths — neither of which authorizes per file — serve it back. - `getBlock` reads own keys only. `BLOCK_REGISTRY` is an object literal, so `constructor`, `toString` and friends returned inherited functions that every consumer then treated as a block, turning a path segment into a 500. `getToolMetadata` already guarded this way. - A folder-scoped log page no longer unions in every job run in the workspace. The guard read `filters.folderIds`, which the public surface never sets — it carries the folder filter in `folderScope` — so the page contradicted the contract's promise that job runs are dropped whenever a filter they cannot answer is set. Also: the log cursor stamps `includeJobRuns` only when it is on, so its `.default(false)` no longer puts a constant in every fingerprint and rejects cursors minted before it existed; and the `folderName` subquery is scoped to the workspace and to workflow folders instead of scanning the whole `folder` table. * fix(api): close two more review findings, one an authorization bypass - `workflows.operations.apply` no longer admits a workspace API key. The use case authorizes against three per-user policies — the EE permission config, block visibility, and credential reachability — and all three take a human subject. An actorless key has none, and both substitutes fail open: attributing to the workspace billing owner evaluates the batch as the least-restricted account in the workspace, and passing no user makes `getUserPermissionConfig` return `null`, which every caller reads as unrestricted. Either way a workspace constrained by an allowlist was edited as though it were not. Personal keys keep the capability, so headless editing is unaffected for a credential that names a human. - `GET /workflows/{id}/state` reads its variables through `parseWorkflowVariables`, and the stored variable response schema drops the two assertions the column cannot honour. The column has carried a JSON string and a legacy array as well as the current record, the realtime `variable.add` op types `type` as `z.any()`, and the parser writes `name` through verbatim — so the input bounds on the read turned a stored workflow into a 500 on the endpoint that opens it. The write schema keeps them, which is where they can still be honoured. - `GET /workflows?scope=archived` projects folder paths tolerantly. Archiving a folder cascades onto the workflows inside it but leaves their `folderId` dangling — which is why restore has to null it — so the strict projector threw a bare `Error` and took the whole page down with no cursor able to step past the row. * fix(files): bind Start-block file keys to the executing workspace The Start block derived a file's storage key by parsing the caller's own `url`, which `isInternalFileUrl` matches on any host and `extractStorageKey` returns verbatim — so a request body could name any tenant's bytes. The key is now accepted only when its own layout names the workspace the execution runs in, and every file is dropped when the execution carries no workspace. Also bounds `includeFileBase64` with an aggregate response ceiling and a worker pool instead of an unbounded `Promise.all`, scopes the bulk download's authorization resource to the workspace when folder paths are requested, makes the folder-restore selector mutually exclusive at the type level, and names the bound in the `maxBytes` validation message. * fix(api): close v2 log review findings Cursor scope: `scope` on the workflow and table lists carries `.default('active')`, so it entered every fingerprint as a constant and refused every cursor minted before the param existed — with the "cursor does not match the requested filters" 400, which is actively misleading for a caller that changed nothing. Both now stamp the default as absent, so only a caller who asked for `archived` gets a new sequence. Dashboard stats: `maxWorkflows` capped the response, not the allocation. Segment series are now densified after the cut instead of before, so returning 200 series no longer materializes one `segmentCount`-length array per workflow in the window. The aggregate still sums every workflow, now from the sparse per-workflow maps. Cost keyset: `cost_total` is an unconstrained `numeric`, so its anchor travelled through `Number()` and was compared back at full precision — rows differing beyond float64 collapsed onto one anchor. Adds `decimalKey`, which carries the digit string and binds it `::numeric`. Run detail: `cost_total` is a backfilled projection, so a run predating the backfill reported `cost: null` even with a real ledger, making `items` unreachable for exactly the runs the ledger explains. Falls back to the ledger total. Also caps the log folder-path index reads at MAX_FOLDERS_PER_WORKSPACE like every other reader, publishing the folder-tree 413 on the four log operations; reverts a dead `status` widening in `v2CommaListSchema`; drops an unread `executionData` select; corrects the segment-count and searchLogs prose; and replaces the sort-cursor SQL-text assertions with a two-page walk over a fixture with a null block. * fix(api): close knowledge v2 review findings - widen knowledge.list_archived to the delete/restore policy so a workspace API key can discover what it may restore - escape LIKE wildcards on the now-public chunk search - derive tag slot capacity from TAG_SLOT_CONFIG per field type - type updateKnowledgeBase's chunkingConfig as ChunkingConfig and project every declared field explicitly - attribute a restore to the calling surface instead of a literal 'api' - gate 'knowledge chunks batch-update' behind --yes, since it can delete - present the tag-cleanup action from the parsed request rather than faulting on the domain result after the delete committed - unbind asserted-scope workspaceId from the nested knowledge cursors, matching the table-row lists - add executed-SQL coverage for the chunk keyset * fix(catalog): close the catalog and registry-boundary review findings The module-graph ratchet treated an entry with no baseline row as informational, so the six catalog routes and the projection barrel were unratcheted while the summary still read "within their module-count baseline". An unbaselined entry now fails --check, the summary counts only what was actually compared, and the baseline is re-recorded. The Copilot block-metadata tool — the reason the shared projection exists, 6,756 modules down to 1,321 — was in no guarded subtree. It is now an entry source and a catalog boundary root. Catalog behaviour: - hostedApiKey is gated on the deployment, so a self-hosted install reports none instead of promising 127 tools' keys it will never supply - block detail resolves an unversioned base type to its newest version and projects through the viewer's visibility, so it can no longer 404 a block the list contains or name it differently - offset-cursor ordering compares code units rather than the process locale - projections copy every array they publish instead of handing out the registries' own - an options function returning a thenable throws rather than silently widening the providers-store substitution across the event loop - a throwing block projection costs the Copilot tool one block, not all of them - the trigger-kind log returns to debug: chat/manual/api are entry-point kinds, not authoring defects Also sweeps the custom-block detail branch against its response schema, guards each projection module rather than the dead barrel over them, and drops a provably dead branch in processHostedKeyCost. * fix(workflows): close v2 workflow-authoring review findings - Read a blockless draft back as an empty graph. `PUT /state` of `{ blocks: {}, edges: [] }` — the contract's own published example — deletes every block row, and the loader answers `null` for a blockless workflow, so the following `GET /state` answered 404 while the list endpoint still showed the workflow. Existence is the workflow row's to decide; the null is now projected as an empty graph. - Rewrite the `readWorkflowGraph` authorization test so it can fail. It called `authorize?.()` and asserted only a negative, so deleting `authorize` or replacing it with a no-op both passed — the invariant the head-safe `HEAD` path depends on. - Route `setWorkflowBlockEnabled` through `replaceWorkflowNormalizedState`, the same door the other two graph writes use, instead of writing the normalized tables itself without state preparation or custom-tool extraction. - Count applied operations directly. Enablement refusals landed in the same skipped-item array and were subtracted from the operation count, which `Math.max(applied, 0)` then masked when it went negative. - Give a `disabled_ancestor` refusal its own member of the published skip enum instead of reporting it as `block_locked`. - Refuse an `atomic` batch whose credential or hosted API key would be stripped, and carry the dropped inputs in the 409 details. - Publish the whole lint report — `sources`, `sinks`, `orphanBlocks`, `emptyOutgoingPorts`, `invalidBranchPorts`, `invalidConnectionTargets`, `fieldIssues`, and the `kind` discriminator on unresolved references — rather than only free-text reference prose. - Stop reporting unresolved lint references as `inputValidationErrors`. `collectUnresolvedReferences` is read-only, so those values stay persisted; they were double-reported, and falsely as dropped inputs. - Replace two unfalsifiable negative-principal tests, which used a principal kind `Exclude`d from `PrincipalKind`, with a reachable one. - Nits: drop a stranded TSDoc block; assert the membership predicate in the selector-validator admin test; make the HEAD test assert a representation; assert the sanitized graph is what `replaceWorkflowState` writes; add a route test rejecting `baseGraph` in a v2 body; carry `principalAuditSource` on restore/duplicate/moveBulk audit; unify the two `base64MaxBytes` ceilings on `MAX_INLINE_MATERIALIZATION_BYTES`. * fix(api): close seven v2 review findings, one an authorization bypass Raise mcp_servers.workflow_deployments.update_server to admin: its body carries isPublic, and a public server executes with no Sim credential, so a write member could remove authentication from every workflow it publishes. create_server already grants the same visibility at admin. Pin the widened operations in registry tests — the six workflow-MCP ones, the four workflow widenings, and files.extract_archive. Reject secret fields on a credential that has no rotatable secret instead of dropping them behind a 200, classify a non-transient provider 4xx as caller error rather than a retryable outage, and reconcile a provider outage to 503 with Retry-After on all three surfaces. Give /v2/meta a declarative principal policy through a new defineOperation factory, carry the key expiry on the auth context instead of reading the api_key table from the application layer, and make the impossible principal branch an invariant error rather than a codeless 403. Enforce the rollout-gate exemption at definition time, against the one contract path it is reserved for, and remove the gate parameter from the exported admission helper so the builder is its only door. * fix(tables): bound the run-state sidecar, and close six v2 review findings Enforces the 2 MiB run-state ceiling INSIDE the sidecar drain rather than over its materialized result, refuses the unbounded query form paired with it, and normalizes the two stored blobs the v2 surface publishes from bare `as` casts. - The run-state budget now travels into `loadExecutionsByRow`, which drains row ids in bounded chunks and refuses before fetching the next one. The post-hoc `requireBoundedRunState` walk is gone: it measured a spike that had already happened, and re-serialized every entry to do it. - Both row reads that accept `includeRunState` cap the page at `V2_MAX_RUN_STATE_ROW_LIMIT`, and `POST /tables/{id}/query` additionally refuses the flag paired with `limit: 0`. - `runState.status` and the enrichment cascade blob are projected onto the published shape before presentation; both were caller-reachable 500s on a well-formed read. - `sim tables bulk-delete` now gates behind `--yes`, and the CLI sweep that should have caught it covers destructive non-DELETE forms. - `POST /tables/{id}/restore` is idempotent (200, no audit) like its knowledge sibling, and bulk folder selection deduplicates after resolution. - The batch-update backstop keeps the looser Copilot ceiling and says so in TSDoc: it is a backstop no surface reaches, because the contracts stop a v2 caller at 1000 and the Copilot tool stops itself at 5000. Each caller sees the bound that actually applies to it; neither surface's cap moved. * fix(chat-deployments): close v2 chat review findings Fixes the chat-deployments slice of the v2 review, several of which are regressions the application-operation extraction introduced. - Stop a `500` on schemaless JSONB: the response now declares a stored shape without bounds and `toV2ChatDeployment` projects `customizations`, `outputConfigs`, and `allowedEmails` onto it. The request schemas keep `.strict()` and their bounds. - Restore the specific validation message on `POST /api/chat` and `PATCH /api/chat/manage/[id]`, and the deleted test that pinned it. - Restore `chat_deployments.read` to workspace `admin`; the detail read serves the visitor gate. - Narrow the list projection so `chat_deployments.list` can stay a `read` operation reachable by a workspace API key: `allowedEmails`, `hasPassword`, and `customizations` are gone from the list entry and available only from the admin-gated detail read. Serialized field by field so a field added to the detail shape cannot reach the list by default. - Classify create-path failures: `performChatDeploy` carries an `errorCode`, so an in-flight deployment is a `409` and an invariant failure a `500` instead of every refusal being a `400`. - Delete the callerless `GET /api/chat`, which served the encrypted password column with no response contract. - Propagate undeploy infrastructure failures instead of concealing them as `404`, and return `ChatDeploymentView` from both delete paths. - Name `CHAT_AUTH_MODE_NOT_PERMITTED` on the create path. - Guard `getBaseUrl` inside `buildChatDeploymentUrl`, which otherwise throws on a self-host with no `NEXT_PUBLIC_APP_URL`. - Correct the published allow-list claim: a replacement `allowedEmails` is applied after the auth-type clear, so it does survive. - Assert `workspaceId` on the v2 detail routes and reconcile the concealment TSDoc with what the error policy actually renders. - Move `resolveActiveWorkspaceApplicationContext` to the workspaces domain so chat-deployments no longer imports workflow application code. * test(credentials): pin the reconciled provider-outage status The internal route alone answered 502 where the v2 surface, the shared status helper and `PROVIDER_OUTAGE_CODES`' own TSDoc all say 503. The test pinned the divergence; it now pins the reconciliation, including the `Retry-After` a 503 carries. Corrects a stale comment that still named 502 as the value callers see. * fix(executor): restore cloud-storage Start files, dropped by the key rule The ownership check accepted a key only when it could be parsed out of an internal `/api/files/serve/...` URL. But the server-side uploader for run inputs returns a *presigned cloud* URL whenever object storage is configured, whose path is the bucket key — so every chat-deployment attachment, API `files[]` payload and generic-webhook file field resolved no key, and because normalization is all-or-nothing the entire `files` input was dropped with no error. It passed locally and under vitest only because the uploader falls back to an internal URL when no object storage is configured, which is exactly why no test caught it. The test is ownership, not provenance: a key is accepted when its own layout names the executing workspace, whether it arrives directly or is parsed out of the URL. Neither field has to be trusted, since both are caller-authored and both are held to the same check. A payload whose key and URL disagree is refused rather than resolved in the caller's favour — a genuine uploader writes the two consistently, so only a forged pairing is turned away. `context` is now derived from the accepted key rather than read from the payload or the URL's `?context=`, so an owned key can no longer be labelled with a bucket its bytes do not live in — the hardening the previous comment claimed but did not perform. * fix(api): close four defects the fix pass introduced - `resolveLatest` built a `RegExp` from the caller's block id and read the registry with a bare lookup, so the catalog detail route — moved onto it by the version-alias fix — routed around the `ownBlock` guard added for exactly this. `GET /api/v2/blocks/%5B` was a `SyntaxError` 500 and `.../constructor` an inherited function. Matched by string comparison now, the way `tools/tool-ids.ts` resolves the same convention, and read through `ownBlock`. - The run-state byte budget was applied inside `queryRows` rather than at the callers that publish it, so the first-party table grid — which reads run state at five times the row limit and publishes no ceiling — turned a large page into a hard failure, with an error naming a parameter it does not expose. The budget is now an explicit option the public reads pass and internal callers omit. - Three graph-write CLI commands shipped ungated because the sweep meant to catch them matched only the names already enumerated, so it could never fail. It now forces every non-`GET` operation into a destructive or non-destructive list, and the three carry confirmations. - Unbinding `workspaceId` from the knowledge-documents cursor was right on the merits and wrong in effect: the value is constant per sequence, so removing it changed the fingerprint and refused every cursor already in flight. Restored there; the chunks list is new in the same change and keeps the cleaner reading. * fix(api): resolve a detail read to a version the viewer can see `getLatestBlockForViewer` took the newest version and then hid it, which inverted the contradiction it was written to close: `slack_v2` and `table_v2` are preview-gated while their v1 deliberately stays in the toolbar, so an unrevealed viewer got a `404` on a detail read for a type `GET /api/v2/blocks` was listing in the same breath. It now walks versions newest-first and answers with the first one visible to that viewer. Also: - The chat password guard ran after `performFullDeploy`, so a request that could never succeed burned a real workflow deployment version and then answered 400. Its two sibling gate guards already refuse ahead of the deploy; this one now does too. - The Copilot sub-block serializer published the registry's own `options` and `dependsOn` arrays by reference. Pre-existing, but the catalog projection this parallels copies every array it publishes precisely because they are process-global and shared by every request. * fix(api): restore the locked read-modify-write and the password validator Two findings verified as real regressions against staging, out of ten checked — the rest were pre-existing, latent, or false. `setWorkflowBlockEnabled` read the graph outside the row lock and wrote it back inside a later transaction. The editor's own save takes that same lock, so an autosave committing in the window was silently discarded: this operation writes a whole graph, not a delta. The persistence primitive now accepts a reader that runs after the lock is taken, and the toggle re-reads and re-decides there. Its lock predicate is also scoped to the workspace and to a live row again, so a workflow archived mid-flight is refused rather than written. The v2 chat-deployment contracts inlined their own password rule twice instead of using `chatDeploymentPasswordSchema`, losing the refusal of a whitespace-only password — which the internal contract rejects precisely because it strands the deployment behind a password the visitor form will not submit. Both sites use the canonical validator now. * fix(api): one folder projection, one dynamic-provider list, honest 413s - `toV2Folder` existed twice, and the second copy had been written without the name/path invariant — so a row the list read refuses loudly would have been served with a mismatched pair by the restore read. One definition, guard included. - The catalog projection restated `DYNAMIC_MODEL_PROVIDERS` and had drifted by one member. Derived from the canonical list instead. - The tables reads documented a `413` for run state that they cannot emit — the budget became opt-in, and the row limit is the bound now — so the claim is removed rather than declared. The workflow run read has the opposite problem: it genuinely emits one, on a single file *or* the run's inlined total, and declared neither. Now declared, and the sentence covers both. - Reclassifies the operations staging added into the destructive sweep, so the triage stays exhaustive. * fix(v2): classify storage and uniqueness failures, drop permanent file delete - remove the permanent file-delete endpoint; the platform offers no such action in the UI, and its manager wrote outside a transaction with no storage accounting - extract a generated document's text from its compiled artifact rather than its generation source, matching the download path; a `.pdf` source was a 500 and a `.docx` source returned generator JavaScript as clean content - report a run file whose object retention has already swept as 404 rather than 500, on both the inline base64 read and the download stream - report a knowledge tag that loses at a unique index as 409, naming whether the slot or the display name is taken - gate `workflows versions revert` behind a CLI confirm; it overwrites the draft graph and was classified non-destructive * feat(v2): report lint from both graph writes and add dry-run previews - `PUT /workflows/{id}/state` now returns the same `lint` report as `POST /operations`; an agent authoring a graph from scratch needs the findings at least as much as one editing incrementally - extract the report into one shared builder so the two writes cannot drift, and one shared presenter so the wire shape is identical - skip the credential/tool reference pass when the caller has no human subject, rather than resolving it against the workspace billing owner: that would misreport what the workflow can reach and disclose another person's grants. `lint.notes` says when it was skipped - add `?dryRun=true` to both graph writes: validates and lints, persists nothing, records no audit, notifies nobody. A query param, not a body field, since the body of a PUT is the resource itself - CLI: a dry run no longer demands `--yes`; requiring confirmation to preview a change teaches callers to pass `--yes` reflexively - CLI: name the graph commands for their verbs — `workflows state get`, `workflows state replace`, `workflows operations apply` — instead of the derived `state list` / `state update` / `operations create` - document when to use `rollback` vs `versions/{version}/activate` on both * chore(docs): sync generated docs manifest for the new CLI pages * feat(v2): add the missing workflow-MCP reads and align bulk naming - add `GET /workflow-mcp-servers/{serverId}` and `GET /workflow-mcp-servers/{serverId}/tools`. The resource could be PATCHed and DELETEd but never read, and its tools could be published and unpublished but never listed — the server list reports tool names only, so nothing published the `workflowId` that addresses a tool for deletion. Both mirror `mcp-servers` beside them, and carry that family's workspace-API-key denial rather than the wider `mcp_servers.read` policy - rename `POST /tables/bulk-move` to `POST /tables/move`, so tables matches the shipped `files` resource exactly (`move` + `bulk-delete`) - name the CLI commands for their operations instead of the derived `... create`: `tables move`, `workflows move`, `tables bulk-delete`, and `tables rows update-each` for the per-row batch, which sits beside the existing filter-based `tables rows batch-update` `POST /tables/{id}/rows/batch-update` keeps its name: a distinct payload per resource is precisely AIP-234 BatchUpdate, and `bulk-` would have collided one word away from the filter form. * fix(v2): correct documented statuses and a caller-reachable 500 - duplicating a workflow into a locked destination folder answered 500: `FolderLockedError` is a plain Error carrying `status = 423`, which the v2 error policy does not classify. Converted to OrchestrationError('locked') at the application boundary, matching the bulk-move path - restore workflow promised a 413 for an oversized folder tree that its response list never published; the cap is real, so the status now is too - move workflows and apply variables documented 409/423 they cannot emit: every per-item lock and conflict is reported in `failed`, not thrown - bulk download and delete knowledge tag can both 409 and did not say so; cleanup tag definitions cannot and did say so - apply workflow operations denies workspace API keys but never documented it - the dry-run responses are not byte-identical to a committed write: `needsRedeployment` describes the pre-write state and persistence warnings cannot appear. Reworded rather than overclaimed - read file text and get file upload are head-safe, so the "HEAD skips the effect" sentence did not apply to them * fix(v2): guard tag field-type changes and publish the 415 every body route can return - `PATCH /knowledge/{id}/tags/{tagId}` accepted a `fieldType` incompatible with the slot the tag already occupies. Slots are enumerated per field type, so a text tag could be relabelled `number` and every later read would interpret its values as the wrong type. Create checked this; update now runs the same two checks - derive `415` from the contract the way `413` already is: the JSON builder answers UNSUPPORTED_MEDIA_TYPE for any body under a content type it cannot read, so all 100 body routes could return a status none of them published - give version activation its own result component instead of publishing it as `RollbackResult`; the shipped rollback keeps that name - correct descriptions that promised behaviour the code does not have: a `processingStatus` field never returned, a `gmail_send` resolution example that short-circuits, bucket widths that overflow the window, a bulk tag save that relocates rather than overwrites, and per-server tool names actually gathered under a page-wide budget - drop 409 from three knowledge and upload reads that cannot emit it * docs(v2): correct the upload transfer contract and 16 other published claims The upload transfer step was documented as Sim's own data plane on every deployment: "success is 204" and "a failure is the v2 error envelope". That holds only when Sim stores objects itself. With object storage configured the URL is the provider's presigned URL, so S3 and GCS answer 200 and Azure 201, and a failure is the provider's XML — a client written to the old text reads a successful cloud upload as a failure. Also states that part ETags do not need retaining: completion takes no body because Sim lists the parts from the provider itself. Other corrections, all to shipped descriptions rather than behaviour: - DELETE table and bulk-delete files archive rather than erase, and neither said so; bulk delete also cannot emit the 409 it declared - complete knowledge upload published a 402 only the create leg can raise - billing status conceals a foreign workspace id as 404, not the 403 its TSDoc and description both claimed - audit entries null a folder's resourceId and strip folder ids from metadata at every level; neither redaction was documented - details=full adds the workflow summary to workflow runs only, never to job runs; GET /logs folderPaths covers a subtree like its two siblings; getLog now carries the retention sentence - list secrets returns description too, and the logs and resources documents described only part of what they serve * improvement(api): consolidate the v2 surface and close seven defects Endpoint consolidation: - Fold POST /logs/query into GET /logs; add sortBy/sortOrder, cap the comma lists, and move the list onto the shared keyset codec - Fold GET /knowledge/archived into GET /knowledge?scope=archived, matching files, tables, and workflows - Re-home chat deployments as a singleton under the workflow they belong to; keep the workspace-scoped discovery list - Move the tag-definition writes off the document path onto /knowledge/{id}/tags, where they already acted - Nest the table export and dispatch reads under their parent table Defects: - Publish isPublicApi on the deployment read; it was write-only, so a workflow could be opened to unauthenticated execution unauditably - Stop publishing raw storage keys and an unusable URL in log files - Classify a chat-identifier unique violation as 409 rather than 500 - Fall back to the root path instead of throwing when a knowledge base's folder is archived - Cap bulk-download at the ceiling it actually enforces - Normalize variables through one helper on both graph write paths - Fix a folder-name log filter that matched workflow names Naming and gaps: - Rename /files/{id}/extract to /unarchive, /rows/find to /rows/search, /rows/batch-update to /rows/bulk-update, /columns/run to /dispatches - Type the last six generic [id] path segments - Add table folder restore and id-addressed dispatch cancel * chore(audits): record the v2 catalog routes in the boundary baseline * fix(api): accept a null chat password and correct three published claims - performChatDeploy validated `password: null` as a password, so the replace-shaped chat PUT answered 400 for every mode that owns no password — public (the default), email, and sso. The declared payload type has always allowed null, and the stored value is cleared by authType regardless, so null needs no validation of its own. The route test could not catch it: it mocks the orchestration module and pinned the exact null the real guard refused. - Redirect the two docs slugs this branch retired that were genuinely published: findTableRows and runTableColumns. - The table folder restore described an idempotent no-op for an already active folder; it answers 404. Say so, and say where the path comes from, since the tables folder list cannot yet report archived folders. - Name the customizations exception to the chat PUT's replace semantics. - A cursor-binding case used status=error, which is a level and not a status, so it failed contract validation and never reached the cursor check. Use an accepted value and pin the reason, not just the status. * chore(cli): classify the new v2 chat operation as non-destructive * feat(cli): expose canonical resource URLs * fix(api): close three caller-reachable failures found by the final probe - GET /files/{fileId}/text called parseBuffer unguarded, and parseBuffer signals every failure as a bare Error that no v2 policy classifies. A zero-byte upload or a mislabelled archive was an unhandled 500. Empty bytes now answer empty text — a zero-length file has no text — and unparseable bytes answer 409, matching the rendered-artifact resolver. - GET /workflows/{workflowId}/state asserted write-side bounds over stored data. workflow_blocks.name and .type are bare text() and the realtime rename op accepts z.string(), so a block renamed past 255 characters made the workflow unreadable, and unrepairable, over v2. The read shape now takes the same input/stored split the variable schema already had. Stored subflow conditions are coerced in the loader beside the existing numeric guards. - GET /knowledge stamped scope into the cursor fingerprint unconditionally. scope defaults to active and is new on that list, so every cursor the deployed build handed out would have been refused with a message saying the caller changed a filter they never sent. Its siblings already carry the guard and the comment. Also: POST /workflow-mcp-servers answers 201 like every other v2 create; GET /logs/stats reuses the log list's entry ceilings; the execute and resume routes install the media-type-aware 415 they publish; and a rationale citing an endpoint that never reached the wire is corrected. * fix(tables): carry the dispatch terminal timestamps through the stale sweep Staging's abandoned-dispatch recovery builds its own `DispatchRow`, and this branch had added `completedAt`/`cancelledAt` to that shape for the id-addressed dispatch read and cancel. The merge was textually clean and left the new mapping short two fields. * fix(workflows): deny workspace API keys on the graph replace `PUT /workflows/{workflowId}/state` stores blocks and their tool wiring wholesale, but the policies deciding which of those a member may add — the EE permission config and block visibility — take a human subject. A workspace API key has none, and both substitutes fail open: the billing owner is a different, typically less-constrained person, and passing no user makes the permission lookup return null, which every caller reads as unrestricted. That made the replace a second graph-write door storing what its sibling `POST /workflows/{workflowId}/operations` refuses, which denies workspace keys for exactly this reason. Both doors now agree. Personal keys keep the capability, so headless authoring is unaffected for a credential that names a human. * chore(api): drop the enrichment catalog endpoint GET /api/v2/enrichments listed the code-defined table enrichments. The per-row enrichment run detail stays; only the catalog read goes. Removes the route, contract, response and query schemas, the semantic operation, the use case, the projection module, its registry-boundary entry, and the CLI command. The "not found" and "blank search" cases it covered are repointed at the connector-type sibling so the shared behaviour stays tested rather than deleted with it. * improvement(api): make the workflow operations endpoint self-describing Two things stood between this endpoint and a caller who has only the published spec. The accepted `params` keys existed only in the edit engine's source. The spec said "the accepted keys depend on the target block type", so a caller reading it could create a nameless empty block and nothing more — while the Copilot tool catalog, over the same engine, has always spelled out the envelope. That guidance now lives in the contract, shared by the add, edit, and insert_into_subflow parameter schemas so the two surfaces cannot describe one engine differently: `inputs` keyed by sub-block id, `retry`/`triggerMode`/`advancedMode` beside it rather than inside it, `connections` keyed by source handle, and `removeEdges` for dropping one edge without restating the rest. A `block_id` that is not already a UUID is replaced with a minted one, and the mapping was computed and then dropped. A caller could not reference the block it had just created except by re-reading the graph and matching on name. The engine now returns it and the response publishes it as `mintedBlockIds`, with the in-batch versus cross-request rule stated in the operation description. * fix(api): close the pre-merge scan's blocking findings Docs and public wire, all of it permanent surface once released. - Two published tag groups, Catalog and Meta, had no sidebar entry in any locale, so six operations shipped unbrowsable. Added to all six, and a test now fails when a published tag has no entry. - deleteWorkflowChatDeployment pointed callers at DELETE on /deployment; the undeploy verb is on /deploy. - PUT /state still promised workspace API keys a degraded lint pass after the operation began rejecting them outright. It also lived in a single-quoted string, so the shared clause would not have interpolated. - POST /tables/move took targetFolderPath as nullable-but-required, which rendered the CLI flag as a required `<json|@file>`: `--to /Archive` failed to parse and omitting it failed outright, while the docs said "omit for root". Now optional and a plain string, matching POST /files/move; the route supplies the null the use case wants. - Table dispatch status and row run state published `cancelled` beside imports, exports, and job state publishing `canceled`, and the note explaining the split was wrong about its own sibling. Both new schemas now publish `canceled`; the stored column is unchanged and mapped at the presenter. The shipped `cancelled` count field is left alone. - applyWorkflowVariables can answer 423 and both workflow-MCP deletes can answer 409; none declared it. Hand-assembled error lists replaced with the shared sets. - /files/{fileId}/unarchive became /unzip. `extract` reads as "extract text" and `unarchive` reads as the inverse of restore on a resource where archived means soft-deleted; unzip collides with neither and is what the implementation calls itself. * improvement(cli,docs): finish the naming and clear the enrichment leftovers - Four single-record GETs derived to `list` while returning one thing: meta, a workflow's chat deployment, log stats, and file text. Renamed to `meta status`, `workflows chat status`, `logs stats`, `files read`, matching the `workflows deployment status` correction already in this branch. None of the old spellings shipped. - Dropped five `renamedFrom` aliases pointing at spellings that never existed, each of which built a hidden command and a permanent deprecation warning for argv nobody could have typed. `tables rows find` keeps its alias — that one really shipped. - Removed the enrichment catalog from three prose sites left behind when the endpoint went: the resources spec description, its Catalog tag, and the contract and pagination-test comments. - Documented the six new command groups in the CLI index table. That page is a guide page, so the docs staleness check cannot flag it. - Published the `customizations` exception to the chat replace semantics. It was in the route TSDoc and invisible to every caller reading the spec, which is where the claim "Replace, not merge" is made. * improvement(cli): use batch- for the tables bulk delete, matching its siblings The CLI renames a bulk form only when it would collide with its singular sibling, and uses AWS's `batch-` prefix when it does — `files batch-delete`, `tables rows batch-delete`, `knowledge chunks batch-update`. `tables delete` exists, so `tables bulk-delete` was that same rename reaching for the other word, and the only `bulk-` command on the surface. No `bulk-` CLI command has ever shipped, so this costs nothing now and would be permanent later. `files bulk-download` keeps its name: there is no `files download` to collide with, and it is one archive rather than N operations. Its config block now says why it exists at all, since the command is never built — the builder skips non-JSON response modes, but the contract sweeps still read the entry and require the folder-path field to be marked. * fix(workflows): allow operations on blockless drafts * feat(workflows): add manual and run-from-block execution * fix(cli): update workflow run description test * fix(tests): align fixtures with current contracts * fix(api): derive chat activity from workflow deployment --------- Co-authored-by: Theodore Li <theo@sim.ai>	2026-08-25 04:19:34 -04:00
Waleed	cae80c5f20	chore(lint): turn on the rules that would have caught the dead code (#7037) ... Three rules were off, so nothing enforced them. Measured, fixed the sites, and enabled them where the cost is bounded. `noAccumulatingSpread` — 2 violations, both real O(n²) reducers, both now `Object.fromEntries`. One duplicates a block's subBlocks on every block duplication; the other rebuilds a Record from every workspace env var. Enabled repo-wide. `noUnusedVariables` / `noUnusedFunctionParameters` — 633 repo-wide, but only 6 under `packages/`. Fixed those 6 and enabled both at error for `packages/**` via an override, which permanently covers 979 files. `apps/sim`'s remaining 627 are left deliberately: that is a sweep of its own, and a rule enabled with 627 outstanding warnings teaches people to ignore it. This is the class of rule whose absence let the dead code in #7019 accumulate — eleven unread loggers, a whole unimported file, write-only locals — none of which any gate could see. Two of the six were in `workflow-renderer`, where the fix is narrower than it looks. `isWorkflowRunning` is destructured-but-unread in both the block and subflow views, and the app passes it from `workflow-block.tsx` and `subflow-node.tsx`. Its TSDoc claimed it "holds every block's action swell open"; nothing reads it, so that behavior does not exist. Removing the prop breaks the callers and implementing it is a UX decision — there is adjacent logic deliberately not pinning the toolbar during a handoff. So only the unused binding goes, and the TSDoc now says what is true. Not enabled: `noDocumentCookie` (3 sites, and its fix is the CookieStore API, which is a browser-support call) and `useExhaustiveDependencies` (384 errors).	2026-08-24 11:10:19 -07:00
Vikhyath Mondreti	81ebb37563	fix(delivery): stable provider idempotency tokens, and correctness fixes around them (#6943) ... * fix(delivery): stable provider idempotency tokens, and correctness fixes around them Six independent fixes found while investigating a Slack transport failure. None of them are that failure; all of them are live. **Money writes could be delivered twice.** Square (8 tools), Brex (5) and Outlook Calendar minted their provider idempotency token with `generateId()` at request-build time. That is stable inside the transport retry loop — `prepareToolRequest` runs once above it — but a BLOCK-level retry re-enters the handler and mints a fresh one, defeating the provider's dedupe. A builder ticking "retry" on a Square block turned a committed write into a second card charge, and a Brex one into a second money transfer. Tokens now come from `deriveDeliveryKey`, a pure function of execution + block + tool + invocation, so every retry layer derives the same value. Stripe joins them: all ~50 tools previously sent no `Idempotency-Key` at all. The comment on `brex/create_transfer.ts` claimed a fresh key per transfer *prevents* duplicate money movement. That is inverted — fresh per *attempt* is what permits it — and is corrected here. **`invocationId` is required, not optional.** Deriving from `executionId` alone would be worse than the bug: five loop iterations paying five invoices would share one token, the provider would honour the first and silently drop four real payments, and it would look like five successes. Also: - `INTERNAL_API_BASE_URL` is now ignored on Trigger.dev workers. It names a route that resolves only inside the app container, and several modules run in both runtimes — `guardrails/mask-client.ts` says so in its own TSDoc — so setting it produced `PII redaction failed: Unable to connect` on every worker-side redaction. Mirrored into `packages/testing`'s urls mock, which reimplements the function and would otherwise have diverged. - `engines.bun` raised to >=1.3.14. Measured on 1.2.15, which the old floor permitted: a fully-delivered POST is silently replayed and the caller sees 200. - CloudWatch `put-metric-data` pinned to `maxAttempts: 1`. The AWS SDK default of 3 already replayed it, and `PutMetricData` aggregates rather than overwrites, so a duplicate silently corrupts the customer's metric series and alarm thresholds. - `webhookIdempotency` given a bounded in-progress lease. An untimed run held a SEVEN DAY lease while concurrent duplicates polled it once a second. Verified: `tsc --noEmit` clean, 30,100 tests pass. * fix(ci): restore staging files the branch split had reverted, and format Three problems, all from assembling this branch by checking paths out of a WIP branch built on an older `staging`. Anything `staging` changed since that base came back as a revert. - Root `package.json` had lost the `opentype.js` / `@types/opentype.js` dependencies `staging` added, which desynced `bun.lock` and failed `bun audit`. It also carried a `check:outbound-delivery` script belonging to other work. Every `package.json` is now taken from `staging` with only the `engines.bun` line re-applied. - `executor/utils/block-data.test.ts` — a 77-line file `staging` added — was deleted outright. Restored. - `executor/handlers/generic/generic-handler.test.ts` had lost a test `staging` added. Restored, with only the one `blockId` assertion re-applied. Also formats `keyed-invocation-identity.test.ts` and sorts imports in `internal-api-base-url.test.ts`, which is what `lint:check` failed on. * fix(providers): thread the model's tool-call id into keyed tool execution `prepareToolExecution` accepted an `invocationId` but no provider supplied one, so a keyed tool invoked through an agent always hit the incomplete-context fallback and minted a fresh token — leaving Stripe, Square, Brex and Outlook Calendar writes able to double-deliver under the hosted-key retry layer even though the block path was fixed. The id is now a positional parameter rather than another optional field on `request`, so a provider that cannot supply one fails to compile instead of silently falling through. 22 of the 27 call sites already had the OpenAI-shaped `toolCall` in scope; `tsc` identified the other five, of which Anthropic (`toolUse.id`) and Bedrock (`toolUse.toolUseId`) name it differently. Gemini is left deliberately unthreaded and documented: its function-call parts carry no model-supplied identifier — the streaming loop has to synthesize a local one — and a positional index would not survive the model re-emitting the call. A token that only looks stable is worse than the loud fallback, which names the missing fields. * fix(executor): keep execution order monotonic across a resume `executionOrder` is not carried in the pause snapshot, so a resumed run restarted the counter at 0 and a loop or parallel body executing on both sides of a pause could reuse a pre-pause value. That was cosmetic while the number only ordered logs. It stops being cosmetic once identity is derived from it: a `keyed` tool takes its provider idempotency token from this value, so two distinct writes would present the same token and the provider would silently drop the second. Suppressing a real payment is worse than the duplicate the token exists to prevent, because it looks like success. The counter is now seeded from the highest `executionOrder` among the restored block logs rather than from a new snapshot field, so snapshots written before this change are repaired on resume instead of needing a migration. * fix(providers): thread the tool-call id through the streaming loops too The previous commit only reached call sites written as a single-line three-argument call. The streaming loops are formatted across lines, so openai-compat (Groq, DeepSeek and everything else routing through it), Anthropic and Bedrock still omitted the id and kept falling back to a fresh token. Both Gemini paths now pass `part.functionCall?.id` — the RAW model id, not the `ensureToolCallId` value used for stream events. That helper allocates an execution-local id when Gemini supplies none, and it is freshly allocated per attempt: passing it would complete the keyed context, silencing the "could not derive" warning, while leaving the token just as unstable. Gemini frequently omits the id, in which case this is `undefined` and the loud fallback stands. All 27 call sites are now covered. * fix(providers): make the tool-call id argument required, not optional The TSDoc claimed a provider that cannot supply an id would fail to compile, but the parameter was declared `toolCallId?: string` — so a new call site could omit it entirely, typecheck, and silently take the unstable-token path the positional parameter exists to close. The comment promised a guarantee the type did not enforce. It is now `string | undefined`: required in position, nullable in value. A provider with no model-supplied id must pass `undefined` explicitly and take the loud fallback, rather than being able to forget the argument. All 27 existing call sites already pass it, so this is enforcement only. Verified by deleting the argument at one site: `tsc` rejects it.	2026-08-21 13:35:20 -07:00
Waleed	8f85ded35f	fix(workflows): pin a stored block retry policy when loading it (#6614) ... `workflow_blocks.retry` is a jsonb column written verbatim. Three of its writers never validate what they store: the realtime batch-add and replace-state ops take untyped block records, and the admin/superuser import routes persist externally-authored workflow JSON. `load.ts` then asserted the blob was already a `BlockRetryConfig` and handed it straight to the HTTP boundary, where `workflowBlockStateSchema` bounds `maxTries` to 2..5 and `waitBetweenTriesMs` to 0..5000. That schema is shared between the PUT `/state` body, where the bound is right, and the GET `/api/workflows/[id]` and `/state` responses, where it is fatal. The response `.parse` in the shared route builder throws a ZodError, which is not an `OrchestrationError`, so the error policy declines it and it falls through to a 500. One out-of-range or partial stored value therefore made a workflow permanently unopenable, with no in-product repair — every UI write path reads the workflow first. The feature already declares clamp-on-read as its contract: the commit that added it says bounds are clamped on read rather than rejected, the TSDoc on `resolveBlockRetryConfig` says the same, and `block-retry.test.ts` asserts it. Execution has always honoured that. Only the read boundary disagreed, so that is what this fixes: the loader now constructs a real `BlockRetryConfig` from the blob through `normalizeBlockRetryTries` / `normalizeBlockRetryWaitMs`, filling defaults for missing fields and carrying `enabled` across unchanged. `loadWorkflowFromNormalizedTablesRaw` is the single read choke point for both apps — `@sim/workflow-persistence` for the Next app and the realtime server's full-state emit — so one edit repairs every reader, including rows that are already out of range, and the row self-heals on the next save. It matches `clampParallelBatchSize` a few lines below, which already pins a stored subflow value on the same path. Alternatives rejected: - Validating on write. It leaves every existing bad row fatal forever, and it would have to be repeated across three realtime ops plus roughly a dozen `saveWorkflowToNormalizedTables` callers, none of which share a validation seam. - Bounding `BlockRetrySchema` in `@sim/realtime-protocol`. Its own TSDoc is correct that batch-add and replace-state bypass it, so this closes one writer and leaves the 500. - Relaxing the response contract. It stops the 500 but leaves the editor rendering a number execution will never run. A test now pins the write bound so that shortcut fails loudly. - `resolveBlockRetryConfig`. It returns null for a disabled policy, which would erase the numbers a builder configured every time state is read. Tests: six cases in `packages/workflow-persistence/src/load.test.ts` (four red before this change) covering out-of-range enabled, out-of-range disabled with `enabled` preserved, missing fields, a non-boolean flag, an untouched in-range policy, and NULL meaning "runs once"; plus a contract test that the write bound still rejects out-of-range input.	2026-08-12 01:42:11 -07:00
4 people	245ad1bd46	feat(workflows): new workflow block card, progress indicator, colors, dsl for natural language preview, retry configs (#6458) ... * improvement(workflow): refine canvas interactions and rendering * fix(workflow): keep outputs on the right, focus newly created blocks Connection anchors: an output now always leaves a card from the right. The cursor swell lets a drag start on any edge, but the left side is the input, so anchoring an outgoing edge there drew a line out of the input port and read as a second input. `normalizeCursorSourceHandleId` resolves every drag to the right anchor, `normalizePositionedSourceHandleId` collapses `source-left` alongside the legacy vertical anchors (so data from the API, an older client, or a stale save self-heals on load), and only the right-side source anchor is mounted. Drops in `onConnectEnd` are always source -> target. The branch that reversed the edge for a drag starting on an input could never run: the `target` handle is `isConnectableStart={false}` and the positioned side anchors are `isConnectable={false}`, so React Flow never reports an input as a drag origin. Removed it and its now-unused imports. A newly created block is centered once its node mounts and is measured, so a card added from a drag-release, the block menu, or the toolbar is never left off-screen or under the editor panel. The editor panel's block icon uses the same type accent as the card's badge instead of the block's legacy `bgColor`, which had left the panel on the old per-integration brand colours. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(workflow): floor header-only card height, adopt brand tag palette The Start card intermittently collapsed after load, squashing the action-menu tab so its icon row sat over the card. `.workflow-drag-handle` is the host the border renderer measures, and both it and the header row took their height from `blockHeight && blockHeight > 0`. `blockHeight` comes from the deterministic-dimensions pass and is already floored at MIN_PAINTED_HEIGHT (48), but it is absent on the first frames — and with no floor the host collapsed to its natural content height (25.5px for a header-only trigger, exactly the title's line box). The border builds its perimeter from `host.offsetHeight`, so that window painted a sub-floor card: too little straight edge remained on the vertical runs for the action-menu tab, which collapsed into the corner arcs. Whether you saw it depended purely on whether the dimension publish had landed, which is why it reproduced on one workflow and not another. Floor all three: the host, the header row (so `items-center` centres the title and type tag rather than pinning them to the top), and the border's own `offsetHeight` read. Also raise ACTION_MENU_CONTENT_READY_THRESHOLD to 0.9. At 0.8 the 24px icon row was revealed while the swell had only reached 22.4px of its 28px — shorter than the row it contains. Secondary to the above, but a real overflow window on its own. The test now pins the ratio rather than the constant. Tag palette moves to fixed brand values (hex, not derived oklch) with two inks — #F8F8F8 on dark fills, #1A1A1A on light. Tones are renamed to match what they render. `green` (2.55:1) and `orange` (3.15:1) sit under WCAG AA against their paired ink; both are deliberate brand decisions and are documented in the component. Deploy and Run take two new Button variants rather than className overrides, so `tertiary` stays green everywhere else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * improvement(workflow): polish workflow canvas interactions * fix(workflow): restyle loop drop target outline * fix(workflow): shorten human block catalog label * fix(workflow): canonicalize realtime edge handles * code cleanup * sizing fixes * improvement(blocks): sentencify every block * change tebse * improvement(workflow): land notes UI and execution progress, consolidate duplicates Ports the notes canvas editing and execution progress-indicator work, then removes the parallel paths it arrived with so each concern has one owner. Fixes found while consolidating: - Note height was measured while the card was expanded to NOTE_EXPANDED_WIDTH, where text re-wraps shorter, and published as the node's compact height. The collapse then animated to a height never measured at the compact width. - The edge pulse glow filter used the default objectBoundingBox units, so a straight horizontal edge — what an auto-laid-out chain produces — resolved the filter region to zero height and stopped the glow rendering entirely. - A subflow's inner Start pill still read isNodeSelected while its border read usesSelectedVisuals, so the two disagreed during execution. - The Run/Stop button's disabled prop gated only Run while its handler cancelled unconditionally, offering a Stop the cancel route answers with 403. Consolidated: - One note editor. The view's built-in textarea was unreachable in production (the app always injects the markdown editor) and was kept alive only by tests asserting against it; renderContentEditor is now required. - onBlur/onCancel collapse to onEndEditing — content persists per keystroke, so there was never a draft for a cancel path to discard. - DEFAULT_NOTE_COLOR, the note height bounds, the note content reader and the card width each had two or three definitions; each now has one. - Removed with zero consumers: graphite/graphiteSubtle button variants, data-subflow-selected, inputPlaceholderClassName, an effect that could never fire, and getNoteColorOption's unreachable fallbacks. Restores the role='status' announcement the progress rewrite dropped, and hardens isNoteColor against inherited Object keys. Co-Authored-By: Claude <noreply@anthropic.com> * improvement(workflow): reuse the platform markdown editor in notes Notes carried their own TipTap wiring — a second markdown editor that reimplemented, more thinly, what `RichMarkdownField` already does for the skill modal, skill fields and the deploy version description. It is now a ~20 line skin: the Note supplies its type scale and per-colour selection tint, and the field supplies the extension set, frontmatter held out-of-band, the round-trip safety gate and its raw-source fallback, and markdown paste. `RichMarkdownField` gains two additive props, both defaulting to today's behaviour so the file editor is untouched: `surface` ('field' | 'bare') and `proseClassName`. All three existing consumers pass an explicit `minHeight` and no `surface`, so they take the original path unchanged. Exiting the note editor moved to the card, because the editor's `/` and `@` menus consume Escape to close themselves and ProseMirror checks `editorProps` before plugin handlers — intercepting it inside the editor would have broken both menus. The card now honours Escape only when nothing already consumed it, which also let `onEndEditing` leave the injection contract. The note editor is lazy now, matching every other consumer: it was pulling TipTap and the full extension set into the canvas's initial chunk. Also: - One `areRunFromBlockDependenciesSatisfied`. The ActionBar, the canvas context menu and the run-from-block handler each carried a byte-identical copy, and the handler expressed the snapshot requirement differently, so the affordance and the action could disagree. Each copy also re-scanned `edges` once per incoming edge, on every ActionBar on the canvas. - Reduced motion is one `usePrefersReducedMotion` in @sim/emcn rather than a sixth ad-hoc `matchMedia`. The edge pulse now stops rendering instead of hiding: `motion-reduce:hidden` is `display: none`, which left four SMIL timelines running per edge. - The pulse glow bleed covers the canvas minimum zoom. The strokes are `non-scaling-stroke`, so the 6px tail spans 3/zoom user units — 30 at 0.1. Co-Authored-By: Claude <noreply@anthropic.com> * improvement(workflow): port canvas styling from workflow-updates Ports the 14 styling commits your colleague added since the last sync, leaving the ~68 staging PRs on that branch alone — those are platform/core work, not this. Cherry-picked individually rather than merged so each conflict was small enough to reason about. What came in: - Core block colors unify behind a two-level map: block type -> semantic role -> accent, replacing the flat per-type table. Adds `purple` and `content` tones to ChipTag, and a shared `WorkflowTypeIcon` that replaces the hand-rolled ChipTag + accent lookup at each discovery surface. - Native triggers take semantic colors; the deployments block moves to the shared Rocket icon and drops its now-unused `iconColor`. - Running-state polish: loader artwork and position, stop hover in dark mode, the loader blended into the execution swell, and tooltips suppressed for actions that are hidden mid-run. - The toolbar drag preview clones the rendered icon container instead of rebuilding a bgColor tile, so it matches what the canvas paints. - The sidebar shows route-derived workspace identity instead of a skeleton while the full record loads. Conflict resolutions worth knowing: - The running-loader artwork went through the shared `Loader` and back to the custom SVG on their branch; the second commit is the intent, so that is what landed — keeping our `role='status'` announcement layered on top. - Two commits carried the lucide-react -> in-house icon migration along with them. That migration is a staging change we have not taken, so our imports stayed on lucide: adopting it in two files would leave the icon set split across the app. - `getMappedWorkflowTypeAccent` referenced a constant their refactor removed. It had no consumers left once the search modal moved to `WorkflowTypeIcon`, and their branch deletes it too, so it is gone here. Co-Authored-By: Claude <noreply@anthropic.com> * feat(workflow): make the subflow Start swell a real connection source Dragging an edge into whitespace opens the add-block picker, but starting that drag from a loop/parallel Start pill did nothing: the pill's border swell was visual-only. Regular blocks and the container's own exit mint a draggable cursor handle from their swell; the pill rendered only its invisible 14px static strip, so grabbing the glowing affordance started no connection at all. Everything downstream already worked and was nearly unreachable: - the drop hit-test skips subflow containers, so a release inside the loop opens the picker there (z 2000, above containers) - handleToolbarDrop parents the new block into the container at the drop point - it already carries the exact boundary rule for this source: a container start handle only wires to a child of that container The pill now runs the same cursor-handle machinery as the container view, with one deliberate difference: its temporary handle carries the branch-cursor form of the start id. The plain cursor id normalizes by block type — for a container that is `loop-end-source`/`parallel-end-source`, the exit — so a swell drag from Start would have persisted as an edge leaving the container. The branch form passes `loop-start-source`/`parallel-start-source` through normalization verbatim on both the picker and direct-connect paths; a test pins that contract. Co-Authored-By: Claude <noreply@anthropic.com> * fix(notes): stop the field's prose classes recoloring bare-surface editing Opening a note for editing shifted the text and turned it black: the ProseMirror root unconditionally carried `rich-markdown-prose rich-markdown-field-prose`, which pin the field's own ink and type ramp — `--text-primary` at 15px/25px, then 14px/22px — overriding the card's `text-current` at 14px/20px the moment the editor mounted. `surface='bare'` means the host owns typography (the Note card mirrors its rendered view via `proseClassName`), so on that surface the root now carries no shared prose classes. The field surface is untouched. Edit mode inherits the note colour's ink — including the caret — and sits on the same metrics as the read view. Co-Authored-By: Claude <noreply@anthropic.com> * fix(workflow): give the in-flight connection line contrast inside containers The drag line was drawn but camouflaged: its default stroke was the resting-edge grey (#e0e0e0), which disappears against a loop body's opaque `--surface-3` fill (~1.1:1) — so dragging an edge inside any container, nested included, showed nothing. The z-order was never the problem; the connection line layer already sits above every node. The default token is now `--text-muted`, one value with contrast on every canvas surface, still lighter than the `selected` variant so the variant hierarchy holds. No per-surface special-casing. Resting edges inside containers share the same camouflage (`--workflow-edge` on `--surface-3`) — left alone deliberately: recoloring placed edges is a design decision, not a bug fix. Co-Authored-By: Claude <noreply@anthropic.com> * fix(notes): match edit mode to the read view, and land the caret where clicked Three defects, all from the read and edit views being built independently. 1. Blocks jumped up ~12px on entering edit mode. Streamdown wraps its output in a container carrying `space-y-4` plus first/last margin resets, which outrank the per-element margins in NOTE_COMPONENTS — so that wrapper, not those margins, is what the read view actually paints. The editor had no equivalent. The rhythm is now named (NOTE_MARKDOWN_FLOW), passed to Streamdown explicitly so a dependency upgrade cannot move the read view out from under the editor, and mirrored on the ProseMirror root. Tailwind's JIT only sees literal strings so the mirror cannot be composed from the constant; a test pins the two together instead, and fails if either side drifts. 2. The caret was barely visible: it inherited the note's 75%-opacity ink. The palette owns per-colour chrome, so it now names the caret alongside the selection tint. 3. The caret always landed at the document end. The read view sits under a full-bleed overlay that must swallow the click to enter editing, so the point never reached the editor and `autofocus: 'end'` was all that was left. The view now forwards that point and the field resolves it through `posAtCoords` on create — after the DOM is laid out, which `autofocus` cannot wait for. Keyboard activation carries no point and still lands at the end. `autoFocusAt` is additive on the shared field and defaults to null, so the file editor and the other three consumers are unchanged. Co-Authored-By: Claude <noreply@anthropic.com> * fix(workflow): stop edges rendering behind top-level subflows Containers are z-indexed by nesting depth, so a top-level subflow is 0. Edges derived their z from their parent container — `+1`, or 0 with no parent — so a root-level edge landed on exactly the same z as a root-level subflow. Equal z-index falls back to DOM order, and React Flow paints the nodes layer after the edges layer, so the container's opaque body won: any edge crossing a top-level loop or parallel was drawn behind it, in-flight or persisted. Edges now sit in their own band above the whole container scale and below cards, keeping both the deeper-container-wins ordering and the rule that a line always passes behind card chrome. This is why the edge became visible only once a block was dropped: the new block is selected, and an edge inside the container was already `containerZ + 1`, clear of the tie. The in-flight connection line is declared in the same scale rather than inheriting React Flow's stylesheet default of 1001, which is both below a selected container child and outside the scale this file owns. Its stroke moves to `--text-secondary`, the token the canvas already uses for an active edge — the previous `--workflow-edge` grey is ~1.1:1 against a subflow body. Co-Authored-By: Claude <noreply@anthropic.com> * feat(notes): paste and drop images through the workspace-file pipeline Traced the file editor's image path end to end and reused it verbatim rather than minting a note-specific source: insertImages -> useUploadWorkspaceFile -> POST /api/workspaces/{id}/files/presigned -> direct-to-S3 PUT -> workspace_files row -> the editor persists the workspace-scoped /api/workspaces/{id}/files/inline URL, which the serve route authorizes by workspace membership and the embedded-image-ref machinery already recognizes for share rewriting and referenced-by-doc tracking. The shared field gains an optional `uploadImage(file) -> {url, alt} | null` capability. With it, image paste/drop uploads sequentially and inserts each at the evolving position, mirroring the file editor's flow, with a bail if the editor unmounts mid-upload; without it, the existing swallow-guard on file drops is unchanged, so the skill modal, skill fields and version-description consumers behave exactly as before. The upload mutation owns its own toasts. The note host wires the capability with folderId null, so note images land in the workspace Files root — visible, manageable and deletable there like any other upload. The note read view renders images through its Streamdown components map with the card's own sizing. Co-Authored-By: Claude <noreply@anthropic.com> * improvement for notes, subflows * fix(uploads): surface the server's message when a multipart upload is refused A file over the 50MB direct-PUT threshold goes through multipart initiate, which is where the storage quota is enforced — but the client threw away the response body and reported `Failed to initiate multipart upload: Payload Too Large`. That is the string the upload mutation puts in its toast, and it names neither which limit was hit nor by how much, so the one place that answer surfaces didn't have it. It now prefers `errorBody.error` exactly as `getPresignedUploadInfo` already does on the single-PUT path, and passes the body through as the error's details. Control flow is unchanged: still throws, still `MULTIPART_ERROR`, and the cloud-storage-absent branch above still claims its 400 first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(notes): restore GFM in the note read view Streamdown's `remarkPlugins` prop REPLACES its default plugin list rather than extending it, and remark-gfm is one of those defaults. The note passed `[remarkBreaks]` — so the read view silently lost every GFM construct: task lists, tables, strikethrough and autolinks. The editor writes all of them (it has TaskList, TableKit and Strike), so a note round-tripped through editing came back as raw source the moment editing closed: `- [x] HELLO` rendered as a disc bullet followed by the literal text `[x] HELLO`. `NOTE_COMPONENTS` has carried table/thead/tbody/tr/th/td entries this whole time that could never fire. Restoring the plugin is only half of it: remark-gfm marks a checklist `contains-task-list` and emits a native checkbox, which under the note's generic `ul` styling renders a checkbox sitting behind a disc bullet — the same defect the editor had before the chrome/typography split. The read view now drops the marker and indent for a task list, lays the row out as a flex line, and styles the checkbox to match `.rich-markdown-nodes input[type="checkbox"]` declaration for declaration, tick clip-path included, so the two views agree either side of a click. `remark-gfm` is now a declared dependency of the renderer package rather than one borrowed transitively from streamdown. Five tests cover the GFM surface — checked/unchecked boxes, no literal `[x]`, the marker only dropped for checklists, tables, strikethrough — and four go red with the plugin removed. Checked the other three Streamdown call sites (Chat, the chat interface renderer, the changelog): none override `remarkPlugins`, so none were affected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(canvas): drop dead markers and share the tile-brightness maths Review pass over the branch against staging. Dead code removed: - `tileIconColorClass` in the renderer package — never called; only its `isLightTileColor` sibling is. - `data-connection-selector-search-frost`, `data-workflow-cursor-edge` and `data-workflow-cursor-source-side` — written on three elements, read by no stylesheet, selector or test. - `CHIP_TARGET_SELECTOR_TYPES`, `MAX_CHIPS` and `chipPriority` were exported from `canvas-rows.ts` but only used inside it. Consolidated the one real divergence: the renderer package carried a hand-copied mirror of the app's perceived-brightness maths, because it may not import app code. The copy had already drifted — it dropped the `white`/`black` keyword handling, so a block shipping `bgColor: 'white'` would render a white `currentColor` icon on a white tile on the canvas while every other surface drew it black. No block ships one today, which is exactly why nothing caught it. The function now lives in `@sim/utils/color` and both sides import it; only the 0.75 threshold stays local to each. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(canvas): share the z-scale, fix the preview's edge layering, and re-home strays The preview canvas carried its own z numbers and had the collision the editor canvas was fixed for: containers at nesting depth, top-level cards at an implicit 0, and edges at 0/5/10 by execution status — so a default edge tied with a top-level subflow and painted behind it, while a success edge painted over unselected cards. The scale now lives once, in `@sim/workflow-renderer/canvas-layers`, and both canvases read it. The preview keeps its status ordering, expressed inside the shared edge band rather than as a second set of magic numbers. Placement and duplication: - `perceivedBrightness` moved to `@sim/utils/color`, with its unit test, and its consumers import it directly. It had been re-exported through `lib/colors/brightness.ts`, and the renderer package kept a hand-copy. - `filterAcyclicEdges`/`wouldCreateCycle` were pass-through wrappers in the workflow store's utils over the real implementations in `@sim/workflow-types`. Deleted; the three consumers import the source. - `lib/ui/glass-surface.ts` was a one-constant, one-consumer app-wide module, and its consumer then aliased it a second time. Collapsed into the navbar shell. - `nested-subflow-node` was set on nested container nodes in both canvases with no stylesheet, selector or test behind it. `packages/workflow-renderer` now has its own vitest config, so the four mount tests for its components live with the components instead of in `apps/sim/lib/workflows/**`. That immediately earned its keep: `apps/sim` excludes test files from type-check, and once these were checked, tsc found three `SubflowNodeView` renders being handed a `renderContentEditor` prop it does not accept — a copy-paste from the note cases that had been silently ignored. Verified: type-check 23/23, 21,143 app tests + 49 renderer + 147 utils, biome clean, all 23 audits pass (`check:bare-icons` imported the moved helper and was repointed). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix field noun bug + notes * fix(notes): let a note service the canvas actions the panel editor cannot The panel editor clears any note put in front of it and renders nothing, but the block menu still routed Rename and Open Editor through it. Rename latched the editor's rename state onto the note — `handleStartRename` reads the store directly, so it saw the id the menu had just set — and nothing reset it when the clear ran. `handleSaveRename` writes to `renamingBlockIdRef`, so the header went on showing a rename field over whatever was selected next and saved that name to the note. Open Editor was a plain no-op that opened an empty pane. Rename now goes to the card, which expands and opens its own title — the same menu-to-card routing Add Image already used, so both events now live in one `lib/workflows/notes/canvas-requests.ts` and `add-image.ts` keeps only its markdown concern. Open Editor is hidden for notes. The panel editor also drops any rename whose block stops being the selected one. That is belt-and-braces for notes now, but it closes the same hole for ordinary blocks, where only the input's blur ended a rename and blur only fires if it held focus. A rename interrupted that way is now discarded rather than left pending. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(canvas): author sentences for Snowflake and Mintlify, repoint Instagram's Staging's two new integrations shipped without a `canvasPresentation`, so their 39 operations painted the field rows the rest of the canvas has stopped using. The Instagram break is the more interesting one: staging renamed the insight metrics subblock `metrics` -> `insightMetrics` while this branch was adding sentences that named `metrics`. Both hunks merged cleanly — the union check reports the file as an exact union — and the result was two clauses pointing at a field that no longer exists, which resolves to nothing with no error and no log. Only `check:canvas-sentences` sees it. Two Snowflake sentences say something the block does not do, so they anchor elsewhere: `taskName` filters `list_task_runs`/`get_task_run` rather than keying them, and `table` filters `introspect_schema` — blank means "every one", not "not filled in yet", and a core chip would have claimed otherwise. Coverage is back to 4727/4727 operations across 321/321 blocks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(canvas): keep the block-type tag naming its type after a rename The header tag dropped its label whenever the block's title already said the same word, so the same block read two ways depending on nothing the user did deliberately: a freshly dropped Wait showed a bare icon, and its second copy — auto-named "Wait 2" — showed "Wait". The tag looked like a badge that appeared on rename rather than a fixed part of the header. It now always names the type, which is what loop and parallel containers already do with their own tag, so every card on the canvas reads the same way. `blockName` was only ever read for that comparison, so the prop is gone rather than left behind for a future reader to wonder about. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(deploy): compare edge handles by port, not by spelling Two places answer "does this need redeploying?" and they load their sides differently. The client diffs the live store against `/api/workflows/[id]/deployed`; the server diffs the normalized tables against the version's raw jsonb. Only some of those paths run handles through `loadWorkflowFromNormalizedTables`, so a snapshot holding a side-anchored id (`source-right`) met a canonical one (`source`) on the other side and the set comparison read it as every edge being removed and re-added. Each answer therefore differed, and they arrive on separate query timelines: the button reads the client's, the modal badge reads the server's, so the state flipped between Live and "Update deployment" with whichever query landed last until both settled. `normalizeEdge` now canonicalizes both handles, so the comparison cannot tell two spellings of one port apart no matter how its inputs were loaded. The existing normalization in `materializeDeploymentState` stays — that path also feeds React Flow, which needs the handle it mounts to match. The preview's error port had the mirror problem: it rendered for every non-trigger block regardless of `errorEnabled`, so a card with no error row grew a red knob anyway. It now gates the way the editor canvas does, keeping the port mounted when an error edge already leaves it so React Flow cannot drop that edge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(deploy): stop counting the error flag twice in change detection `errorEnabled` has two homes. It persists inside the block's `data` jsonb — the realtime server `jsonb_set`s it there, and load mirrors it back onto the block as a field — so it reached the diff twice, and only some paths populate the copy. `setBlockErrorEnabled` writes the mirror alone, so right after toggling the port the live block said `errorEnabled: true` with `data.errorEnabled: false`, while the snapshot the deploy had just taken from the tables said true in both. The diff read the stale `data` and reported the workflow as changed the instant it finished deploying — then a state refetch rehydrated the block and it agreed again. That is the flip between Live and "Update deployment": the button and the modal read two different queries, so each landing swapped the answer. A block created in-session had the same shape from the other side, its `data` carrying no key at all against a persisted `false`. Excluded from `normalizeBlockData` alongside the other fields that are duplicated out of the block's own state. The block field is still compared on its own, with `!!`, so absent and `false` agree and turning the flag on is still a change. Fixing the store to write both homes was the other option and is not taken: nothing reads the in-memory `data.errorEnabled` (save and load both let the block field win), so it would add a second copy that only the diff could see — which is the shape of this bug, not its fix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(blocks): give the error-output flag a column instead of two homes `errorEnabled` had no column, so it persisted inside the block's `data` jsonb and was mirrored onto the block as a field on load. Every writer had to route its `data` through `withPersistedErrorEnabled` or silently drop the toggle, the realtime op `jsonb_set`, and change detection saw the same value twice — which is what made the deploy badge flip between Live and "Update deployment" after toggling the port. Its siblings — `enabled`, `horizontal_handles`, `advanced_mode`, `trigger_mode`, `locked` — are all boolean columns; `data` is for React Flow and subflow state. The flag belongs with them, so it now has `error_enabled` and one home. The shuttle helper, its `BlockData` mirror, the store's fallback read, and the comparison exclusion the duplication forced are all gone. Backwards compatibility, since released versions draw the error port with no toggle in front of it: a block already wired to an error edge HAS the output on, because there was no other way to draw that edge. That rule is now stated in three places and none may be narrowed to read the flag alone — - the migration backfills `error_enabled` from the edges, so live rows are true before any new code reads them; - `materializeDeploymentState` derives it for a version's frozen jsonb, which the migration cannot reach — otherwise every workflow deployed before the toggle would ask to be redeployed once; - `workflow-block.tsx` keeps it at render time for states that reach the canvas through neither (imports, copilot edits), where unmounting the port would make React Flow drop the edge leaving it. The migration also moves any `data.errorEnabled` a developer created on this branch onto the column and strips the key; both statements match zero rows in production, where it never shipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(canvas): realign the Snowflake and Dynatrace sentences with staging's blocks Both breaks are the class the union check cannot see: separate hunks of the same file merged cleanly, and the result names fields that no longer exist. A sentence that does resolves to nothing, with no throw and no log. Snowflake's rewrite (#6474) moved database, schema, table, warehouse and procedure onto canonical selector pairs, so seven clauses anchored on ids that are gone. Each now names both members of its pair, which is also what keeps the card readable for someone working in advanced mode. Its nine new operations have sentences. Dynatrace (#6463) scoped the mute reason to the operations that mute, because unmuting accepts exactly one — so the two unmute sentences were asking for a field their card no longer shows. They drop the clause. Coverage is 4736/4736 operations across 321/321 blocks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(api): drop the block-data error flag from the workflow contract Left behind by the consolidation: the flag no longer lives in `data`, and a schema that still declares it there invites the mirror back through the wire. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(deploy): compare edge handles by port, so a falsy one cannot read as changed `loadWorkflowFromNormalizedTables` now runs handles through the canonicalizer, which falsy-coalesces — so an edge persisted with `sourceHandle: ''` loads as no handle at all. The server diffs that against the deployment version's raw jsonb, which still has `''`, and the set comparison reads one edge as removed and another added. Every workflow holding such an edge would ask to be redeployed the moment this ships, for nothing. Two write paths use `?? null` rather than `|| null`, so `''` is reachable. Canonicalized inside `normalizeEdge` rather than at either call site: the two sides are loaded by different paths and only some of them normalize, so the comparison has to be unable to tell two spellings of one port apart however its inputs arrived. This is the change reverted in 066e18ac28. That revert reasoned only about side-anchored ids, which are genuinely unreachable — it missed that the same coalesce collapses the empty string, which is not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(deploy): keep ignoring the error flag's old home in block data Deploying never converged: the badge asked to redeploy again the moment it finished, forever. The flag lived in `data` before it had a column. `0287` moves it, but a migration only reaches the live tables — every deployment version already written is frozen jsonb that keeps the old key. The wire schema no longer declares it either, so Zod strips it from the live state on its way to the client. So the two sides of the check genuinely differ: live `data: {}` against a snapshot's `data: {errorEnabled: false}`, reported as `data.errorEnabled` changed. Deploying cannot fix that — the next snapshot is taken from rows that still carry the key. Confirmed against a real stuck workflow: 18 versions, both blocks reporting `data.errorEnabled`, and the same two states comparing equal with this restored. Removing the exclusion in 7934df7f88 assumed the migration could reach every copy of the value. It cannot reach a frozen snapshot, so the comparison has to keep tolerating the old key regardless of where it survives. The block field is still compared on its own, with `!!`, so the flag itself is not ignored. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * revert(deploy): stop tolerating a block-data shape that never shipped The flag's stint inside `data` began and ended on this branch: `main` and `staging` have zero mentions of `errorEnabled`, and `ci.yml` gates every deploy job on a push to main/staging/dev, so opening a PR deploys nothing. No released version ever wrote the key, which means no production row and no production deployment snapshot can hold one. That makes the exclusion permanent code apologizing for a shape that cannot reach the database it defends. Migration 0287 already strips the key from the live tables, which is where a one-time data fix belongs; in production it matches zero rows, and on a developer database it makes the next deploy write a clean snapshot. Reverts 5ece9f9e7e. That fix was correct about the mechanism and wrong about the scope: it read a local database as evidence about production. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * improvement(canvas): retract bystander cards during a run, slant the sweep mark Two things about a running canvas. A run pinned every card's action bar open and suspended every card's hover on top of it, so the canvas became a wall of open swells that could neither retract nor respond to a pointer — and the `isWorkflowRunning && !isRunning` hover treatment already written for those cards was unreachable. Only the card that is actually running is pinned now; the rest behave as they do at rest, which is what makes hovering one bring its bar up again. The sweep's filled slot painted a full 24px square. It now paints a slanted band across the slot, as a hard-stop gradient rather than a `clip-path` — the two end slots already carry one for the swell silhouette and a second would have to win a specificity race with it. The stops hold `--surface-2` exactly, so only the shape changes. Each variant is spelled out because Tailwind's JIT reads literal class strings and a composed `hover-hover:${FILL}` compiles to nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * improvement(canvas): fill the running sweep one way, and tighten its mark The sweep drained back to empty after each pass, which reads as undoing the progress the block is making. It fills left to right and starts over. The direction flag goes with it — the state is just the count now. The slanted mark also sat too far off its neighbours. Slant and tightness trade against each other here: the transparent wedge has to be at least as wide as the edge's horizontal travel, or the cut clips a corner instead of crossing the slot. Leaning 7° off vertical instead of 17° travels 2.9px across the 24px slot rather than 7.3px, which brings the wedge in from 26% to 12% — 3.2px a side against 7.8px, so the gap between marks drops from ~17.6px to ~8.4px with the slant still crossing cleanly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * improvement(canvas): fill the running bar once, edge to edge The sweep restarted from empty every time it filled, so the bar kept re-running ground it had already covered. It fills left to right once and holds. The mark also sat inset in its slot, which put a gap on both sides of every join and made the row read as separate chunks instead of one bar. It now spans its slot edge to edge, leaving only the row's own `gap-[2px]` between marks, and takes its weight off vertically instead: `bg-clip-content` with symmetric padding paints a 10px band inside the 24px slot without changing the slot's size, so the swell measured around it does not move. `--surface-2` is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * improvement(canvas): paint the running bar as right-leaning uprights The fill read as a row of horizontal slabs. Each filled slot now paints one narrow upright bar leaning right, so a run fills as `/ / / /`. Geometry, since the two constraints fight: leaning the edge 15° off vertical carries it 4.3px across the bar's 16px height, so the transparent margin has to stay above 15% or the cut clips a corner instead of crossing top to bottom. 38%/62% leaves a 7px bar with room to spare. Height comes from `bg-clip-content` plus symmetric padding, which does not change the slot's own size, so the swell measured around it stays put. `--surface-2` is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * improvement(canvas): pitch the running hatch to the row, not to one bar a slot One bar per 24px button left the rest of the button empty, so the marks inherited the button grid's rhythm and sat ~19px apart — a row of isolated ticks rather than a loader. The fill repeats now, at a pitch that divides the row's own rhythm: a slot plus its `gap-[2px]` is 26px, so a 13px horizontal pitch puts exactly two bars in every slot and stays in phase across the gaps, including the 40px end slots. Bars land every 13px with a uniform 6px between them, whatever the run's length. Stops are measured along the 105° axis rather than horizontally, so they carry the `sin(105°)` factor: a 7px bar on a 13px pitch is 6.76px on a 12.56px period. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(canvas): paint the running hatch once across the row, not per slot The bars came out bunched in some places and spread in others. Per-slot backgrounds cannot avoid that: each button starts its own gradient at its own origin, so the phase resets at every slot — and the end slots are 40px against the others' 24px, so the resets are not even uniform. Three passes of tuning the stops were all chasing a constraint the approach could not satisfy. The hatch is now one element spanning the row, so there is one gradient and one phase. It sits behind the buttons and grows by width: the run/stop button keeps an opaque fill while running and masks the part growing underneath it, and every other slot is transparent mid-sweep so the hatch reads through. The slots no longer paint anything themselves, and the per-slot filled flag goes with them. `--surface-2` is unchanged; only where it is painted moved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * improvement(canvas): close the running hatch's gaps The hatch ran a 50/50 duty cycle — 6px of fill to 6px of air — which read as sparse. It now runs 8px to 3px. Both stops are measured along the 105° axis rather than horizontally, so each carries a `sin(105°)` factor; the note records that, and that closing the gap further is a matter of moving the first stop toward the second. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(emcn): drop the brand highlight from popover menus (#6506) Context menus opted into a palette of their own — `variant='secondary'` for a brand-blue row highlight and `colorScheme='inverted'` for a dark card — so the canvas, block, toolbar, terminal, sidebar, and preview menus looked nothing like the menus everywhere else in the product. Removes both overrides so they inherit the same surface, border, and `--surface-active` highlight the terminal's overflow menu already uses, and drops the brand state from the Popover itself along with the `variant` prop that only ever selected it. One fewer way to style a menu. * feat(executor): opt-in per-block retry (#6505) * feat(executor): opt-in per-block retry Adds a per-block retry policy, off by default, surfaced in the editor's additional-fields disclosure alongside the block's other advanced settings. A block that opts in replays its handler while tries remain, then rethrows the final error so the error port behaves exactly as it does for a block that never retried — retrying only delays the existing outcome, never changes it. Retry is deliberately indiscriminate about the failure, since there is no reliable way to tell a transient error from a permanent one and classifying would silently do nothing for the generic errors people turn it on for. Only throws that are not failures are excluded: a deliberate stop, a child workflow whose own blocks already ran their policies, and the block types whose throw is control flow (human-in-the-loop, sentinels, subflow containers, notes, triggers). Eligibility lives in one predicate read by both the editor and the executor, so a block can never keep retrying after an edit that hides its control. `retry` is a nullable jsonb column; NULL means "runs once", which is how every existing block already behaves, so the change is inert until someone opts in. Bounds are clamped on read rather than rejected, so a value written before a bound moved still resolves to something runnable. Also decouples the additional-fields disclosure from `block.advancedMode`. That flag decides which member of a canonical pair serializes, so opening the disclosure used to be able to drop a block's configured credential. Expansion is now view state; the stored flag is no longer written by the editor. Retried blocks report their try count on the trace span, shown in log details. * fix(realtime): allow the write role to persist a block retry policy `update-retry` was added to the protocol but not to the write-role allowlist, so the editor applied the change optimistically while the server dropped it and the policy never reached the database. Adds a test asserting the write role holds every per-block operation the protocol declares, so the next block setting cannot repeat this silently. * fix(editor): keep a retry number field's value when it is blurred untouched Committing on blur normalized the draft unconditionally, and an untouched field's draft is null — which normalizes to the default. Focusing and leaving Max tries silently reset a configured 5 back to 3. * improvement(canvas): close the running fill to solid, slant its leading edge Gaps gone entirely: the bar is one solid fill now. The slant moves onto the growing edge, because a repeat with its gaps closed has no edges left to show. 4px of run across the 16px height is the same 15° lean the bars carried, so the fill still leans right — it just leans at its front instead of throughout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * improvement(canvas): return the running fill to the squares' rhythm, sheared Marks are 24px with the row's 2px gap after them again — the geometry the slots carried before any of this — so they land where the squares did. The shear is the only thing that is new. It stays on the single spanning element rather than going back to per-slot backgrounds: one gradient means one phase, which is what lets the 24/2 rhythm hold across the 40px end slots instead of resetting at every boundary. Stops are measured along the 105° axis rather than horizontally, so both carry a `sin(105°)` factor: 24px of mark is 23.18px of stop, and the 26px pitch is 25.11px of period. Writing 24/26 directly renders ~3.5% wide and drifts out of the squares' rhythm across the row. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(canvas): trim the running fill to the swell's tapered end The fill ran off the block. The row is a rectangle but the swell is not — its last slot cuts a diagonal so the shape narrows toward the top, and a rectangular overlay therefore painted past the gray edge up there while still sitting inside it at the bottom. The per-slot version never showed this because each button's own clip contained its fill; moving the paint onto one spanning element took that containment away with it. The overlay now carries the same taper, read off that slot's own path: 16.67px in from the row's right at the overlay's top, 3.33px at its bottom, a slope of 20/24. Only applied to the swell variant, which is the shape that tapers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(canvas): stop the handoff highlight pinning a bystander's toolbar open Dropping `isWorkflowRunning` from `forceOpen` was not enough: it also read `usesSelectedVisuals`, which is `isNodeSelected || isExecutionHighlighted`, and the handoff highlight covers the block feeding the running one. So the upstream card kept its bar down for the whole run — the wall of open swells this was supposed to end, one card smaller. Those are two different questions. `usesSelectedVisuals` still drives the TREATMENT — the graphite silhouette and `data-node-selected`, so the eye can follow the baton — while whether the toolbar is pinned open now keys off selection alone. The container keeps `isRunning` by itself. Selection was never a pin there, and its own tests hold it to opening on hover. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(realtime): refresh the error flag on a block upsert `BATCH_ADD_BLOCKS` wrote `errorEnabled` on insert but left it out of the conflict clause, so re-adding an existing block id kept whatever the row already held while every sibling flag — enabled, advancedMode, triggerMode, retry, locked — was refreshed from `excluded`. The client's value was silently discarded and the old error-output state came back on the next load. Mine: the insert side gained the field when the column landed and the conflict set did not. The other two block writers delete before inserting, so no stale row survives them; this upsert was the only path that merged into one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * update loader animation * change the loader --------- Co-authored-by: andresdjasso <andresdjasso@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Waleed <walif6@gmail.com>	2026-08-10 19:21:32 -07:00
Vikhyath Mondreti	37628f1566	fix(workflow-persistence): make persistMigratedBlocks' updated_at guard millisecond-tolerant (#5790) ... * fix(workflow-persistence): make persistMigratedBlocks' updated_at guard millisecond-tolerant * improvement(sockets): state improvements * address comments	2026-07-21 14:36:11 -07:00
Waleed	d078c84ee6	chore(typescript): upgrade to TypeScript 7 (native Go compiler) (#5521) ... * chore(typescript): upgrade to TypeScript 7 (native Go compiler) Bumps typescript to ^7.0.2 across every workspace package. Full bun run type-check/lint/build/test all pass; apps/sim's type-check (the one needing an 8GB heap bump) drops from ~55s to ~7s wall time. Migration fixes required by TS7's stricter defaults: - baseUrl removed: drop it from 5 tsconfigs (paths already resolved relative to tsconfig dir, so behavior is unchanged) and prefix the one bare (non-relative) paths entry each in apps/sim and apps/realtime with './' - moduleResolution=node10 removed: switch packages/cli and packages/ts-sdk to "bundler", matching the rest of the monorepo - types now defaults to [] instead of auto-including every @types/* package: add "types": ["node"] to the shared base tsconfig (this is fundamentally a Node monorepo, so this restores prior behavior in one place instead of duplicating it per-package), add explicit @types/node deps to packages that now rely on it transitively via @sim/db/@sim/logger, and add "declare module '*.css'" to the two packages with plain (non-module) CSS side-effect imports that TS7's stricter checker now flags - packages/logger's isomorphic `typeof window` check no longer needs DOM lib in every consumer: replaced with `'window' in globalThis` - packages/testing and apps/realtime's fetch/DOM mocks need DOM lib where they're compiled, since they model the browser Fetch API - the `typescript` npm package no longer exports the classic Compiler API from its main entry (moved to unstable/ast subpaths); apps/sim's Function-block route used it at runtime to strip import statements from user code, so that one call site now uses Microsoft's official transition package, @typescript/typescript6 - Next.js 16.2.6's own TypeScript-detection heuristic hardcodes a path TS7 no longer ships, and its auto-install fallback assumes npm/pnpm; added @typescript/native-preview as a devDependency to apps/sim and apps/docs so Next detects a valid native compiler instead of trying (and failing) to auto-install one Not merging yet: TS 7.0.2 published today and is still inside this repo's bunfig.toml minimumReleaseAge (7-day) supply-chain gate, so `bun install` will fail for everyone until 2026-07-15. Opening this now to get it through review; hold the actual merge until then. * fix(typescript): address Greptile review findings on TS7 upgrade - packages/logger: 'window' in globalThis treats a shim that leaves globalThis.window explicitly undefined as browser-only, silently dropping production server logs. Restore the original typeof !== 'undefined' semantics via an inline cast instead, so it stays correct without requiring DOM lib in every consumer. - packages/ts-sdk, packages/cli: both are tsc-built, published as Node ESM (package.json "type": "module" with an "exports" map). "moduleResolution": "bundler" is too permissive for that target - it accepts import patterns (e.g. extensionless relative imports) that Node's actual ESM resolver rejects at runtime. Switch both to "module"/"moduleResolution": "nodenext", the correct pairing for a published Node ESM package. Verified real tsc builds (not just --noEmit) still succeed for both. * chore(bunfig): temporarily disable minimumReleaseAge gate for TS7 install TS 7.0.2 published today, still inside the 7-day gate. Lowering to 0 to unblock this merge; will restore to 604800 in an immediate follow-up commit right after merging.	2026-07-08 16:41:23 -07:00
Waleed	9d34fbea1b	refactor(utils): consolidate duplicated helpers onto @sim/utils (#5509) ... * refactor(utils): consolidate duplicated helpers onto @sim/utils Replaces ~90 hand-rolled reimplementations of error-message extraction, postgres error-code checks, sleep, Math.random, retry/backoff, object filtering/omission, noop, string truncation, date/time formatting, email normalization, and plain-object type guards with the shared @sim/utils exports. Wires check:utils into CI (test-build.yml) so these patterns don't regress. * fix(test): mock @sim/utils/random instead of Math.random in schedule-execute tests The schedules/execute route previously used Math.random() for jitter delay; this consolidation PR switched it to randomInt() from @sim/utils/random, which is backed by crypto.getRandomValues() rather than Math.random(). The route.test.ts spies on Math.random() no longer had any effect, so jitter became real random delay instead of the deterministic 0ms the tests expect, causing intermittent 10s timeouts in CI. * fix(retry): preserve uncapped Retry-After comparison in tools/index.ts parseRetryAfter() caps its return value at 30s by default. tools/index.ts compares the parsed Retry-After against a caller-configured maxDelayMs to decide whether to skip a retry entirely -- capping before that comparison silently defeats the skip check whenever maxDelayMs is configured above 30s, since a Retry-After between 30s and maxDelayMs would incorrectly look "within limits" and get retried instead of skipped (caught by Cursor Bugbot). Added an optional maxMs param (default unchanged) so tools/index.ts can request the raw, uncapped value for its own comparison while backoffWithJitter still clamps the actual sleep duration to maxDelayMs. Added a regression test covering maxDelayMs > 30s. * fix(utils): fall back to Intl-resolved abbreviation for unmapped timezones getTimezoneAbbreviation only covered 9 hardcoded IANA zones and returned the raw IANA string for everything else, degrading schedule descriptions for zones like Europe/Berlin or America/Toronto (caught by Greptile). The deleted local implementation in schedules/utils.ts resolved any valid IANA timezone generically via Intl.DateTimeFormat's short timeZoneName. Restore that as a fallback so only genuinely invalid timezone strings return themselves unchanged.	2026-07-08 11:26:23 -07:00
Vikhyath Mondreti	43f53bb7d3	feat(execution): payload size bottlenecks with lazy execution value hydration, safer materialization, and batched parallel execution (#4560) ... * improvement(resolver): lazy resolution for underlying fields greater than 10MB * progress * feat(parallel): batching * codegen to allow inline substitution * address comments * ui inconsistencies * cleanup redundant code * address more comments * address comments * replace helper * fix tests	2026-05-12 14:57:32 -07:00
Vikhyath Mondreti	1e5cc392d5	improvement(deployment): solve multiple client side races, and deployed state management issues (#4502) ... * improvement(deploy): state transitions * more fixes * improvements and shift to outbox policy with eager call * address comments * ux improvement * address comments * address bugbot	2026-05-07 17:20:04 -07:00
Vikhyath Mondreti	09425553b3	fix(sanitize): repair malformed subblock states (#4490) ... * fix(sanitize): repair malformed subblock states * address comments * address comment	2026-05-07 00:54:38 -07:00
3 people	5f0f0edd63	improvement(repo): separate realtime into separate app (#4262) ... * improvement(repo): restructuring to make realtime image narrower scoped * improvements * chore(repo): rebase fixes and quality improvements for realtime split Addresses merge-time issues and gaps from the realtime app split: - Retarget stale vi.mock paths to @sim/workflow-persistence/subblocks - Restore README branding, fix AGENTS.md script reference - Restore TSDoc on workflow-persistence subblocks helpers - Use toError() from @sim/utils/errors in save.ts - Add vitest config + local mocks so @sim/audit tests run standalone - Move socket.io-client to devDependencies in apps/realtime - Add missing package COPY steps to docker/app.Dockerfile - Add check:boundaries/check:realtime-prune scripts and wire into CI Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(security): consolidate crypto primitives into @sim/security Move general-purpose crypto primitives out of apps/sim into the @sim/security package so both apps/sim and apps/realtime can share them. @sim/security exports (all pure, dependency-free): ./compare safeCompare (constant-time HMAC-wrapped equality) ./encryption encrypt/decrypt (AES-256-GCM, iv:cipher:tag format) ./hash sha256Hex ./tokens generateSecureToken (base64url) Migrate apps/sim call sites to use these + @sim/utils helpers: crypto.randomUUID() -> generateId() from @sim/utils/id createHash('sha256').digest -> sha256Hex timingSafeEqual on hashed hex -> safeCompare new Promise(setTimeout) -> sleep from @sim/utils/helpers No behavior change: encryption format, digest output, and token length are preserved exactly. * refactor(copilot): use toError in remaining otel/finalize sites Replace the last two `error instanceof Error ? error : new Error(String(error))` patterns with toError from @sim/utils/errors. Completes the sweep of clean candidates — no behavior change. * refactor(security): consolidate HMAC-SHA256 primitives into @sim/security Adds hmacSha256Hex and hmacSha256Base64 to @sim/security/hmac and migrates 15 webhook providers plus 5 other hot paths (deployment token signing, outbound webhook requests, workspace notification delivery, notification test route, Shopify OAuth callback) off bare `createHmac` calls. Secret parameter accepts `string | Buffer` to cover base64-decoded Svix-style secrets (Resend) and MS Teams' HMAC scheme. AWS SigV4 signing in S3 and Textract tools intentionally retains direct `createHmac` usage — its multi-step key derivation chain doesn't fit a generic helper. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(packages): post-audit test + packaging polish - Add safeCompare unit tests (identity, length mismatch, hex-nibble diff). - Add Buffer-secret cases to hmac tests to lock in Svix/MS-Teams contract. - Declare `reactflow` as a peerDependency on @sim/workflow-types — only used for type imports. - Add a barrel export to @sim/workflow-persistence for consumers that prefer package-level imports; subpath exports retained. - Document the data-field invariant in load.ts for loop/parallel subflow patching. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(realtime): address PR review feedback - Remove redundant SOCKET_PORT=3002 env from Dockerfile runner stage (env.PORT already defaults to 3002 via zod schema). - Reorder PORT fallback so an explicitly-set SOCKET_PORT wins over the schema default for PORT; keeps SOCKET_PORT functional as an override instead of dead code. - Add dedicated type-check CI step for @sim/realtime so TS errors surface pre-deploy (the Dockerfile runs source TS via Bun and has no implicit build-time type check). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(realtime): remove unused SOCKET_PORT env var SOCKET_PORT has lived in the socket server since the June 2025 refactor but was never actually set in any deploy config — docker-compose.prod, helm values/templates, .env.example, and docs all use PORT or the 3002 default exclusively. No self-hoster was ever pointed at SOCKET_PORT, so removing it is safe. Simplifies realtime port resolution to `env.PORT` (zod-validated with a 3002 default) and drops the orphaned sim-side schema entry. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Waleed Latif <walif6@gmail.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-22 23:06:16 -07:00