817 Commits

Include renames

Author	SHA1	Message	Date
Waleed	f3ccdfe7de	feat(providers): add GLM-5.3 and GLM-5.3-Flash to Z.ai (#7104)	2026-08-26 10:12:18 -07:00
Waleed	a42299066d	fix(integrations): repair broken endpoints, silent failures, and a path traversal (#7096) ... * fix(integrations): repair broken endpoints, silent failures, and a path traversal - qdrant: search_vector returned the /points/query envelope instead of the points array, so downstream blocks saw an object where an array was declared - elasticsearch: search/count/create_index silently swallowed malformed JSON — count fell back to counting the entire index, search to match_all - serper: only 4 of 6 advertised verticals were mapped; videos and shopping returned empty (still billed) result sets. Replaced the if/else chain with a vertical dispatch table that hard-fails on an unknown type - enrow: find_email flattened the nested info object incorrectly, dropping firstname/lastname, and advertised a linkedin_url the API never returns - linkedin: share_post read postId from an empty body; /v2/ugcPosts returns it in the x-restli-id header - vercel: edge config endpoints moved to /v1/global-config - vercel: encode edgeConfigId so a traversing value cannot escape the base path - sixtyfour: enrich endpoints moved to /people-intelligence and /company-intelligence - langsmith: hard-coded API host consolidated to one constant, added missing non-ok guards, encoded run ids, capped echoed upstream error bodies - daytona: file upload moved to /files/upload-v2 - memory: PUT persisted a bare object where POST and the declared type both use an array Adds 53 tests across the affected tools. * fix(integrations): close path traversal fleet-wide, stop a credential reaching the wire Follows up the review round on this branch. Security: - The previous encodeURIComponent-only guard was incomplete. '.' and '..' are unreserved, so they survive encoding and the URL parser then removes them as dot segments — popping one path segment on a fixed host with the caller's bearer token still attached, including on DELETE. Adds a shared safeUrlPathSegment helper that rejects empty, '.', '..', and any residual path separator, and applies it across every Vercel and Daytona tool that interpolates an LLM-writable id into a request path - langsmith: create_run and create_runs_batch spread the whole params object into the request body, so the LangSmith API key was sent to LangSmith and stored in the run record. Request bodies are now built from an explicit allowlist of run-ingest fields, so an unlisted param cannot reach the wire Correctness: - langsmith: the run-payload normalizer was not idempotent and ran once in request.body and again in transformResponse, so a caller who left Run ID blank got back an id that was never sent. Downstream update_run/create_feedback wired to it would 404 - langsmith: batch patch entries were normalized as if they were new runs, minting ids and overwriting start_time/trace_id/dotted_order - langsmith: the 500-char error cap appended its ellipsis after slicing, so the advertised bound was actually 503 - serper: scholar and patents mapped a date field neither vertical returns, and a test asserted it. Their organic response key is now confirmed against Serper's published per-vertical examples rather than assumed - serper: an unknown vertical derived from the response URL turned a successful response into a thrown error; only a user-supplied type now hard-fails - linkedin: warn when a success status carries no x-restli-id header Also corrects the Vercel endpoint rationale in the PR description: the old /v1/edge-config path still routes and is not scheduled for removal, so the move to /v1/global-config is canonical alignment rather than a break-fix.	2026-08-25 21:04:04 -07:00
Theodore Li	d1786e92e6	fix(custom-blocks): restore self-host entitlement gate (#7098) ... * fix(custom-blocks): restore self-host entitlement gate * chore(helm): bump chart version	2026-08-25 23:45:00 -04:00
Waleed	2d85c0d1e1	fix(cli): correct three descriptions the CLI publishes, and restore --no-recursive (#7093) ... Follow-ups from review of the v0.8.12 release PR, all on surfaces the CLI audit touched. - `credentialId` was one shared schema across PATCH and DELETE, so the disconnect reference offered "update or disconnect" for an operation that cannot update. Split into two, matching the two components OpenAPI already publishes for them. - A boolean query param documents the spellings an HTTP caller may send and closes by calling them the whole accepted set. The CLI renders those fields as bare flags that take no value, leaving the sentence pointing at a list neither `--help` nor the reference ever prints. Stripped for bare flags only; the REST prose and OpenAPI specs are unchanged. - `files list --recursive` became a bare switch in the audit, which removed the only way to send false. The API turns it on by itself as soon as `--search` is set, so a folder search always descended. The twin is back, declared per flag so the one-way toggles do not grow a meaningless negation.	2026-08-25 19:28:06 -07:00
Waleed	4508ec75d2	fix(cli): resolve blockers and majors from a full command-surface audit (#7083) ... * fix(cli): resolve blockers and majors from a full command-surface audit Audit of all 222 CLI leaves against a live deployment, plus fixes for every defect it confirmed. Blockers: - An unrecognized --profile resolved to built-in defaults, so a typo silently targeted production and transmitted the API key there. - sim logs follow sent an undeclared query key and failed on every invocation. - sim workflows run exited 0 on a failed run, so CI reported success. - knowledge connectors documents update matched rows already in the target state, making exclude and restore permanent no-ops. - PDF text layers below the OCR threshold are transcribed by a model and stored verbatim with no record that it happened. Majors include: rollback --version was swallowed by the program-level flag and silently did nothing; nullable string flags could not send null despite their help promising it; six protocol commands discarded excess arguments, dropping files on upload; sim chat crashed with EPIPE when piped to head; tables import dropped malformed CSV rows without reporting them; audit-logs required an organization id no API surface exposed; secrets could not opt out of redaction or read a value from a file; bulk deletes and moves exited 0 having done nothing; and MCP registrations were destroyed by undeploy rather than restored. Adds extraction_method to documents so OCR output is distinguishable from parsed text, and reports the applied scope on billing logs so the two ledger questions are no longer indistinguishable. * fix(mcp): bound MCP restore by server and re-check uniqueness under the lock Two gaps in the archive/restore lifecycle this branch introduced. The candidate query bounded archived rows and deduplicated to one per server afterwards, so several archived generations stacked on one server consumed the whole budget and every other server the workflow had been published on fell out of the result with no warning. Deduplication moves into SQL so the bound applies to servers, preserving most-recently-updated-per-server. The live-registration check ran before the server lock was acquired, so a concurrent tool create could land in between and the restore would un-archive a second live row for the same server and workflow, violating the partial unique index and rolling back the whole deployment. That check now runs under the lock alongside the tool-name, capacity, and metadata-budget checks it belongs with. * fix(review): address round-three review findings across CLI and server Restore now picks candidate servers by recency: DISTINCT ON forces its own key to lead the sort, so bounding on that statement kept the lexicographically lowest server ids and left a workflow's most recently used servers archived. Deduplication and bounding are now separate stages. CLI: a total miss on tables move reported only in notFound exited 0; unsetting a key or removing a profile mutated the first duplicate INI block while reads merged later ones, so the removal appeared to succeed and did nothing; an import that rejected cells but no rows showed a clean progress line; and the --run-id help implied idempotency it does not provide. Server: a run with no recorded output projection let block-name selectors past the new validation; the billing window comparison still fired on a bound that parsed but failed the shared schema; CSV rejection accounting reached only the streaming path, so buffered and synchronous imports still dropped records silently, through to the Copilot tool that reports them; case-insensitive tag name uniqueness now serializes on the knowledge-base row the delete paths already lock; and a failed sync claim reports the lifecycle reason rather than always claiming a sync is in progress. Reverts an over-scrub from the previous commit: workspace-file-imports is consumed only by Copilot, so naming save_upload and glob there is the correct remediation rather than a leak, and the sweep that guards against leaks now exempts it explicitly. Corrects two contract descriptions that promised a bulk tag save would rename or relocate an occupied slot, which it deliberately no longer does. * fix(cli): remove flags a caller cannot use, and correct three that misled Removes surface that should not have shipped: - `files uploads get` is hidden. Its `--upload-token` was required, and the token is minted and consumed inside a single `files upload`, which completes or aborts its session before returning. Nothing in the CLI could produce the value, so the command answered every invocation by asking for something unobtainable. The same flag is dropped from the two table-import commands, where a CLI-created import is already queryable without it. - The `--no-<flag>` companion that sent JSON null is gone. `--no-X` means "send boolean false" on thirty-seven other flags, and one spelling should not carry two meanings. `--description ''` already clears the displayed value, and the help now warns that the literal word null is stored as text rather than suggesting a substitute, because on the OAuth client fields null revokes a stored grant and an empty string does not. - The document extraction method column and its contract field are reverted. Nothing read them, they were null for every existing document, and the name collided with the parser metadata field that already exists. Corrects flags that misled: the workflow move destination is `--to`, matching its two siblings rather than meaning the opposite of `--folder` one command over; `files list --recursive` is a bare flag like the four folder deletes rather than a twelve-alias string; the dispatch row cap takes a count instead of its wire object; `--yes` no longer claims to be required on commands that accept `--dry-run`; cancelling every run on a table is confirm-gated; and the retry-processing negation, which the route rejects, is suppressed. Extends the guard that missed all of this: it swept only `--x-` prefixes over generated commands, so a header spelled without one was invisible to it. It now derives every header name from the operation table and sweeps the assembled program. * fix(mcp): budget MCP restore against the workflow's live server fanout Restore bounded its candidates at the per-workflow server limit counting archived rows only, never subtracting the memberships the workflow already holds live. The fanout validation a few lines later in the same deploy transaction counts live servers against that same limit, so a workflow with both live and archived registrations could restore past it and roll the whole deployment back. The candidate query now bounds on the remaining headroom, and the budget is re-checked under the server locks and spent once per accepted candidate, so a create that lands between the count and the unarchive cannot push it over. Candidates that do not fit are dropped by recency, matching how the set is already selected, and stay archived with a warning naming the workflow, the server, and the reason — restore still never throws inside the deploy transaction. One residual is left open deliberately: a create on a server outside the candidate set is serialized by neither the locks nor the recount. Closing it would need a workflow-level lock, which would change the ordering every other writer here depends on, and the create path runs its own limit check. * fix(mcp): stop restore spending its budget on servers it cannot restore A server can hold both a live registration and archived ones for the same workflow: the partial unique index constrains only the live row. Such a server was counted twice — once shrinking the restore budget, once consuming one of its slots — before the liveness check under the lock skipped it. While the bound was the full server limit that waste was invisible; once the bound became the remaining headroom, every slot spent that way cost a registration that could have been restored. The candidate query now excludes servers the workflow is already live on, so the budget and the candidate set agree. The exclusion sits on the inner stage, before deduplication and the bound, and is a pre-lock optimisation only: the check under the server lock stays authoritative, because the query can go stale between reading and unarchiving. Candidates rejected for a tool-name collision, the per-server cap, or the metadata budget are still not replaced. That case is only knowable under the lock, so replacing it would mean fetching past the bound and locking servers outside the candidate set, widening an ordering every writer here relies on.	2026-08-25 18:43:48 -07:00
Theodore Li	19b8652f35	chore(flags): remove released feature gates (#7087) ... * chore(flags): remove released feature gates * chore(helm): bump chart version	2026-08-25 20:26:40 -04:00
Waleed	98a453d7e6	feat(integrations): add Microsoft Word (#7069) ... * feat(integrations): add Microsoft Word * fix(microsoft-word): guard document edits against concurrent overwrites * fix(microsoft-word): reject non-Word targets, scope SharePoint access, and tighten input bounds * chore(docs): regenerate docs manifest for the Microsoft Word page * fix(microsoft-word): strip XML-forbidden control characters from generated documents * fix(microsoft-word): fail closed when a document reports no version to compare	2026-08-25 11:38:20 -07:00
Waleed	ed60fbaa4e	feat(integrations): add Semrush (#7068) ... * feat(integrations): add Semrush Adds the Semrush SEO API as a block with 44 operations across overview, domain, subdomain, URL, keyword, comparison, and backlink reports. Reports answer as delimited CSV, so the shared decoder maps each header cell back to the export column it was requested as rather than reading by position: the API may return fewer columns than were asked for, and two columns can render under the same header label. * fix(integrations): sync docs manifest for the Semrush page * fix(integrations): address Semrush review findings - Only the API key stays user-only; report selectors (target scope, limit, offset, date, sort, filter) are user-or-llm so an agent can set them - Hold the row limit at one row: a positive fraction floored to zero and sent display_limit=0 - Locate Domain vs. Domain metric columns by their own headers, so a dropped position column shortens the compared-domain run instead of shifting competition, search volume, and CPC onto the wrong values - Describe competitor and domain-list metrics as belonging to the row's domain, not to the target - Describe paid and URL traffic cost as an estimated cost, matching the Traffic Cost header those reports return, not the organic Traffic Cost (%) - Drop the newest-first claim from history outputs, which order by display_sort - Replace the erased any casts in the request tests with a typed helper	2026-08-25 11:17:58 -07:00
Theodore Li	4c0bd944dc	feat(slack): launch v2 triggers and backfill custom bots (#6873) ... * feat(slack): launch v2 triggers and backfill custom bots * fix(slack): propagate legacy webhook dispatch failures * fix(slack): continue shared legacy webhook fanout * fix(slack): acknowledge filtered webhook deliveries * fix(slack): finalize custom bot migration rollout * fix(slack): dedupe migrated bots per workflow * fix(slack): harden custom bot rollout * fix(slack): acknowledge permanently ignored deliveries * fix(slack): retry failed webhook deliveries	2026-08-25 14:11:02 -04:00
WaleedandTheodore Li	656840a1b1	feat(api): make the platform operable headless over v2 (#6912) ... * feat(v2): download run output files by API key Adds GET /api/v2/workflows/{id}/runs/{runId}/files/{fileId}, closing the async-run loop for headless callers. A run's output carries UserFile URLs pointing at /api/files/serve/..., which rejects x-api-key outright, so an async run that produces a file previously had no byte path out for an API key at all. The file is addressed by the id the run reported and resolved against the run's own recorded execution data, from which the storage key is read. The request never supplies a storage key, so the endpoint cannot be aimed at bytes the run did not produce. Resolution deliberately reads the materialized-but-undisplayed recording, because the display projection strips exactly the `key`/`context` fields a byte read needs. Also hardens normalizeStartFile to derive a file's storage key only from a validated internal serve URL, discarding any caller-supplied `key`/`context`. A workspace API key has no human subject, so the executor resolves its actor to the workspace billing owner (preprocessing.ts -> resolveSystemBillingAttribution); verifyFileAccess then authorizes a workspace-context key as that owner, whose reach is not bounded by the key's workspace. Accepting an attacker-authorable key made that substitution exploitable as a confused deputy. Normalization is all-or-nothing, so a forged file now drops the whole files input. * feat(workflows): one graph-write door, principal-derived audit source, and v2 authoring endpoints Extract replaceWorkflowNormalizedState as the single persistence primitive for a workflow graph replace and route both the internal editor save and the Copilot edit tool through it, so neither can skip state preparation, the row lock, the lastSynced stamp, or custom-tool extraction by choosing a different entry point. Derive the audit source from the acting principal instead of hardcoding 'copilot', then widen workflows.variables.apply_operations and workflows.bulk.move to every principal kind. Add GET/PUT /api/v2/workflows/{id}/state, POST /operations, /duplicate, /restore, PATCH /variables, and POST /api/v2/workflows/move over surface-neutral application use cases; move the edit engine to lib/workflows/editing. * test(workflows): cover the graph-write primitive, audit source, and the v2 authoring surface Pin the two-doors fix (preparation runs, the row is locked, custom-tool extraction is post-commit and best-effort) and the false-audit fix (a session principal writes source: 'session', a delegated one writes its service). Both were verified to fail with the fix reverted. Add the application matrix for replaceWorkflowState, applyWorkflowOperations, readWorkflowGraph, and restoreWorkflow — role floor, principal-kind rejection before canonical load, asserted-scope concealment, lock, validation, atomic conflict, plan gate, and audit-then-notify ordering — plus route tests for every new endpoint. * test(workflows): pin the internal graph-write door and the v2 list scope Characterize saveWorkflowNormalizedState's statuses, messages, and notification after the persistence extraction, and cover the new scope filter on GET /api/v2/workflows including a cursor replayed under a different scope. * feat(api): add v2 block, tool, connector-type, and enrichment catalogs Adds six read endpoints under /api/v2 that publish Sim's code-defined catalogs: GET /blocks, GET /blocks/{blockId}, GET /tools, GET /tools/{toolId}, GET /connector-types, and GET /enrichments. These read like static reference data and are not. What a caller may place is decided per workspace by its permission-group integration allowlist, per organization by which unreleased blocks have been revealed, per deployment by ALLOWED_INTEGRATIONS, and per workspace again by the workflows it has deployed as blocks. So all six are plain defineWorkspaceOperation reads at minimumRole 'read' with workspaceApiKey 'allow' — the exact policy of credentials.providers.list — and every response keeps Cache-Control: private, no-store, because an unrevealed preview block's existence must not leak across organizations through a shared cache. Trigger blocks ride as ?capability=trigger rather than a second endpoint, and workspace custom blocks ride inside /blocks discriminated by `source`, so "what may I place?" stays a one-call question. The block projection is extracted out of the Copilot get_blocks_metadata tool and rewritten onto @/tools/metadata and @/tools/metadata-outputs. That cuts the tool's own @/tools/registry edge as a side effect: its module graph drops from 6,756 to 1,318, and the new routes land at 1,673-1,734, next to the shipped /v2/credentials/providers baseline of 1,668. Supporting changes: - scripts/sync-tool-metadata.ts derives hostedApiKey ('always' | 'conditional' | 'none') from each tool's `hosting`. The config itself stays excluded because it holds closures, but "does Sim host the key" is a first-order authoring question, so the answer is emitted. - getCopilotToolDescription takes hostedApiKey as an option instead of reading `hosting` off the tool, so both an executable ToolConfig and the generated metadata can answer it through one shared derivation. - principalUserId / allowedIntegrationTypes move out of lib/credentials/application/provider-catalog.ts into lib/integrations/principal-scope.server.ts. Two copies of the workspace integration gate would diverge first on the workspace-key path, which has no user for permission groups to key on. - scripts/check-tool-registry-boundary.ts walked page.tsx/layout.tsx under app/workspace only, so a route importing the executable registry passed green. It now walks a list of entry sources, seeded with the four catalog route subtrees and the shared projection barrel. Routes are covered per subtree rather than wholesale because 122 of ~1,130 route files legitimately execute tools. Registry sweeps parse every block, tool, connector type, and enrichment through its published response schema and compare against the wire round-trip. They caught a real drift while being written: an operation's inputs were typed as a union of the tool-param and block-input shapes, and the union resolved to whichever member matched first, silently dropping a block input's `schema`. * docs(api): stop publishing a 413 GET /workflows/{id}/state cannot emit * feat(v2): read upload-session state Adds GET /api/v2/files/uploads/{uploadId}. Only DELETE was exported, so a caller that lost track of a transfer could abort it but could not ask whether the session was still alive, already finalized, or failed — the resume story was missing. Runs on a new files.upload.read operation at minimumRole 'read' rather than reusing uploadCancel, which is a 'write': asking about a session must not require permission to destroy it. The GET is a control leg like every other, so it carries the signed upload token and re-authorizes the caller's present workspace permission through reauthorizeWorkspaceUploadPurpose instead of resolving the session on its id alone. * fix(api): reconcile v2 catalog and workflow-authoring integration Merging the catalog and workflow-authoring branches surfaced four issues that neither produced in isolation. - Route and OpenAPI counters were bumped to the same value on both branches, so git merged them as one change while the merged tree holds the sum. Corrects the route ratchet to 1142 and the workflows document to 29 operations (152 total), then regenerates the OpenAPI documents and the CLI surface from the reconciled contracts. - The seven new workflow operations were published in the spec but absent from the workflow API reference groups, which `check:openapi` rejects. - `route-policies.ts` reached `WorkflowOperationsNotAppliedError` through `apply-workflow-operations`, dragging the edit engine — and its diff and comparison dependencies, which reach a client OAuth hook — into every route that uses the shared workflow error policies. The class moves to its own leaf module, mirroring `WorkflowImportError`, and each importer now takes it from there. - The operations route test shadowed that class inside its module mock, so `instanceof` matched a fake and the assertion pinned a message the production class never emits. It now uses the real class and asserts the real message. * feat(v2): extract ZIP archives over the public API Adds POST /api/v2/files/{fileId}/extract and widens files.extract_archive from principalKinds ['session'] / workspaceApiKey 'deny' to admit personal and workspace API keys at the unchanged 'write' role. The widening is an authorization change, so the justification lives in the operation's TSDoc: extraction grants no capability an API key lacks, since every file it writes could be created one at a time through files.create and files.upload.create, both already 'allow' at the same role. It only collapses many calls into one. The previous ['session'] restriction read as an artifact of the UI having been the only caller. Delegated services stay out — no copilot or executor caller exists and admitting one is a separate decision. The response is counts plus the destination folderPath, never the extracted files: a large archive would otherwise materialize thousands of objects into one body. Callers page GET /api/v2/files?folderPath=... instead. The use case returns the internal display path and the adapter projects it to a v2 path, keeping the use case surface-neutral. * feat(v2): extract file text over the public API Adds GET /api/v2/files/{fileId}/text. Text extraction previously sat behind checkInternalAuth on /api/files/parse, a route that also mixes in external-URL fetching, execution-file upload, and multi-file aggregation, so it could not be reused. The parse call is lifted into a thin application use case instead. Runs on the existing files.read_content operation unchanged — it is already workspaceApiKey 'allow' at the read role, and turning bytes it already authorizes into text grants no further reach. `degraded` is a required, non-optional boolean on the response. The legacy doc and ppt parsers deliberately return best-effort or placeholder content rather than throwing, so an omittable flag would let a client that never checks it treat guessed text as extracted text. It is reported honestly rather than converted into an error, because the parsers' behaviour is deliberate and characterization-tested. The read is bounded on its input at 25 MiB before extraction rather than on its output after, given the parsers' documented DoS history; a caller may lower the ceiling but never raise it. * feat(v2): restore archived folders and list the archived set DELETE /api/v2/files/folders archives recursively, so a recursive delete was unrecoverable over the API: the archived files stayed visible through GET /api/v2/files?scope=archived, but nothing could rebuild the folder structure. Adds POST /api/v2/files/folders/restore, path-addressed like the rest of the v2 folder family, and a `scope` selector on the folder list so a caller can find the archived path to hand it. `scope` extends the files folder-list query rather than the shared v2ListFoldersQuerySchema: only workspace files have an archived folder set, so adding it to the shared schema would give tables, workflows, and knowledge a parameter they ignore. GET /api/v2/files/folders is a FULL_SET_LIST, not paged, so no cursor binding changes — list-pagination.test.ts passes unchanged. Restore resolves the archived folder from its path by scanning the archived set rather than walking the live tree, which by definition does not contain the folder being restored. The folder-restored analytics hook now reports the folder actually restored rather than the requested selector, which carries no id on a path-addressed surface. * feat(v2): bulk-download a file selection as a zip Adds GET /api/v2/files/bulk-download, an adapter over the existing downloadWorkspaceFileItems use case and its internal binary route. Path collision: a static segment beside [fileId] permanently shadows a file whose id equals it, and workspaceFileIdSchema does accept [A-Za-z0-9_-]+. Rather than invent a new shape, this follows the existing bulk-delete sibling: the hyphenated form cannot be produced by either minted id shape (UUID v4 or wf_<shortId>), so the shadowed id is unreachable in practice. Documented on the contract so the reasoning is not lost. Folders are addressed by path, matching the rest of the v2 file surface. The paths resolve against the folder set the selection already loads, so it costs no extra query, and a path matching no folder is rejected rather than silently dropped — a misspelled folder must not yield a zip of whatever else was selected. The empty-selection and folder-count guards now account for folderPaths, which a path-only selection would otherwise have tripped. Selections are comma-separated only: v2 rejects a query parameter sent more than once, so a repeated-parameter form would never reach the schema. Pinned by a test so the contract cannot advertise a form the boundary rejects. * feat(v2): expose run output files and optional inline bytes on the runs read GET /api/v2/workflows/{id}/runs/{runId} now reports the files a run produced, each with the downloadPath that fetches its bytes, and can inline them as base64 on request. Gated by includeOutput, matching `output`'s nullability: a caller that did not ask for output does not receive a file list it did not request. The async execute request's rejection of includeFileBase64 is deliberately left alone — at submit time the run has not happened, so there is nothing to inline; reading a finished run is the first moment the question means anything. Inlining is capped per file at the executor's 16 MiB inline ceiling, which a caller may lower but never raise. A file above it answers 413 naming that file's downloadPath, so the caller is told exactly how to get the bytes rather than being left stuck. The descriptor deliberately omits the storage key — files are addressed by id and the key is re-derived from the run's recording — and omits an expiry, which the recording does not carry and which would be fabricated if published. The route becomes headSafe: false, since inlining reads object storage. The builder enforces that this requires the use case to expose authorize(), so HEAD still answers from a real authorization rather than from authentication alone. * feat(v2): permanently delete an archived file DELETE /api/v2/files/{fileId} only archives — the OpenAPI says its stored bytes are never removed — so there was no way to actually destroy a file over the API. Adds the repository primitive, application use case, operation, and DELETE /api/v2/files/{fileId}/permanent. A distinct path rather than a flag on the ordinary delete: a query parameter that turns a recoverable archive into an irreversible destruction is set by accident, and the two acts carry different minimum roles, which one route declaration cannot express. The file must already be archived; a live file answers 409 naming the archive step, so no single request can turn a live file into lost bytes. minimumRole 'admin', which forces workspaceApiKey 'deny' since the workspace-key ceiling is 'write' — the desired policy anyway: unattended credentials should not destroy bytes. Row first, then object. The two legs commit independently, so one can survive a crash between them: deleting the row first leaves at most an orphaned object for the storage sweep, while the reverse would leave a live row pointing at bytes that no longer exist — a file that lists and opens but can never be read. A failed object delete is therefore reported as objectDeleted: false rather than thrown, because the request has genuinely succeeded once the row is gone. Both directions are pinned by failure-injection tests, verified to fail when the order is reversed. Audited as a distinct FILE_PERMANENTLY_DELETED action, not a reuse of FILE_DELETED, which records the recoverable archive step. * feat(api): v2 log analytics, itemized cost, filters, and sortable query Adds the aggregate and rich-read halves of the public logs surface, and fixes three defects the existing reads carry. Aggregate analytics. `GET /api/v2/logs/stats` returns time-bucketed run counts, success rate, error count, mean latency, and the window bounds, per workflow and for the workspace. The first-party route was a raw handler with inline SQL and inline aggregation, so it is split into a repository (`lib/logs/stats-queries.ts`), a pure aggregator (`lib/logs/stats.ts`), and an application use case. That route keeps its legacy authorization — it answers a caller without workspace access with a zeroed 200, where v2 conceals the workspace as a 404 — and consumes only the two surface-neutral halves. `segmentCount` had no `.int()`, `.min()`, or `.max()`, so `0` divided by zero and `1e9` allocated two billion-element arrays: both caller-reachable 500s. Bounded on both contracts. `workflows` is capped, with the workspace totals still computed from every workflow and the cut reported as `workflowsTruncated`. Detail reads gain the itemized `cost.items` ledger (`null` and `[]` are distinct answers and both reachable) and `workflowInput`, restoring a v1→v2 regression. The list gains `workflowName` and `status` filters, and `includeJobRuns`, which unions Chat and Sim-agent job runs into the sequence behind a new `kind` discriminator — without it a job run is indistinguishable from a run whose workflow was deleted. A filter no job row can answer drops the branch outright rather than meaning two things across the union. `POST /api/v2/logs/query` carries the additional sort columns. `GET /logs` is untouched: its single `order` param rests on there being exactly one sortable column, and both escapes from that are ruled out, so the rich read gets its own endpoint — the split the table surface already ships. It uses the shared keyset scheme with the two nullable sort columns read through a sentinel, since a keyset cannot compare against null. `folderPaths` now covers a folder's whole subtree on the public path, as it already did everywhere else; it previously omitted every nested run with no error. The path strings did not change, so a folder-scope version is stamped into the cursor and in-flight tokens restart rather than silently skipping rows. Also fixes `folderName`, which ILIKEd `workflow.name` — a copy of the clause above it — and so searched workflow names instead of folders. `buildLogSortCursorCondition`'s `IS NULL` disjunct is documented and pinned: under `NULLS LAST` the null block is only reachable through it, so removing it as a duplicate-row fix makes those runs unpageable. Ratchets: route count 1142 -> 1144; logs OpenAPI operations 2 -> 4; total operations 152 -> 154. * feat(api): v2 tables run state, dispatch polling, batch update, bulk, archive Closes the headless gaps on the v2 tables surface. - Per-cell run state is now readable through an opt-in `includeRunState` on `GET /rows`, `POST /query`, and `GET /rows/{rowId}`. The default projection is byte-identical; a page whose sidecar outgrows its byte budget is a 413 rather than a silent truncation. - Run dispatches are addressable: `GET /tables/dispatches/{dispatchId}` publishes the column's full four-state domain so polling a finished run is not a 500, and `GET /tables/{tableId}/dispatches` lists what is in flight. - `POST /rows/batch-update` takes one distinct patch per row. Its transaction moved out of the Copilot-only module into a surface-neutral use case both surfaces now call. - `GET .../enrichment/{groupId}` publishes the provider cascade, cost, and timing behind one enrichment cell. - `POST /tables/bulk-move` and `/bulk-delete` reach the existing bulk use cases, which now accept folders by canonical path and resolve them inside the application layer. - `DELETE` is recoverable: `scope=archived` on the table list plus `POST /tables/{tableId}/restore`. * feat(api): expose knowledge chunks, tag writes, archive/restore on v2 Closes the knowledge cluster's remaining public-surface gaps. Chunks: list/read/create/update/delete/bulk under `/api/v2/knowledge/{id}/documents/{documentId}/chunks`. `queryChunks` gains an `id` tiebreaker on every sort so the list pages on a keyset rather than an offset — `tokenCount` and `enabled` are both non-unique, so a page boundary inside a run of equal values used to repeat or drop the tied rows. The internal offset caller is unchanged; the two positioning schemes share one read. Tag definitions: create, update, delete, next-slot, usage, and the document-scoped save and cleanup. Without them a caller could write a tag value into a slot with no definition and then had no way to name it, so tag-filtered retrieval was unbuildable end-to-end. `v2KnowledgeTagSchema` gains `id`, without which PATCH and DELETE are unaddressable. The document-scoped DELETE is pinned to `action: 'cleanup'`: the domain's `'all'` deletes the whole knowledge base's tag vocabulary from a document path. Archive/restore: `GET /api/v2/knowledge/archived` as a sibling route rather than a `scope` param — the two reads bind different operations and a v2 route declares one — plus `POST /api/v2/knowledge/{id}/restore`. `knowledge.restore` is a new workspace operation carrying `delete`'s policy, since an operation's inverse must not be harder to reach; the internal session route now delegates its workspace branch to the shared use case and keeps only the legacy personal one. Also: `POST .../documents/from-workspace-files` surfaces `addWorkspaceFiles`, so a file already in workspace storage no longer has to be re-uploaded byte-for-byte to be indexed; the `chunkingConfig` write widens to the first-party five-key schema with its refines and separator bounds, while the response stays `.catchall` so a legacy JSONB row cannot 500; and `CONNECTOR_MANAGED_RESOURCE_READ_ONLY` joins `FORBIDDEN_DETAIL_CODES` now that the bare 403 on connector-managed chunk writes is wire-reachable. Document upsert is deliberately not included. * feat(api): add v2 credential rotation and a gate-exempt capabilities endpoint PATCH /api/v2/credentials/{credentialId} rotates service-account secret material or renames a credential in place, preserving the credential id so existing workflow, deployment, paused-run, connector, and webhook references keep working. Re-posting to POST /api/v2/credentials answers 409, and delete-and-recreate mints a new id, so rotation previously had no door. The route is adapter-only: updateWorkspaceCredentialUseCase already owned the rotation, its audit projection, and credentials.update. It gains one additive assertedWorkspaceId field for the v2 workspace assertion, and the per-principal credential-type table that deleteCredentialUseCase already applied is lifted into requireManageableCredentialType so both operations share it. Without it a personal API key could rename an env_workspace row and toV2Credential's throw would surface as a caller-reachable 500. CredentialProviderOperationError now maps to 503 with Retry-After when the provider is unreachable, instead of the 400 its OrchestrationError('validation') base projected. A transient outage rendered as a permanent input error invites a caller to revoke a working credential. GET /api/v2/meta reports the calling key's rollout cohort, type, and expiry. It is the one route declaring the new typed gate: 'exempt' option, because the rollout gate and the unknown-path catch-all answer byte-identical 404s and a gated /api/v2/meta could never resolve that ambiguity. Authentication still runs first, so the only fact disclosed is one about the caller's own credential. * feat(api): publish deployment lifecycle and workflow-MCP v2 surfaces Adds the four deployment-lifecycle operations v2 was missing, and the workflow-as-MCP publishing surface, both as adapters over application use cases that already existed. Deployment lifecycle: - PATCH /api/v2/workflows/{id}/versions/{version} relabels a version. Deliberately not the internal route's body-shape dispatch between "rename" and "promote to live". - POST .../versions/{version}/activate promotes a version. Same use case as rollback under a different transition, on its own path because the two mean opposite things to a caller. - POST .../versions/{version}/revert overwrites the draft. Accepts the literal `active` alongside a version number. - PATCH /api/v2/workflows/{id}/deployment toggles unauthenticated public execution. `workflows.public_api.update` widens from session-only to session plus personal API key: it is an admin-role change the same accountable human may make from a script. Workspace keys stay denied. Its EE refusal now carries PUBLIC_SHARING_NOT_ALLOWED instead of a bare forbidden. Workflow MCP servers: - /api/v2/workflow-mcp-servers list, create, update, delete, plus publish and unpublish of a workflow as a tool. Named apart from /api/v2/mcp-servers, which registers the external servers Sim calls. - The six mcp_servers.workflow_deployments operations widen from ['delegated'] to admit sessions and personal API keys; roles and the workspace-key denial are unchanged. - The server list gains keyset pagination, matching its external sibling, since nothing caps how many a workspace publishes. - Server, tool, and workflow reads move out of the use case into lib/mcp/queries. Route ratchet 1150 -> 1160; OpenAPI operations 161 -> 171. * feat(api): extract chat deployments and publish the v2 surface Chat deployment was a shipped module with no public API and two authorization systems: `lib/workflows/application/chat-deployments.ts` had deploy/undeploy extracted, but only Copilot used them — the REST routes reimplemented workflow authorization inline, and `PATCH /api/chat/manage/[id]` additionally owned password encryption, the auth-type field-clearing matrix, identifier uniqueness, the redeploy-gating protocol with two 409s, a raw db.update, and a manual recordAudit. New `lib/chat-deployments` domain: - `chat_deployments.list/read/update/delete`, keyed on the deployment whose workspace is derived by joining its workflow. Creation stays `workflows.chat.deploy`, which is keyed on the workflow. - The PATCH extraction, including the field-clearing matrix and the asynchronous-cutover invariant the route had hand-mirrored from `performChatDeploy`. - One `buildChatDeploymentUrl`, replacing three constructions that had already drifted onto two different host helpers. There is no chat subdomain, so nothing publishes a host. - Repository reads moved out of the use cases into `lib/chat-deployments/queries`. Internal routes are now adapters over those use cases. `GET /api/chat` is deliberately not migrated: it scopes by `chat.userId` while every other chat operation authorizes by workspace admin, and reconciling the two is a product decision. `PATCH` keeps its 400 for an identifier collision through a typed `ChatIdentifierInUseError`; v2 reports the 409 the condition actually is. v2 surface at `/api/v2/chat-deployments`: list, create, read, update, delete. Workspace-scoped, keyset-paged, and a stored password is never readable — reads carry `hasPassword` only, and the session-only reveal endpoint deliberately has no v2 counterpart. Also: an email- or SSO-gated chat with an empty allow-list is now refused in the use case rather than only at the internal boundary, since it is unenterable; and the doc comment on `processHostedKeyCost` claiming a `usageLog` write is corrected — no such write exists. Route ratchet 1160 -> 1165; OpenAPI operations 171 -> 176. * fix(api): close three review findings, two of them caller-reachable - Run output files are filtered to keys under the run's own execution prefix. The recording they came from is not a trustworthy key source: the start block copies every caller-supplied input field verbatim into its output and `collectUserFilesById` accepts anything carrying the `UserFile` shape, so a caller could name any storage key and have the download and base64 paths — neither of which authorizes per file — serve it back. - `getBlock` reads own keys only. `BLOCK_REGISTRY` is an object literal, so `constructor`, `toString` and friends returned inherited functions that every consumer then treated as a block, turning a path segment into a 500. `getToolMetadata` already guarded this way. - A folder-scoped log page no longer unions in every job run in the workspace. The guard read `filters.folderIds`, which the public surface never sets — it carries the folder filter in `folderScope` — so the page contradicted the contract's promise that job runs are dropped whenever a filter they cannot answer is set. Also: the log cursor stamps `includeJobRuns` only when it is on, so its `.default(false)` no longer puts a constant in every fingerprint and rejects cursors minted before it existed; and the `folderName` subquery is scoped to the workspace and to workflow folders instead of scanning the whole `folder` table. * fix(api): close two more review findings, one an authorization bypass - `workflows.operations.apply` no longer admits a workspace API key. The use case authorizes against three per-user policies — the EE permission config, block visibility, and credential reachability — and all three take a human subject. An actorless key has none, and both substitutes fail open: attributing to the workspace billing owner evaluates the batch as the least-restricted account in the workspace, and passing no user makes `getUserPermissionConfig` return `null`, which every caller reads as unrestricted. Either way a workspace constrained by an allowlist was edited as though it were not. Personal keys keep the capability, so headless editing is unaffected for a credential that names a human. - `GET /workflows/{id}/state` reads its variables through `parseWorkflowVariables`, and the stored variable response schema drops the two assertions the column cannot honour. The column has carried a JSON string and a legacy array as well as the current record, the realtime `variable.add` op types `type` as `z.any()`, and the parser writes `name` through verbatim — so the input bounds on the read turned a stored workflow into a 500 on the endpoint that opens it. The write schema keeps them, which is where they can still be honoured. - `GET /workflows?scope=archived` projects folder paths tolerantly. Archiving a folder cascades onto the workflows inside it but leaves their `folderId` dangling — which is why restore has to null it — so the strict projector threw a bare `Error` and took the whole page down with no cursor able to step past the row. * fix(files): bind Start-block file keys to the executing workspace The Start block derived a file's storage key by parsing the caller's own `url`, which `isInternalFileUrl` matches on any host and `extractStorageKey` returns verbatim — so a request body could name any tenant's bytes. The key is now accepted only when its own layout names the workspace the execution runs in, and every file is dropped when the execution carries no workspace. Also bounds `includeFileBase64` with an aggregate response ceiling and a worker pool instead of an unbounded `Promise.all`, scopes the bulk download's authorization resource to the workspace when folder paths are requested, makes the folder-restore selector mutually exclusive at the type level, and names the bound in the `maxBytes` validation message. * fix(api): close v2 log review findings Cursor scope: `scope` on the workflow and table lists carries `.default('active')`, so it entered every fingerprint as a constant and refused every cursor minted before the param existed — with the "cursor does not match the requested filters" 400, which is actively misleading for a caller that changed nothing. Both now stamp the default as absent, so only a caller who asked for `archived` gets a new sequence. Dashboard stats: `maxWorkflows` capped the response, not the allocation. Segment series are now densified after the cut instead of before, so returning 200 series no longer materializes one `segmentCount`-length array per workflow in the window. The aggregate still sums every workflow, now from the sparse per-workflow maps. Cost keyset: `cost_total` is an unconstrained `numeric`, so its anchor travelled through `Number()` and was compared back at full precision — rows differing beyond float64 collapsed onto one anchor. Adds `decimalKey`, which carries the digit string and binds it `::numeric`. Run detail: `cost_total` is a backfilled projection, so a run predating the backfill reported `cost: null` even with a real ledger, making `items` unreachable for exactly the runs the ledger explains. Falls back to the ledger total. Also caps the log folder-path index reads at MAX_FOLDERS_PER_WORKSPACE like every other reader, publishing the folder-tree 413 on the four log operations; reverts a dead `status` widening in `v2CommaListSchema`; drops an unread `executionData` select; corrects the segment-count and searchLogs prose; and replaces the sort-cursor SQL-text assertions with a two-page walk over a fixture with a null block. * fix(api): close knowledge v2 review findings - widen knowledge.list_archived to the delete/restore policy so a workspace API key can discover what it may restore - escape LIKE wildcards on the now-public chunk search - derive tag slot capacity from TAG_SLOT_CONFIG per field type - type updateKnowledgeBase's chunkingConfig as ChunkingConfig and project every declared field explicitly - attribute a restore to the calling surface instead of a literal 'api' - gate 'knowledge chunks batch-update' behind --yes, since it can delete - present the tag-cleanup action from the parsed request rather than faulting on the domain result after the delete committed - unbind asserted-scope workspaceId from the nested knowledge cursors, matching the table-row lists - add executed-SQL coverage for the chunk keyset * fix(catalog): close the catalog and registry-boundary review findings The module-graph ratchet treated an entry with no baseline row as informational, so the six catalog routes and the projection barrel were unratcheted while the summary still read "within their module-count baseline". An unbaselined entry now fails --check, the summary counts only what was actually compared, and the baseline is re-recorded. The Copilot block-metadata tool — the reason the shared projection exists, 6,756 modules down to 1,321 — was in no guarded subtree. It is now an entry source and a catalog boundary root. Catalog behaviour: - hostedApiKey is gated on the deployment, so a self-hosted install reports none instead of promising 127 tools' keys it will never supply - block detail resolves an unversioned base type to its newest version and projects through the viewer's visibility, so it can no longer 404 a block the list contains or name it differently - offset-cursor ordering compares code units rather than the process locale - projections copy every array they publish instead of handing out the registries' own - an options function returning a thenable throws rather than silently widening the providers-store substitution across the event loop - a throwing block projection costs the Copilot tool one block, not all of them - the trigger-kind log returns to debug: chat/manual/api are entry-point kinds, not authoring defects Also sweeps the custom-block detail branch against its response schema, guards each projection module rather than the dead barrel over them, and drops a provably dead branch in processHostedKeyCost. * fix(workflows): close v2 workflow-authoring review findings - Read a blockless draft back as an empty graph. `PUT /state` of `{ blocks: {}, edges: [] }` — the contract's own published example — deletes every block row, and the loader answers `null` for a blockless workflow, so the following `GET /state` answered 404 while the list endpoint still showed the workflow. Existence is the workflow row's to decide; the null is now projected as an empty graph. - Rewrite the `readWorkflowGraph` authorization test so it can fail. It called `authorize?.()` and asserted only a negative, so deleting `authorize` or replacing it with a no-op both passed — the invariant the head-safe `HEAD` path depends on. - Route `setWorkflowBlockEnabled` through `replaceWorkflowNormalizedState`, the same door the other two graph writes use, instead of writing the normalized tables itself without state preparation or custom-tool extraction. - Count applied operations directly. Enablement refusals landed in the same skipped-item array and were subtracted from the operation count, which `Math.max(applied, 0)` then masked when it went negative. - Give a `disabled_ancestor` refusal its own member of the published skip enum instead of reporting it as `block_locked`. - Refuse an `atomic` batch whose credential or hosted API key would be stripped, and carry the dropped inputs in the 409 details. - Publish the whole lint report — `sources`, `sinks`, `orphanBlocks`, `emptyOutgoingPorts`, `invalidBranchPorts`, `invalidConnectionTargets`, `fieldIssues`, and the `kind` discriminator on unresolved references — rather than only free-text reference prose. - Stop reporting unresolved lint references as `inputValidationErrors`. `collectUnresolvedReferences` is read-only, so those values stay persisted; they were double-reported, and falsely as dropped inputs. - Replace two unfalsifiable negative-principal tests, which used a principal kind `Exclude`d from `PrincipalKind`, with a reachable one. - Nits: drop a stranded TSDoc block; assert the membership predicate in the selector-validator admin test; make the HEAD test assert a representation; assert the sanitized graph is what `replaceWorkflowState` writes; add a route test rejecting `baseGraph` in a v2 body; carry `principalAuditSource` on restore/duplicate/moveBulk audit; unify the two `base64MaxBytes` ceilings on `MAX_INLINE_MATERIALIZATION_BYTES`. * fix(api): close seven v2 review findings, one an authorization bypass Raise mcp_servers.workflow_deployments.update_server to admin: its body carries isPublic, and a public server executes with no Sim credential, so a write member could remove authentication from every workflow it publishes. create_server already grants the same visibility at admin. Pin the widened operations in registry tests — the six workflow-MCP ones, the four workflow widenings, and files.extract_archive. Reject secret fields on a credential that has no rotatable secret instead of dropping them behind a 200, classify a non-transient provider 4xx as caller error rather than a retryable outage, and reconcile a provider outage to 503 with Retry-After on all three surfaces. Give /v2/meta a declarative principal policy through a new defineOperation factory, carry the key expiry on the auth context instead of reading the api_key table from the application layer, and make the impossible principal branch an invariant error rather than a codeless 403. Enforce the rollout-gate exemption at definition time, against the one contract path it is reserved for, and remove the gate parameter from the exported admission helper so the builder is its only door. * fix(tables): bound the run-state sidecar, and close six v2 review findings Enforces the 2 MiB run-state ceiling INSIDE the sidecar drain rather than over its materialized result, refuses the unbounded query form paired with it, and normalizes the two stored blobs the v2 surface publishes from bare `as` casts. - The run-state budget now travels into `loadExecutionsByRow`, which drains row ids in bounded chunks and refuses before fetching the next one. The post-hoc `requireBoundedRunState` walk is gone: it measured a spike that had already happened, and re-serialized every entry to do it. - Both row reads that accept `includeRunState` cap the page at `V2_MAX_RUN_STATE_ROW_LIMIT`, and `POST /tables/{id}/query` additionally refuses the flag paired with `limit: 0`. - `runState.status` and the enrichment cascade blob are projected onto the published shape before presentation; both were caller-reachable 500s on a well-formed read. - `sim tables bulk-delete` now gates behind `--yes`, and the CLI sweep that should have caught it covers destructive non-DELETE forms. - `POST /tables/{id}/restore` is idempotent (200, no audit) like its knowledge sibling, and bulk folder selection deduplicates after resolution. - The batch-update backstop keeps the looser Copilot ceiling and says so in TSDoc: it is a backstop no surface reaches, because the contracts stop a v2 caller at 1000 and the Copilot tool stops itself at 5000. Each caller sees the bound that actually applies to it; neither surface's cap moved. * fix(chat-deployments): close v2 chat review findings Fixes the chat-deployments slice of the v2 review, several of which are regressions the application-operation extraction introduced. - Stop a `500` on schemaless JSONB: the response now declares a stored shape without bounds and `toV2ChatDeployment` projects `customizations`, `outputConfigs`, and `allowedEmails` onto it. The request schemas keep `.strict()` and their bounds. - Restore the specific validation message on `POST /api/chat` and `PATCH /api/chat/manage/[id]`, and the deleted test that pinned it. - Restore `chat_deployments.read` to workspace `admin`; the detail read serves the visitor gate. - Narrow the list projection so `chat_deployments.list` can stay a `read` operation reachable by a workspace API key: `allowedEmails`, `hasPassword`, and `customizations` are gone from the list entry and available only from the admin-gated detail read. Serialized field by field so a field added to the detail shape cannot reach the list by default. - Classify create-path failures: `performChatDeploy` carries an `errorCode`, so an in-flight deployment is a `409` and an invariant failure a `500` instead of every refusal being a `400`. - Delete the callerless `GET /api/chat`, which served the encrypted password column with no response contract. - Propagate undeploy infrastructure failures instead of concealing them as `404`, and return `ChatDeploymentView` from both delete paths. - Name `CHAT_AUTH_MODE_NOT_PERMITTED` on the create path. - Guard `getBaseUrl` inside `buildChatDeploymentUrl`, which otherwise throws on a self-host with no `NEXT_PUBLIC_APP_URL`. - Correct the published allow-list claim: a replacement `allowedEmails` is applied after the auth-type clear, so it does survive. - Assert `workspaceId` on the v2 detail routes and reconcile the concealment TSDoc with what the error policy actually renders. - Move `resolveActiveWorkspaceApplicationContext` to the workspaces domain so chat-deployments no longer imports workflow application code. * test(credentials): pin the reconciled provider-outage status The internal route alone answered 502 where the v2 surface, the shared status helper and `PROVIDER_OUTAGE_CODES`' own TSDoc all say 503. The test pinned the divergence; it now pins the reconciliation, including the `Retry-After` a 503 carries. Corrects a stale comment that still named 502 as the value callers see. * fix(executor): restore cloud-storage Start files, dropped by the key rule The ownership check accepted a key only when it could be parsed out of an internal `/api/files/serve/...` URL. But the server-side uploader for run inputs returns a *presigned cloud* URL whenever object storage is configured, whose path is the bucket key — so every chat-deployment attachment, API `files[]` payload and generic-webhook file field resolved no key, and because normalization is all-or-nothing the entire `files` input was dropped with no error. It passed locally and under vitest only because the uploader falls back to an internal URL when no object storage is configured, which is exactly why no test caught it. The test is ownership, not provenance: a key is accepted when its own layout names the executing workspace, whether it arrives directly or is parsed out of the URL. Neither field has to be trusted, since both are caller-authored and both are held to the same check. A payload whose key and URL disagree is refused rather than resolved in the caller's favour — a genuine uploader writes the two consistently, so only a forged pairing is turned away. `context` is now derived from the accepted key rather than read from the payload or the URL's `?context=`, so an owned key can no longer be labelled with a bucket its bytes do not live in — the hardening the previous comment claimed but did not perform. * fix(api): close four defects the fix pass introduced - `resolveLatest` built a `RegExp` from the caller's block id and read the registry with a bare lookup, so the catalog detail route — moved onto it by the version-alias fix — routed around the `ownBlock` guard added for exactly this. `GET /api/v2/blocks/%5B` was a `SyntaxError` 500 and `.../constructor` an inherited function. Matched by string comparison now, the way `tools/tool-ids.ts` resolves the same convention, and read through `ownBlock`. - The run-state byte budget was applied inside `queryRows` rather than at the callers that publish it, so the first-party table grid — which reads run state at five times the row limit and publishes no ceiling — turned a large page into a hard failure, with an error naming a parameter it does not expose. The budget is now an explicit option the public reads pass and internal callers omit. - Three graph-write CLI commands shipped ungated because the sweep meant to catch them matched only the names already enumerated, so it could never fail. It now forces every non-`GET` operation into a destructive or non-destructive list, and the three carry confirmations. - Unbinding `workspaceId` from the knowledge-documents cursor was right on the merits and wrong in effect: the value is constant per sequence, so removing it changed the fingerprint and refused every cursor already in flight. Restored there; the chunks list is new in the same change and keeps the cleaner reading. * fix(api): resolve a detail read to a version the viewer can see `getLatestBlockForViewer` took the newest version and then hid it, which inverted the contradiction it was written to close: `slack_v2` and `table_v2` are preview-gated while their v1 deliberately stays in the toolbar, so an unrevealed viewer got a `404` on a detail read for a type `GET /api/v2/blocks` was listing in the same breath. It now walks versions newest-first and answers with the first one visible to that viewer. Also: - The chat password guard ran after `performFullDeploy`, so a request that could never succeed burned a real workflow deployment version and then answered 400. Its two sibling gate guards already refuse ahead of the deploy; this one now does too. - The Copilot sub-block serializer published the registry's own `options` and `dependsOn` arrays by reference. Pre-existing, but the catalog projection this parallels copies every array it publishes precisely because they are process-global and shared by every request. * fix(api): restore the locked read-modify-write and the password validator Two findings verified as real regressions against staging, out of ten checked — the rest were pre-existing, latent, or false. `setWorkflowBlockEnabled` read the graph outside the row lock and wrote it back inside a later transaction. The editor's own save takes that same lock, so an autosave committing in the window was silently discarded: this operation writes a whole graph, not a delta. The persistence primitive now accepts a reader that runs after the lock is taken, and the toggle re-reads and re-decides there. Its lock predicate is also scoped to the workspace and to a live row again, so a workflow archived mid-flight is refused rather than written. The v2 chat-deployment contracts inlined their own password rule twice instead of using `chatDeploymentPasswordSchema`, losing the refusal of a whitespace-only password — which the internal contract rejects precisely because it strands the deployment behind a password the visitor form will not submit. Both sites use the canonical validator now. * fix(api): one folder projection, one dynamic-provider list, honest 413s - `toV2Folder` existed twice, and the second copy had been written without the name/path invariant — so a row the list read refuses loudly would have been served with a mismatched pair by the restore read. One definition, guard included. - The catalog projection restated `DYNAMIC_MODEL_PROVIDERS` and had drifted by one member. Derived from the canonical list instead. - The tables reads documented a `413` for run state that they cannot emit — the budget became opt-in, and the row limit is the bound now — so the claim is removed rather than declared. The workflow run read has the opposite problem: it genuinely emits one, on a single file *or* the run's inlined total, and declared neither. Now declared, and the sentence covers both. - Reclassifies the operations staging added into the destructive sweep, so the triage stays exhaustive. * fix(v2): classify storage and uniqueness failures, drop permanent file delete - remove the permanent file-delete endpoint; the platform offers no such action in the UI, and its manager wrote outside a transaction with no storage accounting - extract a generated document's text from its compiled artifact rather than its generation source, matching the download path; a `.pdf` source was a 500 and a `.docx` source returned generator JavaScript as clean content - report a run file whose object retention has already swept as 404 rather than 500, on both the inline base64 read and the download stream - report a knowledge tag that loses at a unique index as 409, naming whether the slot or the display name is taken - gate `workflows versions revert` behind a CLI confirm; it overwrites the draft graph and was classified non-destructive * feat(v2): report lint from both graph writes and add dry-run previews - `PUT /workflows/{id}/state` now returns the same `lint` report as `POST /operations`; an agent authoring a graph from scratch needs the findings at least as much as one editing incrementally - extract the report into one shared builder so the two writes cannot drift, and one shared presenter so the wire shape is identical - skip the credential/tool reference pass when the caller has no human subject, rather than resolving it against the workspace billing owner: that would misreport what the workflow can reach and disclose another person's grants. `lint.notes` says when it was skipped - add `?dryRun=true` to both graph writes: validates and lints, persists nothing, records no audit, notifies nobody. A query param, not a body field, since the body of a PUT is the resource itself - CLI: a dry run no longer demands `--yes`; requiring confirmation to preview a change teaches callers to pass `--yes` reflexively - CLI: name the graph commands for their verbs — `workflows state get`, `workflows state replace`, `workflows operations apply` — instead of the derived `state list` / `state update` / `operations create` - document when to use `rollback` vs `versions/{version}/activate` on both * chore(docs): sync generated docs manifest for the new CLI pages * feat(v2): add the missing workflow-MCP reads and align bulk naming - add `GET /workflow-mcp-servers/{serverId}` and `GET /workflow-mcp-servers/{serverId}/tools`. The resource could be PATCHed and DELETEd but never read, and its tools could be published and unpublished but never listed — the server list reports tool names only, so nothing published the `workflowId` that addresses a tool for deletion. Both mirror `mcp-servers` beside them, and carry that family's workspace-API-key denial rather than the wider `mcp_servers.read` policy - rename `POST /tables/bulk-move` to `POST /tables/move`, so tables matches the shipped `files` resource exactly (`move` + `bulk-delete`) - name the CLI commands for their operations instead of the derived `... create`: `tables move`, `workflows move`, `tables bulk-delete`, and `tables rows update-each` for the per-row batch, which sits beside the existing filter-based `tables rows batch-update` `POST /tables/{id}/rows/batch-update` keeps its name: a distinct payload per resource is precisely AIP-234 BatchUpdate, and `bulk-` would have collided one word away from the filter form. * fix(v2): correct documented statuses and a caller-reachable 500 - duplicating a workflow into a locked destination folder answered 500: `FolderLockedError` is a plain Error carrying `status = 423`, which the v2 error policy does not classify. Converted to OrchestrationError('locked') at the application boundary, matching the bulk-move path - restore workflow promised a 413 for an oversized folder tree that its response list never published; the cap is real, so the status now is too - move workflows and apply variables documented 409/423 they cannot emit: every per-item lock and conflict is reported in `failed`, not thrown - bulk download and delete knowledge tag can both 409 and did not say so; cleanup tag definitions cannot and did say so - apply workflow operations denies workspace API keys but never documented it - the dry-run responses are not byte-identical to a committed write: `needsRedeployment` describes the pre-write state and persistence warnings cannot appear. Reworded rather than overclaimed - read file text and get file upload are head-safe, so the "HEAD skips the effect" sentence did not apply to them * fix(v2): guard tag field-type changes and publish the 415 every body route can return - `PATCH /knowledge/{id}/tags/{tagId}` accepted a `fieldType` incompatible with the slot the tag already occupies. Slots are enumerated per field type, so a text tag could be relabelled `number` and every later read would interpret its values as the wrong type. Create checked this; update now runs the same two checks - derive `415` from the contract the way `413` already is: the JSON builder answers UNSUPPORTED_MEDIA_TYPE for any body under a content type it cannot read, so all 100 body routes could return a status none of them published - give version activation its own result component instead of publishing it as `RollbackResult`; the shipped rollback keeps that name - correct descriptions that promised behaviour the code does not have: a `processingStatus` field never returned, a `gmail_send` resolution example that short-circuits, bucket widths that overflow the window, a bulk tag save that relocates rather than overwrites, and per-server tool names actually gathered under a page-wide budget - drop 409 from three knowledge and upload reads that cannot emit it * docs(v2): correct the upload transfer contract and 16 other published claims The upload transfer step was documented as Sim's own data plane on every deployment: "success is 204" and "a failure is the v2 error envelope". That holds only when Sim stores objects itself. With object storage configured the URL is the provider's presigned URL, so S3 and GCS answer 200 and Azure 201, and a failure is the provider's XML — a client written to the old text reads a successful cloud upload as a failure. Also states that part ETags do not need retaining: completion takes no body because Sim lists the parts from the provider itself. Other corrections, all to shipped descriptions rather than behaviour: - DELETE table and bulk-delete files archive rather than erase, and neither said so; bulk delete also cannot emit the 409 it declared - complete knowledge upload published a 402 only the create leg can raise - billing status conceals a foreign workspace id as 404, not the 403 its TSDoc and description both claimed - audit entries null a folder's resourceId and strip folder ids from metadata at every level; neither redaction was documented - details=full adds the workflow summary to workflow runs only, never to job runs; GET /logs folderPaths covers a subtree like its two siblings; getLog now carries the retention sentence - list secrets returns description too, and the logs and resources documents described only part of what they serve * improvement(api): consolidate the v2 surface and close seven defects Endpoint consolidation: - Fold POST /logs/query into GET /logs; add sortBy/sortOrder, cap the comma lists, and move the list onto the shared keyset codec - Fold GET /knowledge/archived into GET /knowledge?scope=archived, matching files, tables, and workflows - Re-home chat deployments as a singleton under the workflow they belong to; keep the workspace-scoped discovery list - Move the tag-definition writes off the document path onto /knowledge/{id}/tags, where they already acted - Nest the table export and dispatch reads under their parent table Defects: - Publish isPublicApi on the deployment read; it was write-only, so a workflow could be opened to unauthenticated execution unauditably - Stop publishing raw storage keys and an unusable URL in log files - Classify a chat-identifier unique violation as 409 rather than 500 - Fall back to the root path instead of throwing when a knowledge base's folder is archived - Cap bulk-download at the ceiling it actually enforces - Normalize variables through one helper on both graph write paths - Fix a folder-name log filter that matched workflow names Naming and gaps: - Rename /files/{id}/extract to /unarchive, /rows/find to /rows/search, /rows/batch-update to /rows/bulk-update, /columns/run to /dispatches - Type the last six generic [id] path segments - Add table folder restore and id-addressed dispatch cancel * chore(audits): record the v2 catalog routes in the boundary baseline * fix(api): accept a null chat password and correct three published claims - performChatDeploy validated `password: null` as a password, so the replace-shaped chat PUT answered 400 for every mode that owns no password — public (the default), email, and sso. The declared payload type has always allowed null, and the stored value is cleared by authType regardless, so null needs no validation of its own. The route test could not catch it: it mocks the orchestration module and pinned the exact null the real guard refused. - Redirect the two docs slugs this branch retired that were genuinely published: findTableRows and runTableColumns. - The table folder restore described an idempotent no-op for an already active folder; it answers 404. Say so, and say where the path comes from, since the tables folder list cannot yet report archived folders. - Name the customizations exception to the chat PUT's replace semantics. - A cursor-binding case used status=error, which is a level and not a status, so it failed contract validation and never reached the cursor check. Use an accepted value and pin the reason, not just the status. * chore(cli): classify the new v2 chat operation as non-destructive * feat(cli): expose canonical resource URLs * fix(api): close three caller-reachable failures found by the final probe - GET /files/{fileId}/text called parseBuffer unguarded, and parseBuffer signals every failure as a bare Error that no v2 policy classifies. A zero-byte upload or a mislabelled archive was an unhandled 500. Empty bytes now answer empty text — a zero-length file has no text — and unparseable bytes answer 409, matching the rendered-artifact resolver. - GET /workflows/{workflowId}/state asserted write-side bounds over stored data. workflow_blocks.name and .type are bare text() and the realtime rename op accepts z.string(), so a block renamed past 255 characters made the workflow unreadable, and unrepairable, over v2. The read shape now takes the same input/stored split the variable schema already had. Stored subflow conditions are coerced in the loader beside the existing numeric guards. - GET /knowledge stamped scope into the cursor fingerprint unconditionally. scope defaults to active and is new on that list, so every cursor the deployed build handed out would have been refused with a message saying the caller changed a filter they never sent. Its siblings already carry the guard and the comment. Also: POST /workflow-mcp-servers answers 201 like every other v2 create; GET /logs/stats reuses the log list's entry ceilings; the execute and resume routes install the media-type-aware 415 they publish; and a rationale citing an endpoint that never reached the wire is corrected. * fix(tables): carry the dispatch terminal timestamps through the stale sweep Staging's abandoned-dispatch recovery builds its own `DispatchRow`, and this branch had added `completedAt`/`cancelledAt` to that shape for the id-addressed dispatch read and cancel. The merge was textually clean and left the new mapping short two fields. * fix(workflows): deny workspace API keys on the graph replace `PUT /workflows/{workflowId}/state` stores blocks and their tool wiring wholesale, but the policies deciding which of those a member may add — the EE permission config and block visibility — take a human subject. A workspace API key has none, and both substitutes fail open: the billing owner is a different, typically less-constrained person, and passing no user makes the permission lookup return null, which every caller reads as unrestricted. That made the replace a second graph-write door storing what its sibling `POST /workflows/{workflowId}/operations` refuses, which denies workspace keys for exactly this reason. Both doors now agree. Personal keys keep the capability, so headless authoring is unaffected for a credential that names a human. * chore(api): drop the enrichment catalog endpoint GET /api/v2/enrichments listed the code-defined table enrichments. The per-row enrichment run detail stays; only the catalog read goes. Removes the route, contract, response and query schemas, the semantic operation, the use case, the projection module, its registry-boundary entry, and the CLI command. The "not found" and "blank search" cases it covered are repointed at the connector-type sibling so the shared behaviour stays tested rather than deleted with it. * improvement(api): make the workflow operations endpoint self-describing Two things stood between this endpoint and a caller who has only the published spec. The accepted `params` keys existed only in the edit engine's source. The spec said "the accepted keys depend on the target block type", so a caller reading it could create a nameless empty block and nothing more — while the Copilot tool catalog, over the same engine, has always spelled out the envelope. That guidance now lives in the contract, shared by the add, edit, and insert_into_subflow parameter schemas so the two surfaces cannot describe one engine differently: `inputs` keyed by sub-block id, `retry`/`triggerMode`/`advancedMode` beside it rather than inside it, `connections` keyed by source handle, and `removeEdges` for dropping one edge without restating the rest. A `block_id` that is not already a UUID is replaced with a minted one, and the mapping was computed and then dropped. A caller could not reference the block it had just created except by re-reading the graph and matching on name. The engine now returns it and the response publishes it as `mintedBlockIds`, with the in-batch versus cross-request rule stated in the operation description. * fix(api): close the pre-merge scan's blocking findings Docs and public wire, all of it permanent surface once released. - Two published tag groups, Catalog and Meta, had no sidebar entry in any locale, so six operations shipped unbrowsable. Added to all six, and a test now fails when a published tag has no entry. - deleteWorkflowChatDeployment pointed callers at DELETE on /deployment; the undeploy verb is on /deploy. - PUT /state still promised workspace API keys a degraded lint pass after the operation began rejecting them outright. It also lived in a single-quoted string, so the shared clause would not have interpolated. - POST /tables/move took targetFolderPath as nullable-but-required, which rendered the CLI flag as a required `<json|@file>`: `--to /Archive` failed to parse and omitting it failed outright, while the docs said "omit for root". Now optional and a plain string, matching POST /files/move; the route supplies the null the use case wants. - Table dispatch status and row run state published `cancelled` beside imports, exports, and job state publishing `canceled`, and the note explaining the split was wrong about its own sibling. Both new schemas now publish `canceled`; the stored column is unchanged and mapped at the presenter. The shipped `cancelled` count field is left alone. - applyWorkflowVariables can answer 423 and both workflow-MCP deletes can answer 409; none declared it. Hand-assembled error lists replaced with the shared sets. - /files/{fileId}/unarchive became /unzip. `extract` reads as "extract text" and `unarchive` reads as the inverse of restore on a resource where archived means soft-deleted; unzip collides with neither and is what the implementation calls itself. * improvement(cli,docs): finish the naming and clear the enrichment leftovers - Four single-record GETs derived to `list` while returning one thing: meta, a workflow's chat deployment, log stats, and file text. Renamed to `meta status`, `workflows chat status`, `logs stats`, `files read`, matching the `workflows deployment status` correction already in this branch. None of the old spellings shipped. - Dropped five `renamedFrom` aliases pointing at spellings that never existed, each of which built a hidden command and a permanent deprecation warning for argv nobody could have typed. `tables rows find` keeps its alias — that one really shipped. - Removed the enrichment catalog from three prose sites left behind when the endpoint went: the resources spec description, its Catalog tag, and the contract and pagination-test comments. - Documented the six new command groups in the CLI index table. That page is a guide page, so the docs staleness check cannot flag it. - Published the `customizations` exception to the chat replace semantics. It was in the route TSDoc and invisible to every caller reading the spec, which is where the claim "Replace, not merge" is made. * improvement(cli): use batch- for the tables bulk delete, matching its siblings The CLI renames a bulk form only when it would collide with its singular sibling, and uses AWS's `batch-` prefix when it does — `files batch-delete`, `tables rows batch-delete`, `knowledge chunks batch-update`. `tables delete` exists, so `tables bulk-delete` was that same rename reaching for the other word, and the only `bulk-` command on the surface. No `bulk-` CLI command has ever shipped, so this costs nothing now and would be permanent later. `files bulk-download` keeps its name: there is no `files download` to collide with, and it is one archive rather than N operations. Its config block now says why it exists at all, since the command is never built — the builder skips non-JSON response modes, but the contract sweeps still read the entry and require the folder-path field to be marked. * fix(workflows): allow operations on blockless drafts * feat(workflows): add manual and run-from-block execution * fix(cli): update workflow run description test * fix(tests): align fixtures with current contracts * fix(api): derive chat activity from workflow deployment --------- Co-authored-by: Theodore Li <theo@sim.ai>	2026-08-25 04:19:34 -04:00
Vikhyath Mondreti	0a5b3801ea	feat(secrets): let workspace secrets opt out of redaction (#7045) ... * feat(secrets): let workspace secrets opt out of redaction * fix(secrets): certify no sandbox exemptions once the registry is incomplete * feat(secrets): carry visible secret values on the v2 list and document visibility * fix(secrets): read visible values by own property so prototype-named secrets cannot poison the list	2026-08-24 13:46:57 -07:00
Waleed	edf07ec3cd	fix(vllm): support LM Studio endpoints (#7036) ... * fix(vllm): support LM Studio endpoints * fix(vllm): validate compatible base URLs * fix(vllm): guard discovery URL validation	2026-08-24 10:58:07 -07:00
Vikhyath MondretiandClaude Opus 5	81ff24a2fc	improvement(secrets): gate Copilot code mounting at use level (#7004) ... * improvement(secrets): gate Copilot code mounting at use level Mounting a saved secret into Copilot code required credential-admin on that key, while a workflow Function block resolves the same secret for the same person at use level through getPersonalAndWorkspaceEnv. Copilot reaches that path itself — edit_workflow plus run_workflow — so the admin bar contained nothing. It redirected a Credential Member through a detour that mutates a persisted workflow, while the direct path is ephemeral and files a usage row. The inconsistency was also internal to Copilot: the secret names advertised to the model come from getAccessibleEnvCredentials and getPersonalAndWorkspaceEnv, both role-agnostic, so Copilot listed every secret the caller could use and then refused to mount all but the admin ones. Widen the workspace and shared-personal predicates to any active grant, and drop the matching role filter from the query. Workspace write is still required, revoked and pending grants are still refused, and a caller with no grant still gets nothing. The view gate stays where Copilot cannot route around it: values remain masked under Settings, and See usage remains admin-only, so a member's use is recorded for whoever can rotate the key. Model-egress projection is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(secrets): stop implying Personal secrets are shareable The Copilot code-execution paragraph listed "any secret shared with you as a Credential Member or Credential Admin" among what mounts, which reads as though a Personal secret can be shared. It cannot through any product surface: CredentialMembersSection renders only for workspace secrets and OAuth credentials, and the personal-credential sync only ever grants the owner. Narrow the sentence to Workspace grants. The comparison table's "Only you can use" row for Personal was correct and is left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-22 18:33:04 -07:00
Theodore Li	8104c33bba	Fix knowledge connector sync follow-up (#6927) ... * Fix knowledge connector sync follow-up * Fix connector sync pause race * fix(knowledge): surface connector sync dispatch failures * fix(knowledge): make connector sync recovery durable * fix(knowledge): deduplicate connector sync dispatches * fix(knowledge): preserve pending connector syncs * fix(knowledge): lock connector sync snapshot	2026-08-22 20:50:48 -04:00
3 people	c26529a82e	feat(bitbucket): add repository webhook triggers (#6934) ... * feat(bitbucket): add repository webhook triggers * fix(bitbucket): harden webhook trigger delivery * fix(bitbucket): address final trigger review * chore(bitbucket): address review conventions * fix(bitbucket): harden triggers and connector sync --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-08-22 15:23:35 -07:00
Vikhyath MondretiandClaude Opus 5	71129cd112	feat(custom-blocks): let a publisher decide whether their block's runs reach consumer traces (#6950) ... * feat(custom-blocks): let a publisher decide whether their block's runs reach consumer traces Joining a custom block's child run into its caller's trace shipped on by default, gated at read time by whether the person reading could already open the source workspace. That gate is doing the wrong job: a custom block's whole point is that consumers need no access to the source, so the check refuses exactly the readers the feature exists for, and it makes the answer depend on who is looking rather than on what the block's owner agreed to publish. The decision moves to the party whose data it is. `custom_block.trace_child_runs` is set by the publisher in Settings, applies org-wide, and is the entire policy — nothing downstream re-checks a caller. `getCustomBlockAuthority` already resolves per invocation and is the one lookup both the canvas handler and the Agent-tool runner pass through, so one column covers both surfaces and no consumer input can assert it. It defaults to FALSE. With the viewer check gone, an opted-in block publishes the source workflow's block names, inputs, outputs, and prompts to anyone who can read a consuming workflow's log. That is the same boundary curated outputs and redacted errors hold, so it opens by an affirmative act of the publisher or not at all — never as the residue of a column default on rows nobody revisited. Closed means the handle is withheld outright rather than persisted behind a flag: with no `childExecutionId` there is nothing for a reader, a migration, or a later refactor to join. What replaces it is a `_childTraceDisabled` marker, because a boundary span with no children renders exactly like a leaf block and an untraced run would otherwise read as one that did nothing. The consumer-facing failure `ref` is untouched either way — it is the only thing that makes an untraced failure reportable. Custom blocks invoked as Agent tools now join too. The child's handle already reached the agent's persisted `toolCalls[].result` (`postProcessToolOutput` strips only `__`-prefixed keys); nothing lifted it onto the tool span. Both span builders lift and strip it, and `hydrateChildTraces` needs no change — its boundary walk already recurses. The same handle is stripped from the model-facing copy of the tool result in `executeProviderTool`, the single point where the raw and model copies diverge: an opaque execution id in a tool result reads to a model like data the tool returned. The live SSE stream keeps one condition beyond the policy: an identified consumer. Not an authorization check — no workspace query — but chat deployments and the public API leave `liveTraceViewerUserId` unset because their consumer may be anonymous, and opting into org-wide tracing is not consent to stream a publisher's raw agent tokens to the internet. Copilot deliberately cannot set the field; exposing a team's internals org-wide is a human decision, not one an agent makes while publishing on their behalf. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(custom-blocks): read the publisher's trace policy at read time, not from the handle's presence Treating a persisted `childExecutionId` as proof of publisher consent is only true for handles this PR's writer produced. Every handle written before it meant something else — "a child ran; authorize the reader" — and the rows carrying them outlive the migration, so removing the reader check turned them into an open door: a consumer could open an old parent log and receive the source workflow's block names, inputs, outputs, and prompts from a block whose publisher never opted in. `hydrateChildTraces` now resolves the policy live, per boundary, from `custom_block.trace_child_runs`. The child log row's `workflowId` is the key — publish enforces one block per workflow — which also covers an Agent-tool boundary, whose span carries no block type to look up. A workflow with no block row (never published, or since deleted) has no publisher left to consent and stays shut, as does a failed policy read. This is not redundant with the write-time withholding. The handler still emits no handle for a block that was closed when the run executed, so such a run stays closed forever even if the block is opened later; this check decides whether the runs that DO carry a handle may still be shown. Turning the policy off therefore also closes what is already recorded, which is what a governance switch has to do to mean anything. Reported by Greptile on #6950. Also drops `any` from the trace-policy tests: outputs read through `Record<string, unknown>` (the handler's declared return does not name these internal keys) and failures narrow through `ChildWorkflowError.isChildWorkflowError`, which pins the failure type as well as its fields. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(logs): sum the child-trace drop counters from the struct, not a hand-listed set `totalDropped` re-listed four of the five counters, so a read whose only drops were policy refusals computed zero and skipped the log entirely. That is the commonest drop there is now — every handle written before the publisher policy existed refuses at that gate — so the one signal telling an operator the live check is closing joins went silent exactly when it started mattering. Summed from the struct instead. A hand-maintained list beside a struct is stale the moment a field is added, which is precisely how `policyClosed` was left out. Reported by Cursor Bugbot on #6950. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(db): renumber the custom-block trace migration around a 0299 collision Staging landed its own 0299 (`table_run_dispatches.heartbeat_at`) while this branch was open. The two migrations are independent — different tables, no shared statement — so only the number and drizzle's snapshot chain collided. Regenerated rather than hand-merged: a drizzle snapshot is a full-schema dump whose `prevId` links it to its parent, so editing one by hand to sit after a migration it was not generated against is how the chain silently stops matching the database. Staging's 0299 and its snapshot are taken verbatim; this is 0300, generated against them, and its SQL is byte-identical to what it replaced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-21 18:21:08 -07:00
Siddharth Ganesan	58aa6379e0	feat(cli): add chat command (#6937) ... * feat(cli): add chat command * fix(cli): harden chat command execution	2026-08-21 12:59:08 -07:00
Waleed	dbbe99e473	fix(integrations): validation pass over Crunchbase, PitchBook, and CB Insights (#6925) ... * fix(crunchbase): widen tier-gated collection allowlists and cap the deleted feed The deleted-entity, autocomplete, and fields-metadata allowlists each held only the collections the narrowest package tier publishes, so requests valid on a richer package were rejected locally before any request went out. An Advanced Financials key could not read the funding-round deletion feed at all. - deleted-entity collections: 9 -> the 14-collection union across all tiers - autocomplete and fields-metadata: 14 -> all 43 collections - clamp the deleted feed to its documented max of 25, not Search's 1000 - offer "All collections" so the cross-collection feed stays reachable - name the richer-tier card additions instead of presenting the base set as exhaustive Also rewrites a test that asserted the broken behavior and tightens a substring URL assertion that passed on the value it was meant to reject. * fix(pitchbook): stop a rejected API key reaching block output and logs PitchBook's 401 body echoes the submitted key back inside `message`. No PitchBook tool declared an `errorExtractor`, so the failure fell through to the generic chain, whose first entry returns `data.message` verbatim — putting the credential in the block error, the run log, and any agent context reading the failure. The existing scrubber sat in `transformResponse`, which never runs on a non-ok response. - add a `pitchbook-errors` extractor that replaces the unauthorized message with a fixed string, and wire it through all 91 tools - the extractor returns undefined unless the body carries a `message`, so a foreign 401 on the shared fallback chain is never labelled a PitchBook failure - correct `investor_preferences.preferredIndustry` to the shape the API returns - make `company_industries.emergingSpaces` opaque; its item shape is undocumented - reject a non-list of article ids instead of throwing a bare TypeError * fix(cbinsights): reject malformed input instead of silently rescoping a billed query CB Insights is metered, so a filter that fails to parse must fail the request — dropping it does not narrow the result, it charges for a query the caller never asked for. - reject an unrecognized boolean rather than dropping it, which had been widening a VC-backed firmographics search - reject a non-numeric limit instead of falling back to the endpoint default - reject non-text filter entries instead of stringifying them to "[object Object]" - accept only asc/desc for sort direction; a typo had returned the bottom of a metered result set as though it were the top - treat a whitespace-only numeric bound as unset, not as zero - drop `totalHits`/`totalHitsRelation` from list business relationships; that endpoint reports no total, so both were permanently null - trim `nextPageToken`, matching the id fields Also moves the token cache onto `lru-cache` per the in-process caching rule, replacing hand-rolled TTL arithmetic and a manual prune. * chore(harmonic): drop the team-key help text from the credential descriptor * chore(tools): regenerate tool metadata for the validation fixes * fix(tools): redact the retained error body, not just the message Scrubbing the extracted message left the raw provider body reachable: `createTransformedErrorFromErrorInfo` attaches `errorInfo.data` to the thrown error and the executor surfaces it on the failed tool's `output.data`, so a PitchBook key rejected with an echoing 401 still reached block output and agent tool results via `output.data.message`. - add an optional `redactData` to the error-extractor contract, so an extractor that exists because a provider echoes a credential can replace the body too - retain `redactErrorData(errorInfo, extractorId)` in place of the raw body - PitchBook replaces only the unauthorized body; every other failure is untouched - cover the executor path itself, since asserting on the redactor directly still passes when nothing is wired to it	2026-08-20 22:22:28 -07:00
Theodore Li	5b28da1989	fix(tables): accept plain row query predicates (#6916) ... * fix(tables): accept plain row query predicates * fix(cli): show table predicate group syntax	2026-08-20 20:48:47 -04:00
3 people	42f6287911	feat(byok): add organization-wide key inheritance (#6834) ... * feat(byok): add organization key management * feat(byok): inherit organization keys at runtime * feat(byok): add organization scope to BYOK settings * fix(byok): refresh org key state after mutations * fix(byok): hide stale inherited status badges * chore(db): drop colliding byok migration ahead of staging merge Staging independently claimed 0293. Remove ours so the merge is clean; it is regenerated at the next free index right after. * chore(db): regenerate byok migration at 0296 Staging claimed 0293-0295 during the merge; the regenerated SQL is byte-identical to the dropped 0293. * docs(byok): document organization scope, precedence, and the full provider list The BYOK section described workspace-scoped keys only. Add the organization scope, its Enterprise requirement, the per-provider precedence rule, what an entitlement lapse does, and the Pi sandbox exposure. Refresh the provider table from the settings page, which had drifted from 14 to 34 entries. * feat(byok): open organization keys to every organization plan Organization BYOK was gated on Enterprise, but an organization is the only thing that can hold the keys, so every plan that can own an organization should qualify — Pro for Teams, Max for Teams, and Enterprise. Add checkOrgPlan/resolveOrganizationPlan beside the Enterprise pair rather than widening checkEnterprisePlan, so the Enterprise-only gates (Access Control, whitelabeling) are untouched, and restore resolveOrganizationEnterprisePlan to module-private now that BYOK no longer needs it. * perf(byok): cache the organization entitlement, not the key material getBYOKKey runs once per agent block and once per hosted-capable tool call, so a loop over N items resolved N times — and each organization-inheriting resolution paid three sequential billing queries on top of the two key reads. Split the two reads by staleness tolerance. Key rows stay fresh, because revocation must be immediate. The entitlement is a billing gate that tolerates bounded staleness in the harmless direction (a lapsed organization keeps using its own key for <=60s), so cache it per organization with an in-flight share so concurrent blocks issue one query set. The management surfaces keep reading it fresh, so an organization that just upgraded is never told otherwise. Also run the block check and subscription read in parallel inside resolveOrganizationPlan, and carry the resolved scope on BYOKKeyResult so a log line can say whether a run used the workspace's key or an inherited one. * feat(byok): let workspaces store the Z.ai and Cohere keys the runtime reads Both ids were already in the BYOK contract enum and both are resolved at execution time — getApiKeyWithBYOK reaches 'zai' (GLM models are in the hosted catalog, so the BYOK branch runs), and 'cohere' backs both the Embeddings block and Knowledge Base reranking — but neither appeared in the settings list, so there was no way to store the key either path looks for. Cohere had no icon; add one from the official multi-color mark so it stays legible on a light and a dark page. Cohere's embed-v4.0 is kbEligible:false, so the description says 'Embeddings and Knowledge Base reranking' rather than claiming KB embeddings. * improvement(byok): shorten the workspace scope chip to 'Workspace' It sits beside 'Organization', so the scope reads from the pair; 'This' only added width. * fix(byok): do not cache a billing outage as an unentitled organization resolveOrganizationPlan maps a failed billing read to false, which is indistinguishable from a real plan lapse. The entitlement cache stored that, so one transient outage held the gate shut for the full TTL and every inheriting run silently fell back to a metered hosted key — and the cache's rejection path, which exists to prevent exactly this, was unreachable. Give the resolver the onError option its neighbours already have and let the cached read ask for 'throw', so a failure stays out of the cache and the next resolution retries. Behavior for the call that saw the error is unchanged: getBYOKKey still fails closed. Reported by Cursor Bugbot. * fix(byok): propagate the subscription read's failure too The previous commit threaded onError through resolveOrganizationPlan's own catch, but getOrganizationSubscriptionUsable soft-fails to null on its own, so a failed subscription read still arrived as an ordinary 'no usable subscription' and returned a successful false — which the entitlement cache then stored for the full TTL. Thread the option into that call as well. Test it at the billing layer rather than the cache layer: the entitlement test mocks resolveOrganizationPlan wholesale, so it could never have caught this. Verified the new test fails against the previous commit. Reported by Cursor Bugbot. * refactor(byok): cache the entitlement with LRUCache, like copilot entitlements The hand-rolled version reinvented three things the codebase already has a canonical answer for. lru-cache is a declared dependency of apps/sim and lib/copilot/entitlements.ts already caches an entitlement with it — by storing the in-flight Promise, which is what makes concurrent callers collapse onto one resolution with no in-flight bookkeeping at all. TTL and the size bound come from the library. That removes the second Map, the manual eviction (and its interaction with an in-flight entry), and the dead value-while-refreshing state: 23 executable lines. The one thing the library does not cover is dropping a rejected promise so a billing outage is not cached for the TTL, which is kept and pinned by a test that fails without it. TTL expiry is no longer re-tested — that is the library's behavior, not ours, and lru-cache reads its clock at module load so faking timers never moved it. * refactor(byok): coalesce the entitlement read with the shared singleflight lib/concurrency/singleflight.ts is the codebase's coalescing primitive and oauth/credential-service.ts already pairs it with a read-through cache. Adopting that shape fixes a case caching the promise directly did not: a *hung* billing read wedged every caller for the full 60s TTL, where coalesceLocally evicts and rejects at its settle deadline. It also removes the hand-rolled rejection eviction — the cache is written only on the success path, so an outage leaves no entry by construction. The cache now holds booleans, which introduces the one trap worth a test: a truthiness check would read a cached false as a miss and re-query billing on every resolution for lapsed organizations. Pinned. * fix(byok): keep an abandoned entitlement producer from writing the cache coalesceLocally does not cancel a producer it timed out — its docstring says so explicitly — so writing the cache from inside the producer let a late billing result overwrite a fresher answer a retry had already cached, and hold it for a full TTL. Move the write onto the value the caller actually received. A caller that timed out throws before reaching it, so an abandoned producer now resolves into nothing. The test reproduces the overwrite and fails against the previous shape. Reported by Cursor Bugbot. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>	2026-08-20 17:45:28 -07:00
3 people	ea70f8dcf1	feat(harmonic): add contact workflow integration (#6902) ... * feat(harmonic): add contact workflow integration * fix(harmonic): sync docs manifest * fix(harmonic): address integration review findings * feat(harmonic): add the missing people endpoints and fix two error paths Extends the integration from 4 to 13 tools, covering every non-deprecated people-scoped Harmonic endpoint, and repairs two defects found by validating the existing tools against Harmonic's OpenAPI and API reference. New tools: - Enrich Person (POST /persons) — the only path from a LinkedIn URL or email a workflow already holds to a Harmonic contact. - Get Person, Get Company Employees — account-based sourcing; employees returns URNs that chain into Batch Get People. - Saved-search net-new results and their acknowledgement, so a monitor stops reprocessing the entire result set on every poll. - Bulk email enrichment: submit, poll, and quota, plus Get Enrichment Status. Fixes: - The error extractor dropped Harmonic's string and object `detail` envelopes. A tool that names an extractor gets no fallback chain, so every FastAPI abort surfaced as "Request failed with status 403". The enrichment 404 also carries the scheduled `enrichment_urn`, which was being discarded — that URN is the only handle on the job, so it is now kept in the message. - The saved-search selector failed the whole dropdown instead of degrading: the response cap was half the sibling value on an endpoint that is unpaginated and returns every saved search with its full query object, and the option ceiling threw rather than truncating. Raised to 1MB and switched to truncate-and-warn, matching the other data-driven selectors. Clearing net-new results now requires an explicit scope. Harmonic treats an absent `entity_urns` as "clear everything", so an empty field would have silently discarded the backlog. Scope deliberately excludes company-side, deal, typeahead, network, and Scout streaming endpoints, and every endpoint retiring on 2026-11-05. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-08-20 17:12:57 -07:00
Waleed	865f8173ab	feat(affinity): add Affinity CRM integration (#6908) ... * feat(affinity): add Affinity CRM integration Adds the Affinity v2 API as a block with 70 tools, covering 86 of the 87 documented endpoints. Only Send Feedback is omitted — it reports product feedback to Affinity rather than doing workflow work. Endpoint families that differ only by an entity segment are one tool with an entityType param, so companies/persons field, list, row, and relationship reads, the company/person merge endpoints, and entity notes each collapse into a single operation. * chore(affinity): regenerate the docs manifest for the new integration page	2026-08-20 16:27:29 -07:00
Theodore Li	a99f61bee9	feat(api): add v2 resource management endpoints (#6900) ... * feat(api): add v2 resource management endpoints * fix(cli): gate destructive v2 commands * fix(test): update v2 request-slice count * feat(cli): add shared workspace profiles	2026-08-20 18:55:57 -04:00
Waleed	a27f376164	feat(connectors): add Bitbucket, Databricks, Google Chat, and Workday Help KB connectors (#6895) ... * feat(connectors): add Bitbucket, Databricks, Google Chat, and Workday Help KB connectors Adds four knowledge base connectors, closing the gap where Sim shipped tool blocks for these services but could not index their content. - Bitbucket: repository source files and pull request descriptions over the existing Bitbucket OAuth credential - Databricks: notebooks (Workspace API) and saved SQL queries, PAT auth - Google Chat: spaces indexed as message transcripts, new google-chat OAuth service under the shared Google client - Workday Help: knowledge article versions via the public helpArticle/v1 API * fix(connectors): second-pass validation fixes and test coverage Adversarial re-validation of all four connectors plus a combined-change regression audit. - bitbucket: stop declaring incremental sync (deletion reconciliation is disabled for incremental runs, so deleted files were never removed on the default code configuration); drop a wasted listing round-trip after the frontier drains; add 33 tests - databricks: reject an explicit maxDocuments of 0, which meant unlimited; add 34 tests - google-chat: correct the sender displayName documentation (user auth populates only name and type) and emit second-precision RFC-3339 in the message filter; 15 -> 24 tests - workday: fix a crash when maxVersions is persisted as a number; refuse a configuration whose status filter Workday did not honor; cap the unresolved-name error; 18 -> 27 tests - document the Google Chat service-account omission - docs: list all four connectors and correct the connector count * fix(connectors): index Google Chat spaces with no messages in the window Review round 1. - orderBy takes a full ordering expression, not a bare direction. The reference documents the default as `createTime ASC`, so send `createTime DESC`; a bare `DESC` either 400s every hydration or is ignored, which would make the cap keep the oldest traffic and the later reverse render the transcript backwards. - getDocument no longer returns null when the message window is empty. A space with no messages is still a live space, and null is the "document is gone" signal the engine treats as last-known-good: returning it dropped spaces whose only prose is their description or guidelines, and left a stale transcript indexed after a space was cleared or lookbackDays was tightened past every message. - The transcript header is omitted when no message contributed text. * fix(connectors): only flag a Bitbucket listing capped when the cap withheld something Review round 2. takeIndexableWithinCap reports capReached as soon as the running total equals maxItems, which is also true of a listing that ended at exactly that count. Setting listingCapped there suppressed deletion reconciliation for a complete listing, so upstream-deleted files and pull requests could stay in the knowledge base indefinitely. applyMaxItemsCap now takes whether Bitbucket had more content beyond the page -- a next link, or directories still queued on the frontier -- and flags the listing only when the cap actually withheld something, matching the Databricks, Google Chat, and Workday connectors. * fix(connectors): keep the Bitbucket cap flag set when it skips the pull request phase Review round 3. Fixes a regression from 622a21bd9a. maxItems is shared across the code and pull request phases, so a code walk that ends with exactly maxItems documents and no next link or frontier stops pagination before the pull request phase runs. Scoping listingCapped to "this phase had more" left the flag unset in that case, and the engine then treated the run as a complete enumeration and could hard-delete previously indexed pr:* documents that were never listed. The cap flag now asks whether anything the connector was configured to list remains unlisted -- including a later phase the cap is about to stop us reaching.	2026-08-20 14:46:29 -07:00
3 people	d9cfd7c68e	improvement(mothership): v0.9 (#6815) ... * checkpoint * Checkpoint * dot fixes * Make async tool resume delivery recoverable * Support split table tools and option recovery * Harden VFS mutation handling * feat(platform): platform subagent support — docs corpus VFS, search_docs, account context Squash of the feat/platform-agent branch (sim side): mounts the Sim docs corpus in the copilot VFS, wires search_docs and retires the legacy docs search tools, and syncs the generated tool catalog and trace contracts for the platform subagent. * Align Copilot tools and resource handling * Expand workflow log query support * checkpoint * Port desktop-improvements-0 desktop and browser-agent work * fix(desktop): keep browser-agent input alive through live-SPA re-renders * Harden workflow sanitization and Slack setup * feat(desktop): coordinate clicks, caret insertion, and drag for the browser agent * Revert subagent group eager auto-collapse * fix(chat): keep sends FIFO across the streaming-to-idle drain gap * Add the steering backend surface for mid-turn sends * Sync generated contracts for async subagent orchestration Pulls the mothership tool catalog (wait_agents / tail_agent / steer_agent / interrupt_agent), trace spans (chat.async_subagent.*, chat.orchestrate.*), and trace attributes (copilot.async_subagent.*) into the generated TS contracts. * Add display titles for the async subagent orchestration tools wait_agents / tail_agent / steer_agent / interrupt_agent get natural-language running titles (naming the agent id being waited on, tailed, steered, or stopped) and a Steering→Steered completed-verb rewrite. * Show orchestrator-chosen subagent names on agent groups A subagent_start whose payload data carries a name (the orchestrator's new name trigger parameter) now labels the agent group with that mission name — the agent-type icon stays. The name flows through the live stream path, the turn model (AgentNode.displayName) and its serialize/rebuild round-trip, and persisted transcripts (PersistedContentBlock.name), so reloads keep the label. * Improve Copilot error handling and logging * Backfill the subagent display name from the second start event The dispatch-time subagent_start fires before the trigger args (and therefore the name parameter) have streamed; the phase-3 start re-announces the lane with the name. The block builder was dropping that duplicate wholesale, losing the name on streaming providers — now it backfills subagentName onto the existing block instead. (The home turn-model path already reconciled this case.) * Support Slack bot connection flow * Harden Copilot error and VFS handling * Harden VFS resource operations * Show 'Waiting for the first of N agents' for mode-any waits The wait_agents title ignored the mode argument, so an any-mode wait over three agents read 'Waiting for 3 agents' while the model narrated waiting for the first — contradicting the transcript. * Collapsed-by-default agent cards with live intent status lines Subagents now narrate their work through <intent>3-5 words</intent> tags (a fleet-wide prompt protocol on the mothership side). The turn model streams each subagent's text through a split-safe tag parser: complete tags update the agent's currentIntent and disappear from the prose, tags split across deltas are carried until their close arrives, and a tag that never closes flushes back as plain text. The agent card renders as one line — display name (or agent label) plus the latest intent, replaced inline as the agent shifts gears — and never auto-expands; expanding to the full tool log is a deliberate click. Only an outstanding permission prompt or a browser hand-back forces a group open. Intents persist on the subagent block (and through the legacy persisted- message paths) so reloads keep the last status, and a renamed reinvocation now takes the latest name instead of pinning the first. * Add the internal in-band tool execution route for live mothership turns POST /api/copilot/tools/execute (INTERNAL_API_SECRET, Go→Sim) runs one sim-server tool through the same server tool router the resume driver uses and returns the result synchronously — no checkpoint. This is what lets background (async) subagents write files/tables/knowledge, and lets the main lane keep streaming (instead of checkpoint-pausing and killing every background run) while async agents are live. * Persist resource side effects for in-band tool execution Files/tables created through the internal execute route now register on the chat's resources exactly like the resume driver's executions — the route runs the same handleResourceSideEffects pass (persistence only; an out-of-band route has no live event sink, so mid-turn chip pushes are a follow-up). * Extract intents from group text on every path, sync and async The turn-model intent filter only fires for span-scoped subagent lanes, but this surface also delivers subagent text through the legacy block path — so <intent> tags flowed through unparsed and rendered as prose rows. Groups now extract intents from their accumulated text at append time: the last complete tag becomes the card's status line and every complete tag is stripped from the rendered prose. Covers span-scoped, legacy, and persisted-reload paths for both synchronous and background delegations. * Fall back to the live tool title for the agent card status line Persisted data proved tool-first subagents (grok search agents) emit zero prose, so intent tags never stream no matter what the prompt says. The collapsed card now always narrates: the agent's own <intent> tag when present, else the latest tool's display title while the lane is live. * Catch subagent <intent> tags in the server relay The relay's subagent text handler now runs the split-safe intent extraction as chunks stream: the latest complete tag is stamped onto the lane's persisted subagent block (subagentIntent) and stripped from the stored prose, so live, persisted, and replayed views all agree. Per-lane carry handles tags split across chunks; a never-closing tag flushes back as plain text. * Drop the tool-title fallback: the status line is the agent's intent With the intent protocol now injected into every spawn's task message, agents open with an <intent> tag; the card shows that narration or nothing. * Replace intents with live tool-title status lines on agent cards Intent parsing is fully removed (turn model, relay handler, persistence fields, group extraction). The collapsed card's status is the latest tool call in its RUNNING phrasing — never the completed rewrite, which stays in the expanded log. Parallel tools show the most recently started still-running title with a +N for concurrent siblings; between rounds the last title stays frozen; a closed lane shows the bare name. Nested agent cards compute their own status recursively from their own items. * Keep the main Sim lane live-expanded; collapse only real subagent cards The mothership group is the turn's own narration, not a delegation card — collapsing it hid main-lane text and tools until manual expand, which read as mis-ordered streaming while async subagents interleaved. It keeps the original live-expand behavior and no status suffix. * Persist subagent lane lifecycle blocks from the span handler Lane-scoped span events route to the span handler, which only recorded trace side effects — no subagent start block was ever persisted (verified: a seven-agent run stored 104 blocks with zero starts). Grouping then fell back to keying lane content by agent NAME, so a respawned agent of the same type merged invisibly into the first one's card until it resolved. The handler now persists the start block (spanId-keyed and deduped, carrying the display name) and stamps endedAt on close, giving every invocation its own card. * Name agents in orchestration titles; '+ n more' overflow format wait/tail/steer/interrupt titles humanize the slugified agent ids back to their display names ('Waiting for the first of Digest Workflow Build + 4 more'), and the agent card's parallel-tool suffix uses the same '+ n more' format. * Harden in-band tool execution and resources * Route in-band execution through the comprehensive tool dispatcher The internal execute route used the bare server-tool router, which rejects VFS tools with 'Unknown server tool: read/glob/grep' — so nearly every background agent's first discovery call failed (102 in-band calls in one run, dozens rejected). It now uses the relay's executeTool dispatcher: registered handlers (VFS, function execute) with permission checks and param normalization, falling back to the app tool router — the same surface foreground execution gets. * Harden chat stream transition handling * Harden VFS provenance and resource writes * Standardize tool environment references * Harden browser panel and chat cleanup * Descriptive, user-language tool titles across the board House rules applied everywhere: use every argument the call carries, never name internal machinery, and never lead with Getting (the Got rewrite is deleted so it cannot return). - Deployments name the workflow: Deploying {workflow} as API/chat app/MCP tool - Workflow reads name the part: Reading {workflow} meta/state/deployment/notes; generic reads always name the file (Reading {leaf}), never bare Reading file - Block runs name block and workflow: Running {block} in {workflow}, Running from {block} in {workflow}, Running {workflow} until {block}, and Enabling/Disabling {block} in {workflow} - The six split-table tools get per-operation verbs (Adding column {name}, Updating rows, Wiring automation, Creating view {name}) instead of a wall of Querying table - The manage quartet drops X-action system-speak for gerunds - get_* internal names become user language (Checking run settings, Tracing block inputs, Reading the deployed version); web_fetch says Fetching - Scheduled-task titles removed entirely (feature deleted from the Go catalog) - New verb rewrites: Fetched, Traced, Wired, Configured, Looked, Rotated * Deploying {workflow} as chat, not as chat app * Loader gerunds; mv names both ends; mkdir names the folder search_integration_tools -> Finding the right integration; load_integration_tool -> Loading {integration} tools; load_skill -> Loading skill {name}; run_enrichment -> Looking up {subject}. mv prefers the model's phrasing, else reads 'Moving {files} to {destination}'; mkdir reads 'Creating folder {name}' from the path. * Overflow counts read '+ n', dropping 'more' * Unify workspace find and search * Scale desktop title bar with page zoom * Serialize account and organization truth into the copilot VFS Workspace standing, membership, billing, org role, access-control restrictions, published-block provenance, and fork topology were reachable only through three parameterless tools (or not at all). They are ambient read-only facts, so they belong in the VFS where they are greppable, cost no tool round-trip, and every agent that can read gets them — the same move that retired get_blocks_and_tools and list_user_workflows. Adds account/{workspace,workspaces,members,billing}.json (always mounted) and organization/{organization,access-control,custom-blocks,forks}.json (only when the workspace is org-hosted). Every file projects an existing use case or util after getOrMaterializeVFS's access assert — no new queries, no new authorization. One relation per file, cross-referenced by id-and-name stub, so overlapping facts cannot disagree. Volatile content (billing, access control, forks) is lazy, so numbers are read-time fresh and unasked-for reads cost nothing. Projection follows the viewer: member emails are admin-only, fork detail requires workspace admin on a forking-enabled org, and the whole organization/ namespace is absent for a personal workspace — which is itself the answer. Retires get_account_billing, get_enterprise_context, and list_user_workspaces along with their handlers; display titles stay for transcript replay. * Fix insert_text refusing an editable field focused inside a frame describeFocusedEditable descended shadow roots but not frames, while activeElementReadback descends both. Focus inside a same-origin frame therefore surfaced to the first as the FRAME element — not an input, not contentEditable, not a canvas, no textbox role — so it fell through to 'not-editable' and insert_text refused a field that press_key had just typed a character into. Two functions answering 'what is focused' with different answers is the bug; the descent loops now match exactly. The refusal also names what actually held focus (tag, role, contenteditable). A bare 'not-editable' gave the agent nothing to act on, so it guessed at the cause — a real run spent twenty rounds on the wrong theory and had to be stopped by the user. * Keep retired browser takeover renderable in history The tool is gone from the catalog, so its generated constant went with it and every path that referenced it stopped compiling. Deleting those paths instead would have silently downgraded every past transcript containing a takeover card to a generic tool row, and dropped the no-timeout budget that an in-flight takeover still needs while a rolling deploy finishes. retired-tools.ts gives the literal a documented home that says what it is and why it survives its tool. * Follow the agent into a tab it opened to work in browser_open_tab created the page with activate: false, so the agent worked in a tab the user could not see while the panel sat on a page where nothing was happening. The panel now follows a tab the agent deliberately opened. Scoped to that tool only. A page spawning its own tab (popup, target=_blank) is the site grabbing the view rather than the agent choosing a workspace, and stays in the background as before — two existing tests pin that and caught the first version of this change, which moved both. A tab the user claimed still wins over both: the work starts in the background instead of pulling the page out from under them mid-read. * Make the browser tools agree with each other An audit of the module found the frame-descent bug was one instance of a pattern: six independent definitions of 'is this editable' and seven of 'what is focused', disagreeing with each other. A tool refusing what its sibling accepts on identical page state is invisible at runtime — the agent follows a snapshot that says one thing into a tool that says another. - browser_type now accepts role="textbox" like browser_insert_text does. The snapshot advertises those elements as [textbox] with a ref, so refusing them meant rejecting exactly what the outline told the model to type into. Both the native and synthetic paths, and their descendant scans. - pressKeyOnPage descends shadow roots and frames like every other focus reader. It was dispatching synthetic keys at the shadow host or <iframe> element, where they bubble but never reach the editor, while reporting success — and contradicting the activeElement reported beside it. - not-editable and ambiguous-editable name what was found: the element's tag and role, and the candidate fields. Both had the data and discarded it, which is what turns one blocked step into twenty rounds of guessing. - obstructedAfterNavigation requires a dialog that ARRIVED with the navigation. It compared against nothing, so every SPA route change under a persistent role=dialog reported a successful click as obstructed. The test that covered this asserted the false positive; it now pins both directions. - browser_insert_text observes the top document when typing inside a frame, like every other input tool. A submit that navigates the top page was invisible to its frame-scoped observation. * Let hover actually see what it mounted Four independent defects made browser_hover blind to the most common thing a hover produces — a row's action bar — so it reported no effect on a hover that worked, and the agent fell back to clicking pixels off screenshots. - The popup scan matched only role=tooltip/menu/listbox. Slack's message shortcuts bar is a labelled toolbar/group, so it registered as nothing at all. Added toolbar, menubar, labelled group, and [popover]. - The baseline was captured BEFORE prepareElementSurface scrolled the target into view, so scrollChanged was always set by the tool's own probe. That pinned every unproductive hover to 'background DOM churn' instead of the honest 'nothing happened', and hid scrolling the hover really caused. Re-baselined once the scroll settles and before the pointer moves. - The MutationObserver attached only on the first observation, while the roots list is rebuilt every call and grows as shadow roots mount. Components that appeared later were never observed, so their DOM changes raised no revision. Roots are now observed as they show up. - observationTruncated was computed and never read, so a scan capped at 12k nodes reported 'nothing appeared' with the same confidence as a complete one — and portalled overlays live at the end of <body>, exactly what the cap drops. Hover now says the page was too large to scan and to confirm visually. * Stop the browser agent acting on the wrong element, and say why it refused Four findings from the module audit, the first of which could silently do the wrong thing rather than merely fail. - A ref whose node is gone is re-adopted by structural resemblance, matching on ORIGIN only so a pushState between snapshot and act does not kill every ref. That leniency also let a ref to a row control in one view rebind to the identical control in a view the app had since navigated to — acting on the wrong message, signalled by nothing louder than recovered: true. Adoption now requires the same path; a view swap reports the ref stale, and the caller re-snapshots. Revalidating a still-connected node stays lenient, because that is literally the node the model chose. - A hit INSIDE the requested element is its own nested control, not an overlay. Both produced 'covered by X — close or move the overlay', advice that cannot be followed because there is nothing to close. Nested hits now say so and point at retargeting. - browser_click_at, browser_insert_text, and browser_drag listed targetChanged in their effect formulas, but none passes an elementId, so no targetState is ever captured and the term was always false — coverage that read as real. Removed, with a test pinning the dependency. - The seven effect formulas are deliberately NOT collapsed into one predicate: drag must trust domChanged where others must not, hover must ignore field and focus changes, click counts focus only for editables. Forcing one would make each tool wrong differently. The differences are now documented in one place next to the shared computation, so divergence is a declared policy rather than an accident. * Let edit_workflow configure block retries * Updates * Always focus the resource the agent is working on, and its browser tab The resource panel had a carve-out: an already-open browser session declined to replace another selection and only got an attention marker, so agent browser work happened off-screen. The panel now follows the agent to whatever it touches — browser included — and an event can still opt out explicitly. The browser panel also follows the agent BETWEEN tabs: the store already tracked automationTabId (and the strip marked it), but the visible tab never changed. It now switches when the agent's target tab changes, so watching the agent never means hunting for the tab it moved to. Keyed on the target changing rather than on it being set, so a user who browses elsewhere mid-run is only pulled along when the agent itself moves. * Never paint a browser snapshot at stale geometry (the modal-open flash) Opening a modal locks scroll, which removes the window scrollbar and reflows the panel — so a capture taken before the lock describes a rect the panel no longer occupies. The handshake painted that frame anyway and only then retried, so the replacement landed visibly offset from the page it stands in for: the flash. A capture is now checked against the host's live rect before it is painted; a mismatched frame is skipped and re-captured at the settled layout instead (modal retries go 2 -> 3 to absorb the extra settle). * Name the workflow in deployment and workflow-scoped tool titles 'Checked deployment status' never said which workflow — nor did the deployed- state read, run settings, block outputs/inputs, redeploy, promote, or the global-variable write. These tools carry a workflowId (often defaulting to the current workflow), so only the client can resolve a name: the enrichment layer now resolves it for the whole workflow-scoped family and passes it as workflowName, which every workflow title already reads. Titles: Checking {workflow} deployment status, Reading deployed {workflow}, Checking {workflow} run settings, Reading {workflow} block outputs, Tracing {workflow} block inputs, Redeploying {workflow}, Promoting {workflow} version {n} to live, and 'Adding workflow variable {name} in {workflow}' — each falling back to its unnamed form when no workflow resolves. * Name the block that ran; never fall back to a raw block id run_block and set_block_enabled carry only a blockId, so their titles would have printed an opaque UUID ('Running 7f3a2b91-… in Invoice Sync'). The enrichment layer now resolves blockId against the workflow store the same way it already did for run_from_block's startBlockId, and the base titles no longer accept an id as a name — an unresolved block reads 'Running block' rather than a UUID. * Add the missing Removing -> Removed rewrite The table work introduced 'Removing automation'/'Removing enrichment' with no past form, so those rows kept their present tense after completing. * Name the target resource in the remaining tool titles Table tools keep their operands nested under args and identify the table by id, so their rows said 'Adding rows' with no hint where: enrichment now lifts the nested args and resolves tableId against the cached workspace table list, giving 'Adding rows to Runtimes', 'Adding column status in Runtimes', 'Reading views of Runtimes'. Also: a block-schema read names the block instead of the file ('Loading Slack', 'Loading Google Sheets tips'); browser type/insert show the text they send, middle-ellipsized; downloads name the file; library-docs searches name the library and query; knowledge-base searches include the query; generated media names its output file; and diff_workflows, list_deployment_versions, and publish_custom_block joined the workflow-name enrichment set. * Bubble nested agents' tool calls into the parent's status line The collapsed status only scanned a group's OWN tool items and skipped nested agent groups, so a parent that had delegated froze on its last own tool while its child did the actual work — the line described nothing that was running. Status now walks the whole subtree: any tool at any depth counts, the most recently started running one is shown, and the rest become the same '+ n' overflow. With nothing running it falls back to the last tool at any depth, so an idle parent still reflects where its subtree got to. * Align nested tool call status rows * Revert branch-local KB connector error-message edits Restores apps/sim/connectors/ to staging state. Two copilot-focused commits on this branch (3a9fc1c5a5, 24b24e5f8e) drove by nine KB source connectors (airtable, confluence, discord, github, gitlab, google-drive, microsoft-teams, notion, slack) to enrich credential-validation error messages. The env-reference resolution machinery those errors supported is kept; the product connector surface stays unchanged in this branch so the staging promotion remains scoped to copilot work. * Fix the failing audits: NUL escape and route-count ratchet check:source-text: resource-vfs.ts used a raw NUL byte as its folder-index key separator (comment and template literal), which makes git treat the file as binary and hide it from review. Written as the '\u0000' escape — the runtime string is identical. check:api-validation:strict: baseline 1120 -> 1122 for the two routes added on this branch since the last bump. * Add Force Reload (Cmd+Shift+R) to desktop View menu * Route Force Reload through focused-resource boundary; regen docs manifest * Align the title-bar surface audit with the zoom rework 'Scale desktop title bar with page zoom' (833d2f126d) changed the CSS contract in two ways its audit test still pinned the old shape of: the lane vars gained a max() floor around the platform env() terms so page zoom cannot shrink the lane below the OS-drawn lights, and the control square became fixed px — CSS px already scale under zoom — leaving only the centering offset derived from the lane height. The test required the bare env() prefix and calc() on all three control vars, so it failed the commit that implemented its own regression comment. Pins now assert the env() term inside the clamp (still platform-derived, the test's actual intent) and split the control vars: offset must stay computed, size and icon are explicit constants. * Budget the two structurally slow tests explicitly sso-trust imports the whole Better Auth module graph (2.5s on an idle machine) and events.attribution scans call sites across the repo. Under a fully-parallel uncached run on a loaded machine both blow the default timeout while passing in isolation and on CI — a verdict decided by machine load, not by the code. 30s budgets make a loaded local run mean what it says. * Carry the cross-service trace id in sim log lines * Improve nested tool status presentation * Refresh secret schemas and shared test mocks * Open the deployed graph of org-published blocks, read-only, org-wide A consuming workspace could see a published block's interface but never what it does: the backing workflow lives in the publishing workspace, and other workspaces are nameable, not readable. Publishing a block org-wide is the act of sharing it, so the graph it executes is now readable from any org workspace — the DEPLOYED graph, not the publishing workspace's live editor state, so nothing in-progress leaks and what you read is what runs. The namespace also adopts the root's index/detail split: custom-blocks.json slims to names with a detail pointer, organization/custom-blocks/{type}.json carries provenance plus the deployed graph (loaded lazily through the cached loadDeployedWorkflowState; credential ids and env references inside it belong to the publishing workspace and say so), and organization/README.md is the namespace guide WORKSPACE.md is at the root — files, usage, the in-depth block inventory, and forks.json documented only when actually mounted. The block list moves to materialize time (same indexed query the components pass already runs) because the README, the index, and the per-block key-view entries all need it; only the graph stays lazy. * Withhold the deployed graph from external collaborators Workspace access and org membership are different grants: an external collaborator can open the workspace and use the published block, so the names index and the interface schema stay visible to them — but the deployed graph is org implementation internals, and their detail files now simply do not exist. isHostOrganizationMember is the viewer bit the host context already resolves for exactly this distinction. * Fill out the organization namespace: workspaces, permission groups, credential groups Three more read-only files, all lazily loaded — the paths appear in the key view so glob discovers them, but no query runs until a read — and all gated by registration, so an unpermitted viewer's file simply does not exist: - workspaces.json (org members): the org's full workspace map with the viewer's access flag and fork parentage — account/workspaces.json only ever showed what the viewer can reach. Inaccessible workspaces stay nameable, not readable. - permission-groups.json (org admins): every group with member count, targeted workspaces, and the restrictions its config activates. access-control.json remains the per-viewer binding. The queries are lifted into lib/permission-groups/queries.ts because their only prior home was inline drizzle in the route handlers, which the VFS cannot import. - credential-groups.json (entitlement-gated): per-option configuration readiness and enrollment progress — the two facts that decide whether a credential_group workflow will do anything at runtime. The note teaches the contract that bit the audit: an active group with zero completed enrollments yields an empty loop, not an error. Enrollee emails are workspace-admin-only, matching the settings page; counts come from the first enrollment page and say so when truncated. The README documents each file only when mounted for this viewer. * Stop reporting a click that navigated as a failed click The field case: 'Begin Assessment' submits a form. The navigation tears the origin document down while the press completes, so everything after dispatch — the CDP call's own completion, the synthetic dispatch's return value, the postcondition reads — fails against a destroyed context, and a maximally successful click was reported 'Failed clicking element'. The agent's own follow-up investigation in the transcript diagnosed exactly this. navigationRescue detects it at the driver level, where the navigation epoch and URL survive the renderer teardown: when the page provably navigated since dispatch began, a dispatch-path failure becomes a success carrying navigatedDuringDispatch and a note explaining why no postconditions exist. Applied to all four click dispatch paths (unframed CDP, framed native, synthetic in-page, click_at). The soft path had the same blindness: with the after-state unreadable, urlChanged computed false and a navigating click reported 'no observable change'. navigatedByDriver now folds into navigated/effectObserved, which also keeps the new-dialog obstruction check meaningful on real navigations. * Re-hide the browser under a modal after main loses the occlusion lease The punch-through: a modal opens, the native view hides behind a painted snapshot, and the renderer records applied: true. Then one heartbeat commit is skipped — renderer jank past the 2.5s bounds-lease TTL is enough — and main expires the lease, resetting panelOccluded on its side. The next heartbeat finds the modal marker still present and calls setDesired(true), but the lease's dedupe sees applied === desired and sends nothing; the bounds commit that follows lays out an unoccluded native view above the open modal, and no later event ever re-hides it. The comment on this branch already claimed it 'reasserts the lease' — the dedupe made that claim false exactly when the lease had been lost. While the occlusion marker is present, each heartbeat now drops the applied belief (assumeRevealed) before setDesired, so the reassert is a real, forced, idempotent hide IPC — one per second while a modal covers the browser — and any main-side lease loss self-heals within a heartbeat. * Make a stale ref name which of its five causes fired A field run burned five snapshot->click cycles on a STATIC landing page, every one refused with the same sentence — 'the page changed since the last snapshot' — and the agent reasonably concluded the page was regenerating its DOM. It was not; the resolver was refusing, and the message could not say why. Five distinct conditions produced that one string: an id missing from the registry, a connected node whose identity drifted, the view-changed adoption gate, no confident replacement, and a replacement tie. The resolver now stamps the reason (with the drifted node's current identity, or the from->to paths for the view gate) and every stale producer carries it into the driver message. Same pattern as not-editable: a refusal that names its cause costs one round; an opaque one costs a loop and a wrong theory in the bug report. * Pulse throttling off when the browser view is revealed, so it actually paints The blank-page report: a navigation completes while the view is hidden, the page 'finishes loading', and the panel shows white until the user re-navigates by hand. invalidate() on reveal was already there but recomposites the LAST frame — and the last frame is blank, because background throttling suspended the rAF the page's SPA paints its first frame from. The reveal now pulses throttling off (forcing the renderer to produce a real frame), invalidates, and hands the policy back to the session a second later through reassertTabThrottling, which preserves the automation-tab exemption. * Let the agent browser join meetings and finish passkey sign-ins Camera and microphone: 'media' joins the agent partition's allowlist, but every grant is gated on the macOS grant first — asked via systemPreferences.askForMediaAccess so the system prompt appears on first use, and answered from getMediaAccessStatus on checks — so System Settings stays the real authority and a page can never hold a grant the OS refused. Granting site permission without the OS grant produced the misleading NotReadableError Google Meet showed. Packaging gains the camera entitlement and usage string (macOS kills the process on prompt without one) and the mic string now covers meetings. Passkeys: WebAuthn itself is Chromium-native and nothing in our handlers blocks it — USB security keys need no permission at all. The hybrid transport (passkey on a nearby phone via QR) rides Bluetooth, which signed builds silently lacked: the bluetooth entitlement and usage string enable it. iCloud-Keychain platform passkeys remain outside what an entitlement here can grant — Apple restricts that to approved browsers. * Compile and preview Sim-styled pages * Wait for the batched prepare intent instead of instantly failing apply_file_edit The model batches prepare_file_edit and apply_file_edit into one round and the Go loop runs same-round tools concurrently, so apply could reach the executor before its prepare staged the intent. The instant no-intent error cost a model retry round and flashed 'Failed creating …' on the shared file row before the retry succeeded. The apply handler now polls briefly (10s cap) for the intent; a truly missing prepare still errors at the deadline. * Store page source, render docs-styled documents on view The pdf model for agent pages: the .html file keeps the markdown-shaped source (frontmatter + prose + sim: fences) and every surface renders the docs-styled document on demand — preview panel, /api/files/serve, public shares, and downloads all call the same pure compiler, now shared in lib/workspace-files. The docs chrome is reproduced from the real fumadocs source: the left sidebar's exact pill metrics, the clerk TOC with its animated scroll indicator, divider-style tables; cards and stats left the vocabulary. Table cells and kv values render inline markdown, and sim:workflow/table/knowledge/file links resolve to real workspace routes, bridged out of the sandboxed preview to the app router. Hand-written imitations of rendered output are rejected at apply_file_edit with a steer back to source, and a streaming page hides its source behind the live rendered preview (batched ~2s) the way a generating pdf hides its script. * Stagger the page rails so the resource panel keeps the docs sidebar Rails were gated at 1100px of iframe width — the chat resource panel never reaches that, so pages rendered single-column there. The rails now stagger the way the docs do on a laptop: >=640px keeps the section sidebar (240px) beside the content, >=1060px restores the full three-column frame with the clerk TOC, and only a truly narrow pane collapses to one column. * Fix in-page anchors escaping the preview; add the docs' toggle, code frames, pagination, and images Clicking a TOC or section link (or pressing Enter in the section filter, which clicks one) navigated the sandboxed frame off about:srcdoc in Electron and landed on a cookie-less sign-in page — the shell now intercepts every '#' anchor and scrolls directly. The page chrome gains the docs' exact theme toggle (emcn Sun/Moon, 30px rounded-lg, top right), framed code blocks with language label and copy button, and footer previous/next cards from prev/next frontmatter. Workspace images () compile to /api/files/view and the preview host inlines them as blob: URLs so the cookie-less frame can render them; sim:accordion joins the vocabulary as the faq component with title keys. * Size the section sidebar to its content so it appears at panel widths The left rail was a fixed 240/300px column, so it only earned its place once the pane was wide. fit-content caps at the docs width but shrinks to the longest section title (150px floor for the pills), and the two-column tier now starts at 560px instead of 640. * Let the clerk TOC join at 860px by sizing it to its content Same move as the section sidebar: the TOC column fits its longest link (capped at the docs' 268px, 150px floor), so the full three-column frame starts at 860px of pane width instead of 1060. * Open external links from pages in a new tab The preview bootstrap cancelled every non-anchor click, so an external link (the Sim docs, a vendor page) did nothing. External http(s) links now compile with target=_blank rel=noopener for the standalone and share surfaces, and the sandboxed preview bridges the click to the host, which window.opens a new tab — same channel the workspace deep links use. * Lock Sim pages to the rendered view via an internal record type The record's contentType is stamped text/x-sim-page when apply_file_edit detects page source — the file stays .html to the user (serving and downloads still emit text/html), but every surface now knows what the file holds before content loads. The viewer forces the rendered view for these files at every moment: the first streamed chunk (whose frontmatter is still partial) no longer flashes raw source, the gaps between an agent's tool calls no longer flip back to raw HTML, and both toggle surfaces (the Files toolbar and the resource-panel tabs) stop offering a code view for them. Mid-stream compiles run lenient — a fence still being written is malformed by definition, so its skip-notice callout is suppressed until the stream settles. * Honor the model's declared page type; default copilot .html to a page An explicit contentType on create_empty_file always wins (the skill now declares text/x-sim-page for pages, text/html for bespoke raw pages); with no declaration a copilot-created .html defaults to the page type. The first apply_file_edit still re-confirms from the actual content, and the category map knows the internal mime explicitly instead of falling through to the extension. * Default undeclared .html back to plain text/html A file is a Sim page only when the model declares it at creation or the first written content proves it — never by extension alone. * Match the docs' PageFooter for page navigation The invented bordered cards with Previous/Next labels are replaced by the docs' actual footer: the destination name with a 14px emcn chevron on a flex-1 hover pill (rounded-lg, px-3 py-3, --surface-active), next right-justified, and a spacer holding the empty half — verified against apps/docs/components/docs-layout/page-footer.tsx. * Scroll the rails invisibly, like the docs The sticky TOC box is overflow-y auto, and the clerk track's absolutely positioned SVGs could tip it a few pixels into overflow — Chromium then painted a full scrollbar beside the rail. Rails now hide their scrollbar chrome entirely (scrollbar-width none + webkit display none), matching how the docs scroll their sidebar and TOC. * Send sim:file links to the Files page, like a markdown link A workspace-file link in a page now navigates exactly as one tagged in a .md does: an in-app SPA push to /workspace/{ws}/files/{id} (the Files page with the file open). The fullscreen /view route stays reserved for the standalone surface; image refs keep the /api/files/view byte route. * Inline workspace images after the page compiles, not before The blob substitution ran on the raw source, where the compiled /api/files/view src it looks for does not exist yet — so the sandboxed cookie-less frame fetched every image itself and got 401s (broken image icons). The substitution now runs on the built document, covering compiled pages, legacy stored-compiled pages, and bespoke HTML alike. * Highlight the section you are AT in the left rail, not the last one visible The rail's current-section pick walked every heading visible in the viewport and kept the last h2 — so clicking a section landed correctly but highlighted whichever later section peeked in from below. Current is now the last h2 at or above the top reading line (matching the 72px scroll-padding a clicked anchor settles at), falling back to the first visible section when everything is below the line. * Stop the TOC jittering sideways as the highlight moves Active TOC links step from weight 430 to 470, and the rail is a fit-content column — every active-section change re-measured the longest link and shifted the rail a pixel or two side to side. Each link now carries a hidden zero-height ghost of itself at the active weight, so the column always occupies its bold width and the highlight moves without the layout moving. * Absolutize links and images in served page documents A downloaded page must behave like a downloaded .md whose links are absolute: clicking a workflow reference opens Sim in the browser at that workflow. The standalone renderer (plain download, fullscreen viewer, shares) now compiles sim: links and workspace image refs against getBaseUrl(); in-app surfaces keep relative paths and SPA navigation. * Drop the eyebrow; add the docs' top page controls The docs have no eyebrow line, so compiled pages no longer render one (old sources still parse; the field is ignored). The title row gains the docs' top controls: Copy page (copies the page text) and prev/next chevrons wired to the same neighbors as the footer cards, disabled-dim when a side is missing. * Read kv keys and table first columns as labels, the docs way kv keys dropped the blanket monospace — they render as the docs' row labels (500, primary, sans), with backticks in the source opting a code-like key (a path, an env var) into the inline-code chip; keys now run through inline markdown to make that work. Table body first columns pick up the same label treatment the docs tables show. * Platform font and emcn chrome for pages Pages live inside the app, and the platform — emcn, every workspace surface — renders the system stack, not the docs' Inter webfont; Inter made pages read as foreign next to the app around them. The face is now the platform stack with weights on the platform scale (400/500/600), the Inter delivery machinery (page-font.ts, the preview data-URI fetch, the public woff2) is gone, and the section filter wears emcn ChipInput's exact chrome (30px rounded-lg, --surface-5 fill flipping to --surface-4 in dark, --border, 14px, no focus ring). The docs' geometry — layout, spacing, rails, tables, code frames — is unchanged. * Color-only active state in the TOC — width can never move again Two attempts at reserving the bold width (a hidden ghost, then freezing measured rail widths) each traded one artifact for another: the ghost did not stop the fit-content column re-measuring, and the freeze made a long link wrap to two lines when it gained weight. The root cause was letting the active state change a width-affecting property at all: the active TOC link now shifts color only (muted to primary — the clerk thumb already carries the emphasis), the search input is a plain text field (the native search clear button is not emcn chrome), and the ghost/freeze machinery is gone. * Drop the Copy page control; keep the top chevrons The title-row actions keep only the previous/next chevrons (rendered when the page has neighbors); the Copy page button is gone. * Set-level sidebar: docs-style groups with the current page expanded Multi-page sets can now carry the whole set's sidebar. nav frontmatter (groups of labelled page links, identical on every page of the set) compiles into hidden set-nav markup whose sim: links resolve like any other; the shell lifts it into the left rail as muted group labels over page links, recognises the current page by title, and nests that page's section list beneath it — the docs sidebar's exact shape. Pages without nav keep the plain section list. * Center the content column at the docs' measure On wide panes the 1fr center cell stretched, so content hugged the left rail with dead space before the TOC. The main column now caps at the docs' ~760px measure and centers in its cell, matching how the docs balance a wide viewport. * Docs Steps, code-tab groups, and API method chips Three docs components join the vocabulary: sim:steps renders the numbered timeline (muted circle markers, hairline connector, title and content per step); sim:tabs renders the docs' grouped code block (mono tab chips, one pane at a time, the icon copy control targeting the visible pane); and a METHOD prefix on a set-nav page entry renders the API-reference chip — sidebar entries only, on the platform badge tokens (blue and purple added to the mirrors and the live bridge). Also: preview images switched from blob: to data: URIs — blob URLs are origin-bound and the sandboxed frame's origin is opaque, so Chromium refused to render them — and the page view-lock is sticky per file so a patch stream cannot flash raw source. * Downloaded pages carry their images Absolute URLs made LINKS survive a download, but an embedded image request from a downloaded file is cross-site and carries no session cookie, so images 401ed outside the app. The standalone renderer now inlines every workspace image the page references as a data: URI at serve time — like a pdf carrying its images — capped at 8MB per image, restricted to the page's own workspace, falling back to the URL reference on any miss. Applies to serve, download, and public shares. * Dead-center the content column between equal gutters The rails are content-sized and unequal, so the old grid (fit-content / 1fr / fit-content) skewed the middle cell toward whichever rail was narrower. The wide tier now uses the docs' geometry: a fixed 760px content column centered between two equal flexible gutters, the sidebar hugging the container's left edge and the TOC its right — rail widths can no longer move the content. * Defer title-bar history state out of currententrychange dispatch The Navigation API fires currententrychange synchronously from the history mutation that caused it, which can originate inside another component's useInsertionEffect (style libraries navigating during commit) — setState there trips React's 'useInsertionEffect must not schedule updates'. The arrow-state sync now defers to a microtask, flushing after the commit unwinds, with a disposal guard. * Show the section sidebar only when there are sections to list Fewer than two sections (and no set nav): the left rail and its filter disappear and the reserved left gutter collapses — the content column leads the container with the TOC trailing. Two or more sections, or a multi-page set, keep the full centered docs frame. * Execute the page shell in a DOM harness jsdom runs the real shell against compiled pages and asserts the layout decisions: both rails on a many-section page, only the left rail dropped on a one-section page. * Medium panes keep the TOC, not the sidebar Between 560 and 860px the frame showed the section sidebar and hid the clerk TOC — backwards by our own reasoning, since the sidebar is the redundant list on a single page. The TOC now survives at medium widths and the sidebar joins only on wide panes. * Sidebar is for doc sets only; stagger the rails like the docs The left rail now exists only for multi-page sets — on a lone page it just repeated the TOC. A set opens its sidebar at 560px with the TOC joining on wide panes (the docs stagger); a lone page waits until 700px and then shows the TOC alone. Set-sidebar spacing tightens to the docs values, with the current page's nested sections styled as small muted entries behind a hairline instead of full chips. * Center the lone-page content and TOC as a pair On a page with no sidebar the content column stretched while the TOC hugged the far edge, leaving a field of dead space between them. The content now caps at reading width with the TOC directly beside it and the pair centered in the pane, and the TOC waits until 800px so narrow panes stay single-column a while longer. * Equalize grid-template specificity across the rail tiers The 560px tier selects .art-cols:not(.no-side-nav) at (0,2,0), so the 860px tier's bare .art-cols template at (0,1,0) could never win and a set page on a wide pane kept the 2-column template — wrapping the TOC to the next grid row, bottom-left. The wide template now carries the same :not() guard, the 860 block's duplicate of the 800px no-side-nav rules is gone, and a test pins every template rule to equal specificity so a future tier can't silently lose the cascade again. * extract_doc_assets: pull a reference deck's assets into the workspace Sim-side handler for the new file-agent tool: given an uploaded .pptx or .docx, unzip it (OOXML is a zip), parse theme1.xml into theme.json (color scheme as hex, major/minor fonts, slide size from presentation.xml) and write every ppt|word/media file into a "<Name> assets" folder with original bytes and real content types. Re-runs overwrite the set in place. Pure extractor unit-tested against in-test-built packages; display label "Extracting assets from <file>". * extract_doc_assets learns .pdf via the doc sandbox PDFs have no zip structure or declared theme, so extraction runs in the same vetted sandbox that compiles and renders documents: poppler's pdfimages dumps every embedded image in its native format (masks filtered via -list), pdfplumber contributes each image's placement rects in page points plus the document's font names, and rendered pages are sampled into an explicitly-inferred color palette. theme.json for a pdf carries fonts, page size and count, the inferred palette, and a per-asset placement map. * Pages have one navigation rail; compile errors go to the agent The left sidebar leaves the renderer and the DSL: the shell builds only the content column and the clerk TOC (pair centered at 800px, one bare .art-cols selector per tier so the cascade cannot invert), the filter box goes with it, and nav frontmatter is tolerated but no longer rendered — sidebar METHOD chips and the set-nav markup are gone. Malformed sim: blocks no longer render a reader-facing "block was skipped" card: the block is omitted and the failure is reported as a diagnostic that apply_file_edit appends to its result, so the authoring agent sees exactly which fence to fix. The lenient flag existed only to suppress those cards mid-stream and is removed. The steps timeline connector now derives its position from the marker size, so it stays centered under the number circles. * Sync tool catalog: extract_doc_assets accepts pdf * Asset extraction yields the rebuild recipe, not just the parts pptx: theme.json now maps every image to its slide-by-slide placements (slide rels resolve rIds to media names; each pic frame's EMU offset and extent convert to inches) plus the slide count. pdf: a second layout.json is written — per page, the text blocks with content, position, font, size, and fill color; the filled rects (backgrounds and scrims); and rect-over-image overlay detection with coverage, which is the "image opacity" effect decks fake with a tinted rect. Stream alpha is unrecoverable, so overlays name the color and the rendered page remains the reference for strength. * Split shared-baseline text runs into separate blocks Two text boxes sitting at the same height merged into one wide line; a gap much wider than a space now starts a new block, so columns and label/value pairs land as distinct entries in layout.json. * Extract faithful document layout recipes * Add .chart files: live interactive ECharts docs, static or table-backed * Size charts by width-driven aspect, not panel height; separate title and legend * Map table chart rows from storage column ids to display names * Inject table rows as datasetIndex 0 so specs can transform; stagger array legends * Give .chart files their own bar-chart icon * Chart table sources gain groupBy/aggregate/pivot shaping; renderer-owned chrome * sim:chart page fence: ECharts SSR to themed inline SVG; shared option builder * sim:chart hydrated embeds: inline or .chart file refs, live table reads per serve * Finish extensionless pages and document staging * Render live charts without server hydration * Cover active-theme page token overrides * Keep artifact tokens synced with the app theme * Use tabs for multi-page Sim docs * Preserve dollar-prefixed tool credentials * Build chart specs from validated fields * Sync new integration docs into Copilot manifest * Add in-document tabs to Sim pages * Rebuild page TOC on tab changes * Keep page tabs with docs chrome * Stabilize tabbed page layout * Regenerate docs manifest for staging's Modal docs page * Share one divider between the bar and the chrome tab row * Pin the keyless OCR path in the unreadable-document test --------- Co-authored-by: Justin Blumencranz <96924014+j15z@users.noreply.github.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>	2026-08-20 14:05:46 -07:00
Vikhyath MondretiandClaude Opus 5	97c1688c49	feat(modal): add Modal Labs integration (#6896) ... * feat(modal): add Modal Labs integration Modal has no public REST control plane — the Python/JS/Go SDKs all speak gRPC — so this covers the two surfaces that are reachable over HTTP: deployed Web Functions/Servers, and the OpenAI-compatible Endpoints API. Three operations: call a deployed function with proxy-token auth, generate a chat completion on an Endpoint, and list the models a token can reach. Auth sends the token pair as Modal-Key/Modal-Secret rather than the combined bearer form, so a Web Function that validates its own bearer token keeps the Authorization header free. Both URL fields require https since Modal terminates TLS everywhere, and a cleartext URL would leak the token. Chat completion declares request.modelInput so the system prompt and user message project to canonical placeholders before egress. Call Function deliberately does not — a Web Function runs arbitrary user code, and nothing proves its body reaches a model. /v1/models fields beyond `id` are inferred from OpenAI compatibility rather than printed in Modal's docs, so they are marked optional. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(modal): type the wire payloads and default chat to the shared endpoint Chat Completion required an endpoint URL and passed a blank one straight into modalOpenAiUrl, which throws — while List Models already fell back to the shared inference host and the generate-on-modal-endpoint skill tells agents to leave the field empty for Shared Endpoints. Skill-driven chat calls against the shared host failed instead of using that default. Chat now falls back the same way and the block field is no longer required. Replaces every `any` in the Modal tools with declared wire types for the OpenAI-compatible /v1 payloads. Fields stay optional because the shape comes from whichever inference engine backs the endpoint, so the readers keep their defensive `??` guards — the types exist so a future change to that mapping fails the compiler instead of shipping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-20 13:12:34 -07:00
Theodore Li	4214a891f4	fix(setup): publish unscoped setup package (#6886) ... * fix(setup): publish unscoped setup package * fix(setup): strip renamed status command	2026-08-20 02:11:06 -04:00
Waleed	f6a9f0dd87	fix(integrations): white CB Insights tile and a borderless Crunchbase mark (#6884) ... CB Insights moves from a dark navy tile to white, matching Jira, Confluence, and Bitbucket. Its icon carries its own fills, so it stays legible on the lighter tile. The Crunchbase icon drops the white rounded-square plate and its border, leaving just the `cb` mark on `currentColor` so the block's bgColor supplies the tile. The viewBox is retargeted to the glyph's true curve extrema with padding that keeps it at the same optical weight as the surrounding brand marks.	2026-08-19 22:34:17 -07:00
3 people	e3a4874ece	feat(integrations): add Bitbucket Cloud (#6860) ... * feat(integrations): add Bitbucket Cloud * fix(bitbucket): enforce selector workspace slugs * fix(bitbucket): overfetch small pipeline log tails * fix(bitbucket): harden provider edge cases * fix(bitbucket): accept provider diff redirect specs * fix(bitbucket): stop advanced-field leakage and harden log, status, and selector paths Splits the `closeSourceBranch` advanced subBlock into per-operation ids. Advanced fields serialize without evaluating their condition, so a value set on Create Pull Request reached Merge Pull Request and closed the source branch unprompted. Also: - read step logs through the byte-capped server transport and map an empty-log 416 to an empty result, keeping a genuine 416 an error - trim a step log's partial leading line after the character cap rather than before, and never return an empty log when the retained window held content - surface Bitbucket's `error.detail` alongside `error.message` - treat commit-status `key`/`state` as nullable so one malformed row cannot drop a page - match repository `full_name` case-insensitively and reject dot segments in a workspace slug before the outbound request - type `reviewerAccountIds` as the comma-separated string it is - trim optional Bitbucket query strings; correct the token lifetime to two hours --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-08-19 20:32:44 -07:00
Waleed	9f346765fe	feat(granola): complete API coverage, note triggers, and connector validation (#6880) ... * feat(granola): complete API coverage, note triggers, and validation fixes Granola's public API exposes nine endpoints; Sim implemented three. Adds the remaining six and wires the new programmatic webhook-endpoint lifecycle into a managed trigger. Tools (6 new, 9 total): - get_transcript, list_audit_events - create/list/update/delete_webhook_endpoint Triggers: note.generated, note.edited, note.access_granted, plus an all-events trigger. The provider handler registers the Granola endpoint on deploy and deletes it on undeploy, scoped to the trigger's own event names, and verifies every delivery with the Standard Webhooks HMAC-SHA256 signature Granola returns on creation. event_id is the idempotency key, which Granola reuses across retries. Validation fixes to the shipped tools: - get_note dropped speaker.attribution ("me"/"them"); now surfaced - a 413 on get_note now explains that the transcript is too large inline and points at get_transcript, instead of surfacing a bare status code - note IDs are URL-encoded rather than interpolated raw - base URL, auth headers, and status-aware error handling are shared runtime helpers; params/outputs stay literal per file so the docs generator still reads them Tests cover signature verification (including replay and body-tamper rejection), event matching, subscription create/delete, and the block/tool contract — plus a guard that ids shared between the tool and trigger surfaces seed the same default, since block state is keyed by id and last-wins. The knowledge-base connector was validated against the spec and needed no changes. * fix(granola): correct array output schemas, listing-truncation signal, and docs Findings from validation passes over the tools, trigger, and connector. Tools — array outputs were declared as `type: 'json'` with `properties`, which describes an object, not an array. Agents and the output picker therefore saw `notes.title` instead of `notes[i].title`. All 15 array outputs (including the pre-existing three tools) now use `type: 'array'` with `items`, matching the 2000+ other tool files. The audit event `data` field stays `json`; it is genuinely free-form per the spec. Connector — `hasMore` was ANDed with the cursor, so a `hasMore: true` response with no cursor was reported as a complete listing. The sync engine treats exactly that shape as truncated and sets `listingTruncated` to block deletion reconciliation; masking it meant a partial first page could be taken for the whole corpus and reconciliation would hard-delete every note past it. Granola would have to violate its own contract to emit that shape, but the engine already handles it and the connector was hiding the signal. Also aligns mimeType with the `.txt`/text-plain bytes the engine actually writes (it was the only connector of 101 claiming text/markdown). Trigger — the setup instructions named a Granola settings path that does not exist; the help center says Settings > Connectors > API keys in the desktop app. Both list parsers now split commas inside array entries, so an array-wrapped free-text value cannot be sent as one malformed identifier. Block — `id`, `events`, and `hasMore` are produced by several operations but their descriptions named only one, unlike `folders` which already documented both meanings. Adds connector tests pinning all four listingCapped quadrants and the truncation signal, and tool tests for the list parser and the PATCH body's per-field "omit means unchanged" semantics. * fix(granola): clean up webhook endpoints created by a failed registration Raised independently by both reviewers. The registration service only rolls external state back when createSubscription *returns* — its rollback is guarded on `preparedProviderConfig`, so a handler that throws is assumed to have left nothing behind. Granola's handler broke that contract: when Granola accepted the POST but the success body was missing `id` or `signing_secret` (including a body that failed to parse and became `{}`), it threw with the endpoint already live. Nothing then recorded an external id, so undeploy could not remove it, and Granola kept delivering to a callback whose signature could never be verified — duplicating on every deploy retry. The handler now removes what it created before rethrowing, matching the pattern grain's multi-hook create already uses. It deletes by id when Granola returned one, and otherwise recovers the endpoint by matching the callback URL, which also covers a connection that fails after the request reached Granola. Endpoints whose URL was redacted to its origin are never matched — that comparison could delete another workflow's endpoint on the same host. Cleanup is best effort and never masks the original failure. A non-2xx is left alone, since no endpoint was created. Also folds the delete call shared with deleteSubscription into one helper. * fix(granola): never recover an orphaned endpoint by callback URL The previous commit's URL-based recovery was unsafe. A redeploy reuses the live registration's `path`, so the candidate and the currently serving endpoint share a callback URL — listing by that URL and deleting every match would remove the live deployment's endpoint and silently stop a working trigger, which is worse than the leak it was trying to prevent. Cleanup is now keyed solely on the id Granola returned. When the success body carries no id there is no way to tell the candidate's endpoint from the live one, so it is left in place: a leaked endpoint produces unverifiable deliveries that Granola disables on its own, whereas deleting the wrong one takes down live traffic with no signal. The 2xx-missing-signing-secret case this originally fixed still cleans up, since that response does carry an id. Adds a test asserting no lookup or delete is attempted when the response has no id, so URL matching cannot be reintroduced unnoticed.	2026-08-19 19:02:27 -07:00
Waleed	f17938c09e	feat(cbinsights): add CB Insights API v2 integration (#6879) ... * feat(cbinsights): add CB Insights API v2 integration Covers every non-streaming v2 endpoint across 25 tools: free organization lookup, firmographics search, funding rounds and cap tables, investments, portfolio exits, business relationships, management and board, the Mosaic / Commercial Maturity / Exit Probability outlooks and their histories, funding windows, revenue, strategy maps, Scouting Reports, ChatCBI, and RAG context. CB Insights authorizes by client-credential exchange rather than a static key, so the tools run through directExecution: the shared executor trades the credentials for a bearer token, caches it briefly, and re-authorizes once on a 401 — the token lifetime is undocumented, so expiry is discovered rather than predicted. ChatCBI and RAG declare request.modelInput so an activated Sim secret in the message is projected to its canonical label before reaching a third party's model. directExecution still runs projectToolModelInputParams, so the two are compatible. The two streaming endpoints are deliberately excluded; they deliver incremental JSON chunks and their non-streaming counterparts return the same content in one piece. * fix(cbinsights): reject malformed ID lists and bound the token cache - Reject an organization ID list containing an invalid entry instead of dropping it. Silently filtering meant a typo ran the request against a narrower set — spending credits on the wrong organizations, or quietly widening a filtered search — and still reported success. - Apply the same rule to the optional firmographics ID filters, where a dropped filter broadens the search rather than narrowing it. - Bound the process-wide token cache so a long-lived worker serving many CB Insights accounts does not grow with the cumulative number of accounts seen. Expired entries are swept on write, then the oldest evicted. * fix(cbinsights): stop paging and blank input bypassing the search guards - Measure the firmographics empty-search guard against the filters alone. limit, nextPageToken, and sort were in the same object, so a request carrying only paging slipped past it and issued an unfiltered search over the whole database — which still spends credits. - Reject a mistyped numeric bound instead of dropping it. A bad headcount, funding, or valuation filter silently widened the search, the same failure mode already fixed for ID lists. - Treat an empty comma segment identically on the required and optional paths. A trailing or doubled comma is a separator artifact that cannot change which records are requested, so both paths now discard it; every other malformed entry is still rejected. * fix(cbinsights): accept only plain decimal organization IDs Number reads "0x10" as 16 and "1e2" as 100, so either notation resolved to a real but unintended organization and the request spent credits on it. Both the path-scoped and the bulk validators now require a plain run of digits, and use Number.isSafeInteger so an ID past the precision limit cannot round to a neighbouring one. * fix(cbinsights): bound a numeric organization ID to the safe-integer range The string path already required a safe integer; the numeric path still used Number.isInteger, which accepts a value past the precision limit. JSON parsing has already rounded such a value, so the request would target a different organization than the caller supplied.	2026-08-19 18:31:14 -07:00
Waleed	a9cf760c0f	feat(pitchbook): add PitchBook integration (#6876)	2026-08-19 18:03:25 -07:00
Waleed	40aa8ad5eb	feat(crunchbase): add Crunchbase Data API integration (#6875) ... * feat(crunchbase): add Crunchbase Data API integration Covers the v4 Data API end to end: dedicated search and lookup operations for organizations, people, funding rounds, and acquisitions, plus generic collection-parameterized search and lookup reaching the remaining 39 collections, single-card paging, autocomplete, the deleted-entity feed, and fields metadata. Adds a crunchbase-errors extractor: the API answers failures with a bare JSON array, which no existing extractor reads, so an auth or predicate failure would have reported only its HTTP status. * fix(crunchbase): honor card paging limits and cursor exclusivity - Cap a card page at the documented 100-item maximum instead of Search's 1000, which the shared Limit field made easy to carry over - Always request the card's identifier so a narrowed cardFieldIds cannot return a full page with a null cursor and stall a paging loop - Reject the mutually-exclusive afterId/beforeId pair on the card and deleted-entity endpoints, not just on search - Report an unexpected card shape as empty rather than wrapping the envelope as a one-row page	2026-08-19 17:23:20 -07:00
Theodore Li	1372977d07	feat(setup): publish standalone self-hosting package (#6849) ... * feat(setup): publish standalone self-hosting package * fix(setup): refresh discovered compose installs * improvement(setup): unify repository command * fix(setup): harden standalone package launch * Update README.md * fix(setup): isolate standalone compose installs * fix(setup): restore default stopped installs	2026-08-19 20:04:05 -04:00
Waleed	4f9d5f33b0	improvement(search): search every folder, and document real API error bodies (#6861) ... * improvement(search): search every folder, and document real API error bodies Search on Files, Tables, and Knowledge was ANDed with the open folder, so a query only ever matched that folder's direct children — and the query was not cleared when you entered a folder, filtering the folder you just opened down to the same matches. A non-empty query now searches the whole workspace, a Location column names each result's folder, and opening a folder ends the search. Also gives GET /api/v2/files a `recursive` flag, and replaces the single shared OpenAPI error example — which showed `BAD_REQUEST` under every status tab — with one real body per status. * fix(search): discard the search term on clear instead of masking it `useSearchFilterValue` returned the debounced term whenever the input was non-empty, so clearing only hid the settled needle. The mask lifted on the next keystroke while the debounce still held the pre-clear term — opening a folder and typing within the window searched the whole workspace for the query the user had just abandoned. A clear now resets the settled term rather than hiding it, adjusted during render so the reset is visible to the render that follows the clear. The initial state is seeded from the first value so a deep-linked `?search=` still filters on the first render.	2026-08-19 13:55:53 -07:00
Vikhyath MondretiandClaude Opus 5	521348b529	feat(secrets): record which secrets each run resolves, and surface it per secret (#6823) ... * feat(secrets): record which secrets each run resolves and surface it per secret Redaction stops a value at a boundary but cannot stop code that never emits it — a Function block can print a key one character at a time and nothing ever matches the secret. That is undecidable in general, so this adds the other half of the posture: attribution. Every run now records which configured secrets it actually resolved, under whose identity, through which surface (workflow, Sim agent, MCP). The data already existed in ResolvedSecretTraceRegistry.addActiveEntry and was persisted only for paused runs; this persists it for every terminal path. Execution logs cannot answer this. They store the whole available encrypted environment rather than what a run referenced, they evidence a secret only where value-matching redaction happened to fire, and they expire under logRetentionHours — while "who has touched this key" outlives any single run. - secret_usage: per-UTC-day rollup keyed by workspace, secret, scope, owner, source, workflow, actor. A one-minute schedule touching three secrets would otherwise write thousands of rows a day, which is also why this is not audit_log. workflow_id/actor_user_id use '' sentinels rather than null so the unique key works on Postgres 14 without NULLS NOT DISTINCT, and are not FKs: they are historical facts, and an onDelete would rewrite a key column. - secret_owner_user_id is part of the key. Two people can hold a personal secret under one name and a shared personal secret resolves for a caller who does not own it, so name and scope alone do not identify a secret. It is NOT the actor: a scheduled run resolves the workflow owner's personal slice under the workspace's execution actor. - Direct environment reads are now detected in JS (TypeScript AST), Python (tokenizer-checked) and shell (quote/heredoc-scanned), so a secret read as environmentVariables['K'] or $K enters the run's provenance instead of going unredacted. Each detector prescans for names that are actually configured secrets before paying for a lex or quote-frame pass. - Copilot integration tool calls are covered: resolveCopilotEnvReferences substitutes {{SECRET}} into user-only params, which is a real use. - See usage lives behind a credential-admin gate, using the same predicate that reveals the value; members get a disabled chip explaining why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(audit): register the secret-usage route in the validation baseline Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): keep rollup metadata with its run, and stop shadowed bindings faking usage Review round 1. - record.ts: last_execution_id/last_trigger were assigned unconditionally while last_used_at was chosen by greatest(), so two runs completing out of order split one row between them — the newer run's timestamp beside the older run's execution id, making "View log" open a run the row does not describe. Both are now guarded on the timestamp actually advancing, so the row's metadata always belongs to the run that owns its timestamp. - javascript.ts: a local binding named environmentVariables (declaration, parameter, destructured binding, or bare reassignment) made reads off the user's own object look like mounted-secret reads. Any such binding now disables detection for the file; the AST already had parent pointers, so this is a kind check during the existing walk. - python.ts: same class of bug with no parser available, so the rule is an allowlist — every mention of the binding must be a literal subscript or .get(), otherwise detection is off for the file. This also subsumes the cross-line attribute case (other.\n environmentVariables['K']), which the previous space-and-tab look-behind missed. Under-reporting is the safe direction here: a trail that claims a use that never happened is worse than one that misses a use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(db): format the generated migration snapshot CI runs lint:check across every workspace; the drizzle-kit output in packages/db had never been through biome, so the branch was green locally (where lint had only been run inside apps/sim) and red on CI. Whitespace only — both files are byte-for-byte identical once parsed, and drizzle-kit still reports no pending schema diff against the reformatted snapshot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): detect every rebinding of the environment identifier, not just declarations Review round 2. A bare `for (environmentVariables of rows)` has no declaration to key off, so the previous check missed it and reads of the loop value were still recorded as secret usage. Rather than extend the hand-rolled node-kind list, this reuses the pair the same file already applies to reject a placeholder in a write position: isDeclarationIdentifier covers declarations, parameters, destructured bindings and imports, and isWriteIdentifier covers every assignment operator, ++/--, destructuring targets, and for-in / for-of initializers. That also closes four forms neither the review nor the original check named: logical (||=) and nullish (??=) assignment, and object and array destructuring assignment. Six of the eight added cases fail against the previous check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): apply the rebinding rule to shell, and say when a run's log is gone Review round 3, plus the docs that were left claiming the old behavior. - shell.ts: a script that writes a configured name (API_KEY=local, export/local/ readonly, read, for, unset) expands its own value from that point on, not the mounted secret, so recording it claimed a use that never happened. Every mention of the name must now be a `$NAME` / `${NAME}` expansion, matching the allowlist shape the Python detector already uses. Applied per name rather than per file: JavaScript and Python shadow one object holding every secret, whereas rebinding one shell variable says nothing about the rest. - The usage trail deliberately outlives execution logs, so a row routinely names a run whose log has been pruned. The read now left-joins workflow_execution_logs on its unique execution_id and reports availability, and the panel renders the chip disabled with the platform tooltip instead of linking into an empty Logs view. Three states: no run to link, a run whose log is gone, and a live link. - Docs said a direct environmentVariables/$KEY read does not activate masking, which this branch changes. Corrected in credentials.mdx, function.mdx and the logging FAQ, and the recognition limits are now written down: runtime-built names, reassigned bindings, and reads that cannot be told apart from text. Added a "See usage" section covering who can see it and why an empty trail means "nothing recognized" rather than "never used". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): writing a name is not reading it, and a bare mention is not a rebinding Review round 4. - javascript.ts / python.ts: `environmentVariables.API_KEY = 'x'` and `delete environmentVariables.API_KEY` touch the name without ever reading the mounted value, but the detectors matched the member access and recorded a use that never happened. JavaScript now asks the same isWriteIdentifier the placeholder rewriter uses (its parameter is widened to ts.Node — the body already walked generic nodes, so this is a type change, not a behaviour one) plus a delete check; Python excludes a subscript followed by `=` and a `del` target. - shell.ts: requiring every mention of a name to be an expansion also fired on text that binds nothing — a comment naming the key, or `echo "API_KEY=$API_KEY"` where the literal is an argument rather than an assignment — and dropping those cost masking on a genuine read. It now looks for actual writes: an assignment at command-word position, a binding builtin, `printf -v`, or a `for` target. The two directions are not symmetric, which is why this errs toward detecting the read: missing a write records a use of a secret the script only had in its environment, a misleading audit row and nothing more, since masking still searches for the real value and will not find it. Over-detecting a write suppresses masking on a value that does reach the log. This also makes the code match what the docs already described — skipping after a rebinding, not after any mention. 13 tests added; 11 fail against the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): an update reads before it stores, and a del target may be parenthesized Review round 5. The first of these is a regression from round 4. - javascript.ts: reusing isWriteIdentifier to answer "is this a read" was wrong. That predicate answers the rewriter's question — is this a target the substitution must refuse — so it treats every assignment operator alike, which is correct there and wrong here: `+=`, `||=`, `??=`, `++` and `--` all load the current value before storing, so they are genuine reads and were silently losing their masking. Only a plain `=` stores without reading. Replaced with a purpose-named predicate, and isWriteIdentifier's parameter is narrowed back to ts.Identifier now that nothing else needs it widened. A test committed last round asserted the wrong behaviour for `+=`; it has been corrected rather than left to pin the bug. - python.ts: `del (environmentVariables['K'])` slipped past a check that looked only at the characters immediately before the match. It now isolates the enclosing logical line and tests whether that is a del statement, which also covers `del((x))`, `del(x)`, `del a, x`, and a del after a semicolon. 12 tests added or corrected; 10 fail against the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): stop excluding Python writes, which kept leaking in the unsafe direction Review round 6. Greptile found that `del environmentVariables[environmentVariables['K']]` had its inner access — which computes a key, so it is a genuine read — skipped along with the delete, leaving that value unmasked. The narrow fix was another textual rule. Instead this removes the write and delete exclusions from the Python detector entirely, because they were optimizing the wrong direction. `resolvedSecretNames` feeds `outputSecretMatcher`, an exact-value matcher over the output. Naming a secret the code never read costs nothing there: the matcher scans for a value that does not appear. Failing to name one that was read leaves it unmasked. The two error directions are therefore not comparable, and the exclusions bought only audit-trail tidiness while every heuristic they needed has so far leaked into the dangerous side — first a parenthesized target, now a nested read. A `del` or an assignment is reported like any other access. JavaScript keeps its exclusion: a real AST answers the question per node, with no text to misread, and it has produced no such hole. Net 30 lines removed from python.ts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): report recognized reads instead of proving they are not reads Review round 7. Greptile flagged both directions at once — false usage from reporting a write target, and unmasked secrets from the file-wide shadow flag — so I traced what the signal actually drives before choosing. The chain: the compiler's names feed outputSecretPlaintextsByName and the exact-value matcher, NOT context.resolvedSecretNames, which starts empty. After execution activateOutputSecretProvenance scans the output and adds only names whose plaintext actually appeared; those become __resolvedSecretNames, which tools/index.ts turns into recordResolved calls, which is what the usage trail reads. So a compile-time false positive produces no usage row on the ordinary path — it only hands the matcher a value the code never emits. It does produce one on the !projection.safe fallback, where the system already over-approximates by design. A false negative, by contrast, keeps the value out of the matcher entirely, so a genuinely read secret is never masked on any path. That asymmetry decides it, so every "prove this is not a read" mechanism is gone: - javascript.ts: the file-wide shadow flag. A helper declaring its own environmentVariables discarded genuine reads of the mounted binding everywhere else in the file — Greptile's security finding, and real. - python.ts: the allowlist requiring every mention to be a subscript or .get(). Same hole: passing the dict to a function suppressed unrelated reads. - shell.ts: the rebinding check. It had the same hole in a form nobody flagged — `echo "$API_KEY"; API_KEY=local` dropped the first read, which is of the real secret. What stays is the question of whether the text is code at all — strings, comments, single quotes, quoted heredocs — plus the receiver check that `other.environment Variables['K']` is a different object, and JavaScript's node-precise write/delete exclusion, which cannot suppress a read elsewhere. Net 215 lines removed across the three detectors and their tests. Docs updated: the rule is now stated as reporting rather than proving, and that See usage may occasionally list a secret the code had available but did not read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(secrets): drop the last write-vs-read special case `environmentVariables` is a plain object deserialized from the run payload (route.ts:206), not a handle on the stored secret. Assigning to it changes nothing outside the sandbox and is discarded when the run ends, so separating a write from a read bought almost nothing while leaving JavaScript as the one language still trying to prove a read is not a read. Every language now follows the same rule: report a recognized read of a configured secret name. The only exclusions left are facts rather than inferences — the text is not executable (string, comment, single quote, quoted heredoc), the receiver is a different object, or the name is not statically knowable. Docs note that assigning to the binding does not edit the secret. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(secrets): ship only the fields the trail actually shows Five fields crossed the API and reached no reader: usageDate, firstUsedAt, actorEmail, workflowId and actorUserId. The panel renders the timestamp, the trigger, what used the secret, the actor's name, the run count and the run link; everything else was projected, serialized and discarded. first_used_at is dropped from the table as well. Nothing read it, and inside a per-day bucket "first used that day" says nothing next to "last used that day" — so it was a column written on every run for no question anyone asks. The upsert loses its least() with it. Migration regenerated; the identifier columns behind the joins stay, they simply are not returned. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): report referenced code secrets, not only ones that surface in output The Function route activated a secret's provenance — and therefore its usage row and downstream masking — only when the exact value appeared in the result, stdout, or error. That gate made the trail miss silent use entirely: a key that authenticates an API call and is never echoed reported nothing, and so did the founding scenario of this feature, a key exfiltrated character by character. The innocent run that echoed a key got a row; the run worth catching did not. Activation now follows the referenced set the compiler already computes: resolved {{KEY}} bindings plus recognized direct reads, filtered to configured values — the same set the unsafe-projection fallback already activated. An extra name only hands the output matcher a value that never appears; configured-but-unreferenced values are still never included. The output-scan activation path and its surface helper are deleted rather than kept alongside. One old test pinned the gate ("does not activate a referenced secret that does not cross the Function result"); it now asserts the reverse, with the reasoning attached. Two new tests pin the char-split exfiltration and the silent API-call case. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): shell escaping is backslash parity, not adjacency Review round 8. `\\$API_KEY` is an escaped backslash followed by a LIVE expansion — bash prints `\` plus the value — while `\$API_KEY` is an escaped dollar and stays literal. Checking only the character adjacent to `$` read every even run as escaped, dropping a real read from usage and masking alike; verified against bash before fixing. The scanner now counts the run of backslashes before the `$` and skips only odd runs, the same parity rule logicalLineEndAfterContinuations in this file already applies to line continuations. Six-case parity table added; the three even-run cases fail against the previous check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): recognize destructured environment reads Review round 9. `const { API_KEY } = environmentVariables` delivers the value by name with no property- or element-access node in the AST, so the member-access walk missed it entirely — and a missed read leaves an emitted value unmasked, the dangerous direction. The AST walk now also recognizes the declaration form (shorthand, renames, defaults, string-literal keys), the assignment form ({ KEY } = env), and a ...rest element — which names no key but takes every value, so it reports every configured name; the alternative left `const { ...all } = env; return all` entirely unmasked. A computed key stays unrecognized, the same runtime-name boundary as a computed subscript, and a receiver that is not the bare identifier is not attributed. Nine cases added; the six positive ones fail against the previous walk. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): one receiver rule for destructured reads, parentheses included Review round 10. Two accurate findings, folded into a generalization instead of two more special cases: - A parameter default (function f({ API_KEY } = environmentVariables)) and a binding-element default are the same by-name delivery as a variable declaration. The detector now keys on the ObjectBindingPattern itself and checks its parent's initializer, so every declaration position follows one rule instead of per-kind arms. - Parentheses group without changing the receiver, so (environmentVariables) is unwrapped before the identifier check — in the destructuring arm AND the member-access arm, which had the same hole unreported. Declined the for-of-over-array-literal finding: the receiver there is a container, not the environment object, and following data flow through containers has no fixed point — the same documented boundary as aliasing and computed keys. A test pins the boundary so it reads as chosen, not missed. Eight cases added; the seven receiver-rule cases fail against the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): a dot in prose is not a qualifier, and a literal computed key is a subscript Review round 11. Both findings were implementation-narrower-than-rule, fixed by consulting authorities the detectors already had rather than adding new ones: - python.ts: the receiver walk crosses whitespace so a parenthesized `other.` on a previous line is seen — but it landed on a comment's final period (`# Load the value.`) and discarded the genuine read on the next line. The landing position is now checked against the same lexer ranges that filter the candidates, which is also why the receiver check moves after lexing. - javascript.ts: `const { ['API_KEY']: key } = environmentVariables` is the element-access rule in pattern position, so a computed key holding a string literal resolves like a literal subscript; any other computed key keeps the runtime-name boundary a computed subscript already has. Eight cases added; the comment-period case and all three literal-computed-key cases fail against the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-18 18:32:20 -07:00
Vikhyath MondretiandClaude Opus 5	3a03774e42	fix(forks): stop copying connector-managed knowledge base documents (#6818) ... * fix(forks): stop copying connector-managed knowledge base documents A fork copies a KB's documents but never its connectors, so a connector-sourced document arrives with `connector_id` nulled and its `external_id` intact. The sync engine keys every existing/tombstone/ exclusion lookup off `connector_id`, so that copy is invisible to it - never updated, reconciled, or purged - and `doc_connector_external_id_idx` does not constrain it either, since its `connector_id` is NULL. Attaching a connector in the child then re-ingests every page as a NEW row on top of the snapshot. Each fork hop re-copies the previous hop's orphans and adds one more generation, so a prod -> UAT -> staging chain leaves three rows per page and a knowledge search returns the same page three times, one of them serving content frozen at the fork date. Exclude connector-managed documents from all four doors a document can enter a fork through: the whole-KB content copy, the in-transaction placeholder pre-creation, the sync-only copy into an already-mapped KB, and the content fill (guarded for payloads planned by a pre-change worker mid-rollout). The placeholder path matters as much as the copy loop - filtering only the content phase would leave a permanently archived row behind a persisted `knowledge_document` mapping. Skipped on both sides, the reference clears like any other uncopied document's. A document whose connector was deleted already has a null `connector_id` (the FK is ON DELETE SET NULL) and is static in the source too, so it still copies. One count(*) per copied KB logs what was left behind, since a fully connector-synced KB now forks to zero documents. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): keep the skipped-document count from failing a copied KB The connector-managed count feeds a log line, but it sat inside the KB's try block, so a transient failure on a COUNT(*) would roll back a copy that had otherwise succeeded and clear every reference to it. Move it into a helper that swallows its own error. Counting is not copying: only the copy itself may fail a resource. Test proven red by removing the catch - the mutation reports a knowledge-base failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): clean up full-KB placeholders planned before the exclusion The mapped-KB fill guarded a pre-change plan, but the full-KB path did not: a placeholder planned by an old worker for a connector-managed document is simply no longer returned by the page query, so nothing fills it and it stays archived behind a live mapping that a remapped document-selector still resolves to. Report those child ids as failed documents so the shared cleanup clears their references and drops the rows, and delete their persisted identity so a later sync does not resolve to a row cleanup removes. Keyed on the SOURCE being connector-managed, which can never become copyable, so it cannot race a concurrent attempt mid-fill the way a "source is gone" check could. The mapping drop is now one helper shared with the mapped-KB catch. Test proven red by removing the reconciliation block. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): make the stale-plan probe best-effort The probe ran inside the KB try, so a transient SELECT would reach the catch, roll back a complete copy, delete the child base, and clear every reference to it. Weighing it as "load-bearing, so fail closed" was wrong: the probe runs on EVERY copied KB that has referenced documents, while the state it repairs exists only inside a rollout window. Failing closed traded a common-path outage against a rare-squared one. It now swallows its own failure with a loud error log, leaving that pre-existing state in place rather than destroying a good copy. Test proven red by removing the catch - the mutation reports the KB failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-18 14:23:19 -07:00
Waleed	1c69372cba	feat(cli): follow a run, wait for one, and tail the log (#6813) ... * feat(cli): follow a run, wait for one, and tail the log Three commands the surface was missing, each polling or streaming something the generated command layer cannot express. `workflows run --follow` renders the SSE the execute route already emits, so a multi-minute agent run stops printing nothing until it ends. It rides on the generated `run` leaf rather than a sibling command — same operation, one different response encoding — and delegates to the handler it replaced, so every non-follow invocation still runs the generated path. Answer text, thinking and tool calls go to stderr; only the final envelope reaches stdout, so redirecting still yields the result. Reasoning and tool frames need the `X-Sim-Stream-Protocol` header, which is sent only when asked for, because negotiating also switches answer text to live chunks the server may retract. `workflows runs wait` closes the loop `--async` opens. Terminal is completed, failed or cancelled; `redacting` is not, since a run whose output is still being scrubbed is not yet a run you can read. A time pause keeps polling because the server resumes it, and a human pause stops with the resume command rather than burning the bound and calling it a timeout. Distinct exit codes keep cancelled and paused from reading as failure. The bound is `--wait-timeout` and not `--timeout`, because SIM_TIMEOUT_SECONDS already bounds one request and two knobs of the same name hide each other. `logs follow` tails runs as they arrive. Dedup keys on run id, not on the timestamp: a schedule fan-out starts many runs in the same millisecond, so a timestamp watermark either drops the siblings or reprints them. JSON output is one object per line, because a follow never closes an array, and the table header is printed once so columns stay aligned across polls. * fix(cli): disclose a truncated burst, and clear a stale retry notice Two review findings in `logs follow`, both verified against the code first. The page budget bounds one poll so an enormous burst cannot stall the follow, but on reaching it the live cursor was discarded: the remainder is older than everything collected and the next poll restarts at the newest page, so those runs were never printed and nothing said so. The budget stays — draining without one trades a bounded poll for unbounded buffering in a process meant to run for hours — but hitting it now warns on stderr, naming the count and pointing at `sim logs list`. That notice is written even off a terminal, because a piped log is where an unexplained hole is hardest to spot. The retry notice was cleared after the empty-rows check, so a poll that recovered but found nothing left "retrying in Ns…" on screen while the follow was already healthy. Clearing now happens as soon as a poll succeeds. The second test needed two failures to be worth anything: the teardown clears the line either way, so what separates fixed from broken is whether a bare erase lands before the second notice or only at the end. The first version passed against the bug. * test(cli): pin that a mixed page is the watermark, not a truncation A page holding a run already printed proves the follow caught up, so the truncation warning must not fire there — that is how every healthy poll terminates, and warning would report a hole on the ordinary path. The straggler sharing that page is still collected, because the filter takes every unprinted row on it rather than only those above the known one. * fix(cli): say when the requested backlog was larger than a page holds The logs API clamps `limit` into 1–1000 rather than rejecting it, so `logs follow -n 5000` came back with 1000 rows, anchored the floor to that partial page, and said nothing. The seed already knew — it computes whether a live cursor remained — but the caller discarded the answer. Guarded on both halves. Fewer rows than asked for is only a shortfall when more were waiting: a workspace holding ten runs answers `-n 50` with ten and nothing is missing, so warning on the row count alone would fire on every small workspace. The cursor is what separates the two.	2026-08-18 11:43:16 -07:00
Waleed	edc25aa976	docs(helm): document null as the way to remove an inherited env key (#6801) ... * docs(helm): document null as the way to remove an inherited env key Setting `app.env.KEY: ""` cannot clear a key that `app.envDefaults` sets: the Secret template drops empty values, and the deployment template treats an empty override as "not overridden" and still inlines the default. Helm's own `KEY: null` deletion is the supported mechanism and already works. The empty-string behavior is load-bearing, not a bug — every key under `app.env` ships as a "" placeholder, and ten collide with a real `envDefaults` value (NEXT_PUBLIC_APP_URL, BETTER_AUTH_URL, ...), so "" has to read as "unspecified" or a default install would blank them out. - README: document `null`, with the --reuse-values and Argo CD valuesObject caveats; correct the claim that `app.env` always wins over `app.envDefaults` - values.yaml + self-hosting docs: same guidance where operators look - sim-helm skill: record why an unset list is the wrong shape here - tests: lock in that null removes a key and "" does not * docs(helm): correct the verify command's chart path and scope the required-secret claim - The verify snippet used a `sim/sim` repo alias that this chart never publishes; every other instruction installs from the local `./helm/sim` path, so the command could not run as written - Nulling a boot-critical key only fails at template time with the chart-managed Secret. `existingSecret` mode skips that validation entirely (the chart cannot read a pre-created Secret), and under ESO the key must instead be mapped in externalSecrets.remoteRefs.app * docs(helm): say null must be applied in every layer that sets a key `null` deletes a key from the map it is applied to, not from the pod. A key set in both `app.env` and `app.envDefaults` survives a null on the app.env entry alone — the deployment then inlines the envDefaults value again. Under ESO a retained `externalSecrets.remoteRefs.app` mapping keeps syncing the key regardless of app.env. - README and self-hosting docs: drop the "works in all three secret modes" shorthand and spell out that every layer setting the key must be nulled, including the ESO remote mapping - tests: cover both halves — nulling only app.env restores the envDefault, nulling both actually removes the key - chart 1.5.4; staging took 1.5.3 in the meantime	2026-08-17 18:16:35 -07:00
Waleed	9dc828f36a	fix(email): greet SMTP relays with a qualified hostname instead of [127.0.0.1] (#6799) ... * fix(email): greet SMTP relays with a qualified hostname instead of [127.0.0.1] Nodemailer derives the EHLO greeting from os.hostname() and substitutes the address literal [127.0.0.1] whenever that name contains no dot. Kubernetes pod hostnames never contain one, so every k8s deployment introduced itself to the relay as loopback and strict relays refused the session before any mail moved. Send the domain the app is served from instead, as RFC 5321 4.1.4 asks, with SMTP_EHLO_NAME to override it for relays that expect a different identity. * fix(email): parse EHLO address literals and drop a port from the app domain Review round 1. The bracketed branch matched a character class rather than an address, so [::::] and [13] reached the relay as a greeting it would refuse. Parse the address with node:net instead, which also admits the RFC 5321 IPv6: form. getEmailDomain reports a URL host, so a deployment served on a non-default port failed the qualified-name check and fell back to nodemailer's default — [127.0.0.1] again on Kubernetes, the exact failure this change exists to fix. Strip the port before validating. * fix(email): accept any casing of the IPv6 literal tag, and stop owning SMTP_EHLO_NAME in setup Review round 2. RFC 5321 tags the IPv6 address-literal form, and RFC 5234 makes ABNF string literals case-insensitive, so [ipv6:2001:db8::1] is as valid as [IPv6:...]. The exact-prefix check routed it to isIPv4 and discarded it. Drop SMTP_EHLO_NAME from the email capability's optional fields. SMTP_SECURE, the same kind of optional transport knob on the same provider, is not modelled there either, and claiming the field obliged the setup wizard to prompt for it — a field whose entire purpose is to stay unset now that the default is right.	2026-08-17 18:09:02 -07:00
Waleed	d17a11f29b	feat(jotform): trigger a workflow on every new form submission (#6802) ... * feat(jotform): trigger a workflow on every new form submission Jotform's only webhook event is a new submission, so the block gets one trigger. Deploying it registers the callback on the form through the API and undeploying removes it again. Two things about this provider needed handling: Jotform posts submissions as multipart/form-data, which the shared webhook body parser did not read — the delivery died as a 400 before any handler saw it. The parser now flattens a multipart body the same way it already flattens a urlencoded one, reducing an uploaded part to its filename so a stray file cannot inflate the execution input. The form's webhooks are identified by their position in the form's webhook map, so an id captured at registration goes stale the moment any other webhook on that form is removed. Nothing persists it; cleanup re-resolves the id by matching the callback URL. Registration checks the same way, because Jotform answers a rejected request with the unchanged list rather than an error. Answers are exposed as the parsed `rawRequest` rather than re-keyed by question label — the labels are not unique, and the payload shape is only documented as the raw q{qid}_{slug} map. The trigger's region field is named `apiRegion` so it does not collide with the block's own advanced-mode `region`. * fix(jotform): make webhook registration idempotent and URL matching tolerant Validated the trigger against Jotform's API reference and a captured delivery (zulip's multipart fixture), which confirmed every mapped field — formID, submissionID, formTitle, username, ip, type, pretty, rawRequest — and turned up three things worth correcting. Jotform keeps a form's webhooks as a plain list and does not treat the URL as a key, so posting one it already holds leaves the form delivering every submission twice. Registration now consults the list first and only posts when the URL is absent. The documented POST sample returns the new entry as "0", renumbering the rest, which is further reason nothing persists an id. URL matching no longer lets a trailing slash decide the outcome. Jotform stores the URL verbatim in every sample seen, but an exact match failing would hard-fail deploy, and Pipedream's client normalizes the same way. The rawRequest description claimed the field holds the submitted answers. A real payload also carries slug, buildDate, submitSource and jsExecutionTracker, and a file answer appears under the bare slugified label as upload URLs rather than under a q{qid}_ key — which is also why filtering to q-prefixed keys would silently drop file answers. * fix(jotform): keep the callback when an active deployment still needs it Redeploying prepares the replacement webhook row alongside the live one and a workflow keeps its path across deployments, so both rows resolve to a single callback on a single form. Registration adopts the callback already present instead of posting a duplicate, which left the retired row's cleanup deleting the one the new row had just adopted — the trigger went silent after a redeploy that changed the trigger config. Teardown now skips when another webhook row belonging to an active deployment resolves to the same form and callback URL, matching how the Telegram handler skips deleteWebhook while an active deployment still uses the same bot. A genuine undeploy has no such row and still cleans up.	2026-08-17 18:08:38 -07:00
Waleed	0e84e92d39	fix(cli): bound, trace, and explain the requests the CLI makes (#6798) ... * fix(cli): bound, trace, and explain the requests the CLI makes Four transport gaps, all of which failed silently. A request had no timeout, so a connection that was accepted and then never answered hung the terminal indefinitely. `SIM_TIMEOUT_SECONDS` now bounds one, defaulting to 3600s — deliberately above every timeout the server itself applies, since a synchronous workflow run is allowed 3000s on a paid plan and a tighter default would abort real work and report it as a transport failure. `0` removes the bound, for a self-hosted deployment that runs executions without one of its own. The caller's abort signal is composed with the timeout rather than replaced, so neither masks the other. Node ignores HTTP(S)_PROXY unless NODE_USE_ENV_PROXY opts in, and only from v22.21 and v24.5, so on a network that reaches the API only through a proxy every command failed to connect while the variable that would have fixed it was already set. The CLI cannot enable that from inside the process — Node reads it at startup — so it says what to do rather than bundling an HTTP stack for a setting the platform now owns. An API key was sent to any http:// endpoint with no signal. Now a warning, not a refusal: http is the documented way to reach a local dev server, and a deployment terminating TLS at a gateway is real. Loopback stays silent. `SIM_DEBUG=1` traces method, URL, status and duration. Bodies and headers are deliberately absent — the request carries the API key, and `secrets set` carries the secret itself. All four write to stderr, so a piped stdout stays parseable. * fix(cli): make the request bound safe on every runtime it supports Two ways the new timeout could fail before the request was made. `AbortSignal.any` arrived in Node 20.3 and this package supports Node 20, so composing a caller's abort signal with the timeout threw a bare TypeError on the earliest 20.x releases. It is now used when present and composed through an AbortController when not. `AbortSignal.timeout` rejects a fractional millisecond outright, and past 2^31-1 ms it does not fail at all — it clamps to 1ms, so the longest timeout anyone asked for became the shortest. The value is now rounded and refused above what Node can actually wait, pointing at 0 for an unbounded wait. Also unstubs env vars between tests: `stubEnv` is not undone by `unstubAllGlobals`, so a SIM_TIMEOUT_SECONDS set for one test configured every test after it. * fix(cli): correct the proxy version table, and classify a timeout mid-body `runtimeCanProxy` treated any release between 22 and 24 as capable, so on Node 23 — which reached end of life before the backport — a configured proxy was ignored and the CLI stayed silent about it, which is the exact failure the warning exists to report. The table is now the two lines that shipped the support, and anything after them. `AbortSignal.timeout` keeps firing after `fetch` resolves, so a bound that elapsed while the body was still being read — a large `files get` — escaped the client's own handling and printed a raw TimeoutError stack. The top-level handler now names it, which covers the streaming path as well as the JSON one. A user's own Ctrl-C raises AbortError and is deliberately left alone. * fix(cli): report a timed-out download as a timeout `files get --output-file` streams the body to disk, and `streamToFile` converted anything the stream threw into a write failure. So a request bound elapsing mid-download read as `Could not write <path>: ...`, sending the reader to check permissions and free space for a timeout they can raise, and hiding the one instruction that resolves it. The predicate and that instruction now live beside the timeout that raises them, so the client, the top-level handler and the download path all say the same thing. The wrapping stays where it is: the staged-download cleanup runs off that failure, and rethrowing past it would leak the temporary directory. * fix(cli): keep a sub-millisecond timeout bounded Zero is how this function says "no bound", so rounding a positive SIM_TIMEOUT_SECONDS down to zero inverted the request: anything under 0.0005s asked for the shortest possible timeout and got none at all, leaving a stalled request to hang. Introduced by the rounding that fixed the fractional-millisecond rejection. Floored at 1ms for every positive value; only a literal 0 still disables.	2026-08-17 18:07:36 -07:00
Waleed	0b4d34137b	feat(secrets): add optional descriptions to workspace secrets (#6796) ... * feat(secrets): add optional descriptions to workspace secrets Workspace secrets already have a backing credential row with a description column, but nothing surfaced it. Teammates had no way to record what a secret is for. - Add a Description field to the secret detail page, matching the integrations credential page, gated on workspace-secret admin - Fold the value and description editors into one Save/Discard pair and one unsaved-changes guard; two guards cannot coexist, since each seeds its own same-URL history entry - Match descriptions in the secrets settings search - Expose description on GET/PUT /api/v2/secrets and in the CLI Descriptions are workspace-only: env_personal credential rows are per-workspace mirrors of one user-global secret, so one saved there would exist in a single workspace, and a personal secret has no teammates to inform. The API rejects a description on personal scope rather than silently dropping it, and omitting it on PUT leaves any existing description untouched so a value rotation cannot erase it. * fix(secrets): address review findings on secret descriptions - Patch the credential detail cache optimistically on update. `onMutate` cancelled the detail query but only patched the lists, so a detail-backed editor stayed dirty after a successful save until the refetch landed — long enough for Discard to restore the pre-save value over the committed one, and for Back to open the unsaved-changes guard. - Memoize `useSecretValue`'s returned callbacks and object, per the hook convention, so the composed form's save/discard stop churning per render. - Reject a description on a personal secret in the domain layer rather than only at the v2 boundary. The internal credential update path accepted one for any type, writing data every reader hides. - Normalize an empty description to null so the API and UI agree. - Correct the secrets documentation, which described a Display Name field the detail view does not have and omitted the scope rule. - Drop the CLI's copy of the 500-character bound; it can't import the contract, so a copy only drifts from the message the API already returns. - Collapse a redundant save guard and align the description write gate with the render gate. Leaves the integrations credential page byte-identical to staging. * fix(secrets): keep the API docs example and CLI column order stable Backward-compatibility fixes for anyone who never sets a description. - Move the blank-to-null normalization out of the contract and into the route. A Zod `.transform()` on any property drops the whole request schema's OpenAPI examples, which had silently removed the Set Secret request example from the published docs. - Append the CLI `description` column instead of inserting it before `updated`. `--output text` is positional, so inserting would shift every field an existing script cuts. - Reject a description on a personal secret with a message that says so, rather than dropping the field and falling through to the generic "no updatable fields" error.	2026-08-17 17:33:35 -07:00
Waleed	38075ad977	fix(sap_concur): align the integration with SAP Concur's documented API (#6790) ... * fix(sap_concur): align the integration with SAP Concur's documented API Validated all 70 tools, the block, and both proxy routes against SAP's published API docs. Auth: - add the password and companyUuid to the token cache key so a request with the wrong password can no longer be served a cached token minted from someone else's - wire the documented company-level flow (username = company UUID, credtype = authtoken) so companyUuid actually scopes a token - expand the datacenter allowlist to the documented set (adds glz, apj1, usg, the impl hosts, and the www- twins) and drop the undocumented cn host; validate the returned geolocation by shape instead of membership - coalesce concurrent token fetches so a fan-out mints one token - forward Retry-After so 429 retries pace off Concur's own hint - handle the errorMessageList, SCIM detail, and legacy Error.Message shapes instead of falling through to a generic HTTP message - pin redirects and cap the response body Block: - collapse six contextType subBlocks that disagreed on their default, so a new block no longer seeds MANAGER for every operation - clamp contextType to each operation's documented set - stop requiring a userId and contextType that the default operation's tool does not accept, and scope the receipt fields to the upload ops - reach six params that had no subBlock, and pass userId on travel request updates so a stale value cannot impersonate Tools: - correct response shapes that resolved to undefined: budget headers, budget categories, allocations, receipts, SCIM nextCursor, and the delete endpoints that return a bare boolean - use the Travel Request Amount schema (currency, not currencyCode) - narrow the four XML-only travel tools to a documented string payload and request application/xml - surface real errors instead of a JSON parse failure when the proxy returns a non-JSON body - cap receipt uploads at the documented sizes before downloading Adds 106 tests covering the token cache, geolocation validation, path traversal, and error extraction. * fix(sap_concur): drop the removed forwardId subblock via a migration Removing the `forwardId` subblock without a migration entry breaks deployed workflows that still carry a value under that key. It fed a `concur-forwardid` request header that is documented nowhere in Concur's Receipts v4 or Image v1 references, so it was never honored. There is no replacement subblock and the value is an opaque caller-chosen string rather than a secret, so it is dropped outright. * fix(sap_concur): stop swallowing upload response-read failures The upload route caught every error from the bounded response read and continued down the success path, so a size-limit breach or a stream failure surfaced as an upstream success with a null or header-only body. Concur returns Content-Length: 0 on a successful image-only upload, and readResponseTextWithLimit already returns an empty string for that without throwing, so dropping the catch keeps the legitimate empty-body case working while letting real read failures reach the route's handler. * fix(sap_concur): unblock company auth and correct the body wand prompt The password grant marked username required, so the company-level flow — which sends the company UUID as the token username and has no user login — could not be configured at all, even though the request schema and token fetch already accept companyUuid without a username. Username is now optional for that grant and the server-side check reports which of the two is missing. Relabels the password and companyUuid fields to say what they carry in the company flow. The shared body wand prompt also still described several payloads the way they looked before this branch: quick expenses in PascalCase rather than v4 camelCase, travel requests and expected expenses using currencyCode where the Request v4 Amount schema uses currency, the standard SCIM SearchRequest URN instead of Concur's, startIndex as a search parameter when it is unsupported, and a cash advance shape that does not match the documented request. A wand-generated body was therefore rejected for most of the create operations it covers. * fix(sap_concur): keep Concur's status when an error body fails to read Removing the blanket catch from the upload read fixed one failure mode and introduced its inverse: a cap breach or stream error while reading a non-success body threw before the route reached the branch that preserves Concur's status, so an upstream 4xx surfaced as a Sim 500 and could trigger a retry the caller should not make. Both routes now split the two cases. On a success status the body is the result, so a read failure still propagates. On an error status the body only supplies the message, so a read failure resolves empty and the upstream status survives, with the message falling back to the generic HTTP-status form. Adds 21 tests covering both helpers over success, error, empty-body and boundary statuses; inverting the status check turns 14 of them red.	2026-08-17 16:27:23 -07:00
Waleed	60097c89b4	fix(cli): default to the host that serves the API (#6791) ... `sim.ai` answers /api/** with a 301 to `www.sim.ai`, and the client refuses to follow redirects — a 301 rewrites a POST into a bodyless GET, so following one turns a write into a silent no-op and hands the API key to whatever host Location names. Defaulting to the apex therefore failed every command for anyone who never set an endpoint. Before the refusal shipped it was quieter and worse: reads succeeded while writes did nothing. Also trims the provider catalogue from eleven inferred columns to seven. `docsUrl`, `helpText`, `requiresClientGeneratedCredentialId` and the nested `fields` are what you read once you have chosen a provider, not what you scan to choose one, and they pushed the table well past a terminal. Both ids stay: `credentials connect` names an OAuth provider by `serviceId`, `credentials create` matches a service account on `providerId`.	2026-08-17 16:25:58 -07:00
Waleed	ae2147645c	fix(cli): resolve findings from a full command-surface audit (#6788) ... * fix(cli): resolve findings from a full command-surface audit Exercised all 147 commands against a live deployment. Fixes the defects that surfaced, plus the docs and generator drift they exposed. Transport - Stop following redirects. A bare domain that 301s to www silently converted POST to GET and dropped the body, so reads worked while every write failed with a misleading validation error and login returned 405. Both the client and the device flow now explain the redirect and name the endpoint to configure, rather than carrying credentials off-origin. - Report a non-JSON response as one instead of printing the HTML page. - Name the personal-API-key remedy on a workspace-key refusal, reading the machine-readable code the API actually sends. - Drop union-branch noise from validation errors that contradicted itself. - Show paging progress on stderr for multi-page fetches. Output - Clamp record values for table only. text is the format built for pipes, and it was truncating signed URLs and tool source mid-value. - Infer timestamp, duration, bytes and boolean formatting for API-owned keys so undeclared commands stop printing raw ISO and float ms. Skips user-defined table cells and leaves json/yaml on the raw payload. - Render a declared-but-absent field as an em dash; billing credits were vanishing silently. Paths, naming and validation - Percent-encode folder paths per segment and decode them for display, so a folder reads and types as the name shown in the app. - Reject a malformed endpoint where it is set and where it resolves, instead of crashing with a URL parse trace. - Request the detail level logs list's own columns need; its workflow column could never populate. - Rename three commands that described themselves wrongly and align two flags with their siblings. Old spellings still work: hidden, warned on stderr, and kept out of help and docs. - Verify whoami against the API, separating a bad key from an unreachable endpoint, and report the workspace by name. - Correct the --yes help text, which advertised skipping a prompt that does not exist. Docs - Teach the docs generator that a flag required by the runtime is required, and that hidden commands are not documented. * fix(cli): clear the paging progress line when a page fails Progress is written without a trailing newline so it can be overwritten in place, and both paging loops cleaned it up only on success. A page that threw part-way through left `fetched 1200…` on the line the error was then printed onto, so the two ran together. * fix(cli): name a working API root when an endpoint redirects The suggested endpoint was the redirect target's origin, which drops a path prefix. A self-hosted deployment reached at https://host/sim was told to set https://www.host — not an API root, so following the advice replaced one broken endpoint with another. Derive it by stripping the request's own path from the target instead, so a prefix survives, and say nothing about --set-endpoint when the target resolves to the endpoint already configured: a trailing-slash or path normalization redirect keeps the origin, and naming the value the caller already has explains nothing. The login poll shared both faults and now shares the helper.	2026-08-17 15:55:04 -07:00
Waleed	75718ab39f	fix(execution): stop a cancelled run reporting success when its wait swallows the cancellation (#6775) ... * fix(execution): stop a cancelled run reporting success when its wait swallows the cancellation Cancellation reaches a running execution over Redis pub/sub, which is at-most-once. The engine turns that into `status: 'cancelled'` via `signalCancelled`. But the wait handler also polled the durable Redis cancellation key itself, and on a hit it broke out of its sleep and returned an ordinary successful block output. The engine's `cancelledFlag` stayed false, so a cancelled run finished as `success: true` — and with a block after the wait, kept executing. Whichever detector fired first won. The engine's pub/sub path normally wins by about one round trip; when the wait's own 500ms poll landed inside that window the cancellation was lost. Consolidate detection in the engine, which is the only component that can project run status: extend the once-at-start durable backstop into a poll that runs for the life of the run and routes through `signalCancelled`. The wait handler and loop orchestrator now observe only `ctx.abortSignal`, which the engine aborts, so no leaf can observe a cancellation the engine has not seen. The loop orchestrator additionally used to ignore `abortSignal.aborted` whenever Redis was enabled, so a mid-loop timeout or client disconnect was invisible to it, and it awaited a Redis round trip on every iteration. Handlers that abort their own I/O off `ctx.abortSignal` are unaffected: that surfaces as a throw, which the cancelled branch of `run` already classifies. * docs(wait): correct the in-line wait ceiling to 5 minutes The Wait page claimed a 10-minute cap for a synchronous wait in three places. `MAX_INPROCESS_WAIT_MS`, the block description, the sub-block hint, and the validation error all say 5 minutes.	2026-08-17 01:55:48 -07:00
Waleed	b38e4e2f91	docs(integrations): add missing manual intros; fix light brand tiles rendering white glyphs (#6774) ... * docs(integrations): add manual intro sections to eight integration pages * fix(styling): scan blocks and ee in the tailwind content globs * docs(snowflake): correct the unload-data capability to a table source	2026-08-17 00:49:54 -07:00
Waleed	31521d6abf	fix(connectors): validate and repair the knowledge-base connector fleet (#6757) ... * fix(connectors): validate and repair all 61 knowledge-base connectors Audits every KB connector against its provider's live API documentation and fixes what the audit found. The dominant defect class is deletion reconciliation: the sync engine hard-deletes any stored document absent from a "full" listing, and most connectors had a path where a truncated or errored listing failed to set `syncContext.listingCapped`. Highest-impact fixes: - linear: `getDocument` was dead code. The query declared `$id: ID!` where the schema is `issue(id: String!)`, so every call failed variable validation. - salesforce: `v62.0` was substituted into a `{version}` template that already contains the `v`, so every REST call 404'd. SOQL `LIMIT` was also used as a page size, silently capping every sync at 200 records. - notion: only the first level of blocks was fetched, so tables indexed empty. - microsoft-teams: `/messages` returns messages without replies, so no threaded content was ever indexed. - gmail: an empty page discarded `nextPageToken`, which reads as a complete empty listing and hard-deletes every stored thread. - confluence: the CQL path paginated with `start` and `totalSize`, neither of which exists on that endpoint, so label-filtered syncs stopped after one page. - box: `supportsRefreshTokenRotation` was unset, so Box's rotated refresh token was discarded and every credential died on its second refresh. Removes the Evernote integration entirely: the classic EDAM API is deprecated, its sandbox is decommissioned, and developer tokens are no longer obtainable. Makes SFTP host-key verification mandatory, and adds an attendee-PII opt-out to google-calendar and google-meet (default on, so existing sources are unchanged). Bumps the contentHash namespace for notion, google-docs and hubspot so existing documents re-hydrate once and actually receive the content fixes above. * fix(connectors): close swallow-into-empty and cursor-taper regressions Ship-gate pass over the connector audit. Every finding was re-verified against the provider's live documentation or machine-readable spec before being acted on; several pass-3 edits were reverted rather than extended. Correctness fixes: - fireflies: a 2xx with an unparseable body returned an empty listing instead of throwing. fireflies runs a full sync every time, so a fault persisting across two syncs would have tombstoned all indexed docs. - linear: same shape via `data.issues || {}` on a non-nullable connection. - greenhouse: a 403 from a key without scorecard permission was treated as transient, appending `:partial` to the hash. That never matches the list stub, forcing full re-hydration of every candidate on every sync forever. - google-meet: `fetchParticipants` carried a 404 swallow copied from its transcript siblings, freezing every speaker as "Unknown". - airtable, asana, ashby: reverted page-size tapers applied over opaque cursor tokens. The cap was already enforced server-side. - google-docs: response byte cap resolved to 800MB and could never fire. - google-forms, google-vault, notion, sharepoint, dropbox: `getDocument` now throws on transient failure instead of returning null, which the engine reads as absence. Security: - Retry headers are attached non-enumerably. TypeScript `private` is compile-time only, so `SecureFetchHeaders.setCookies` was an own enumerable property that the logger serialized into sync logs. Docs and dead code: - Corrected six fabricated doc citations (github, jira, jsm, linear, google-meet, dropbox) and removed the Evernote integration entirely. * refactor(jira): type the ADF node helpers with unknown instead of any * fix(connectors): throw on misconfigured typeform/zendesk sources instead of returning null A null from getDocument reads as documented absence, so on an add the document is dropped with neither a failure counter nor a log. Both listDocuments paths already throw on the same missing config. * fix(connectors): keep confluence CQL page size constant; unify monday API version The CQL search endpoint paginates by opaque cursor, and Atlassian does not document that a cursor issued against one limit survives a request asking for a different one. Narrowing limit to the remaining budget was the same pattern reverted on airtable, asana, and ashby. The page size is now constant and the cap is applied by trimming the returned page, which keeps the cap exact without varying the request. Monday OAuth getUserInfo hardcoded API-Version 2024-10 while every other monday surface reads MONDAY_API_VERSION, defeating the single-source pin. * fix(connectors): act on the final validation sweep Findings from a read-only /validate-connector pass over all 59 changed connectors, verified against provider specs before acting. Silent-drop fixes (a fulfilled null from getDocument records no failure and no log, so the document vanishes): - ashby: candidate.info returning success with an unusable payload. Ashby sets contentDeferred, so this path is live. - azure-devops: an unresolvable branch, likewise live. - dropbox: 409 covers the whole LookupError union, and restricted_content and locked both mean the file still exists. Only not_found is absence. - docusign: fetchFormValues swallowed every non-OK status, baking a permanently incomplete document since the hash is metadata-only. typeform: 'all' sent response_type=started,partial,completed, but Typeform documents only partial and completed. An unknown enum member risks a 400 that fails the whole sync, and staging omitted the parameter entirely, so this shipped as a regression. Now requests the widest documented set. github: removes a utf-8 blob branch justified by a misattributed quote — that sentence describes the encoding REQUEST parameter of Create a blob; the GET response is documented as always base64. Also corrects two comments that hid a real drop: >1 MB files under vnd.github+json 403 rather than returning encoding: none. hubspot: routes HTML detection through a shared anchored helper. The loose pattern matched angle-bracketed prose such as an email address, and htmlToPlainText deletes the span and collapses line structure. This matters now because the hubspot:v2: bump rewrites every live document once. youtube: drops an invented channel-ID format quote. * fix(connectors): converge incidentio partial hash on settled statuses A 403 from a key without incident_updates permission, or a 404, returns on every sync. Marking those incomplete appended :partial to a hash that then never matched the listing stub, so the incident re-hydrated forever without converging. Only a transient failure may mark content incomplete now, which matches how greenhouse already treats the same class. * fix(connectors): flag WIQL truncation unconditionally; stop swallowing docusign form-data failures azure-devops: the 20,000-item WIQL ceiling was probed by asking for a matching item with an id beyond the largest returned. That probe is unsound — buildWiql orders by ChangedDate DESC while ids are assigned in creation order, so the highest-id match is almost always inside the returned window. The probe came back empty for genuinely truncated projects, left the listing unflagged, and let deletion reconciliation remove every indexed item outside it. Flag unconditionally instead; the cost is a project sitting exactly at the ceiling not reconciling deletions until a full resync. docusign: fetchFormValues threw on a non-404 status and then caught its own throw, returning []. The earlier fix was a no-op. The catch now rethrows, so a transient failure produces a failed row instead of a permanently incomplete document under a metadata-only hash. * fix(connectors): restore airtable AI Text indexing; floor sentry maxIssues airtable: staging rendered object cells with a JSON.stringify fallback, so AI Text values and nested lookup arrays reached the index. This branch replaced that with a fixed key-probe list to stop attachment-URL hash churn, but the probe list has no fallback — aiText ({state,isStale,value}) and nested arrays rendered to the empty string and vanished from every document, and the content-derived hash never moved when the text regenerated. Read `value` last, after the existing probes, and recurse on nested arrays. The generated text is stable rather than an expiring signed URL, so this does not reintroduce churn. sentry: maxIssues now feeds the request limit, and Sentry rejects a non-integer. validateConfig accepts a fractional entry, so a config that saved cleanly would fail every sync at listing time. Staging was immune only because it sent a hardcoded page size.	2026-08-17 00:21:04 -07:00
Waleed	cc1a278d73	fix(integrations): close defects found by an independent cold audit (#6767) ... * fix(integrations): repair Update SLO and advanced OData filters An independent audit — eight cold readers, one per integration, given no prior findings — checked the eight integrations merged to staging today. Two defects broke an operation outright; both are fixed here. datadog: Update SLO rewrote every non-metric SLO to `metric`. The SLO Type dropdown carried a `metric` default and its condition covered both create and update, so an untouched control reached mergeSloUpdatePayload as an edit. A metric SLO requires `query`, and the merged body carries monitor_ids or sli_specification instead, so Datadog rejected it — Update SLO was unusable on monitor-based and time-slice SLOs, with no way to express "keep the current type". Update now has its own control defaulting to "Keep current". microsoft_ad: list_users, list_groups and list_service_principals emitted $count=true only alongside $search, so any $filter using an advanced operator (ne, not, endsWith, startsWith on non-indexed properties) returned 400. Graph requires $count=true with ConsistencyLevel: eventual for those. list_devices already did this correctly; the other three now match it. Also from the same audit: Datadog path IDs are trimmed before encoding in all 20 URL builders rather than 2, matching the existing get_monitor test. * fix(integrations): cloudflare, crowdstrike, and mssql audit findings From the independent cold audit. Cloudflare: DNS analytics no longer emits fabricated min/max telemetry (Cloudflare documents both as always empty); purge_everything defaults to specific-purge and errors when combined with target lists; three unsourced description claims corrected; Array.isArray guards on four older list transforms. CrowdStrike: IOC sort placeholder corrected to the dot form (created_on.desc, not the nonexistent created_timestamp); the 500-indicator cap relabelled as a Sim bound rather than a CrowdStrike one; credential failures return 401 rather than 500; prevent_no_ui noted as unenumerated. MSSQL: introspect no longer lets the model choose the database; row and byte caps on reads; introspection collapsed from 4N+2 to 6 fixed queries; WHERE and identifier guards run before the connection opens so rejections are 400 not 500; SAVE TRANSACTION, OPEN/CLOSE key, DEALLOCATE and ADD SIGNATURE added to the statement screen as two-token phrases; encrypt wording corrected to say TDS 7.4 encryption is negotiated, not guaranteed. * fix(splunk): publish the real output tables and read the errors Splunk sends The docs generator parses tool source text and resolves a shared `outputs` const only from the family's `types.ts`, so Splunk's helpers in `utils.ts` were invisible to it: seven operations published the block's union of every output instead of their own. Run Search and Get Search Results each shipped a ~50-row table naming savedSearches, alerts, indexes, and apps they never return, Cancel Search Job lost `messages`, and the four list tools lost `total`/`offset`. Inline the four helpers into each consuming tool and delete them, since relocating a shared const only moves the trap. Also: - Add a `splunk-errors` extractor for the documented `{messages: [{type, text}]}` envelope and set it on all twelve tools. A rejected SPL string, the most common failure, previously fell through to the status text and reported "Bad Request". - Read `searchEarliestTime`/`searchLatestTime` with `asNumber`. The job entry documents them as bare epoch numbers, so `asString` returned null for every `output_mode=json` response. - Project the `<messages>` block of the XML job-control response. It is the only payload that endpoint returns, so `cancel_search_job.messages` was always empty. - Mark the nullable job outputs optional, matching the transform. - Default `run_search` to `max_count=1000`. A oneshot search has no paging escape hatch and Splunk's own default is 10000 rows in one buffered response. - Add suggested skills to `SplunkBlockMeta`, grounded in `tools.access`. The regenerated tool metadata also picks up the Cloudflare and MSSQL output changes from the previous commit, which were never synced. * fix(okta,servicenow): apply integration audit findings Cherry-picked from fix/okta-servicenow-audit-followups (6db0f54), whose base predated the earlier audit round; the duplicate isOktaFlagEnabled that produced is resolved in favour of the existing richer helper, which already accepts 'yes'/1/'on' as well as true/'true'. okta: get_logs no longer advertises hasMore forever. A System Log query with no 'until' is a polling query, and Okta always returns a next link for one, even on an empty page — so any loop driven by hasMore never terminated, including the one our own shipped skill instructs the agent to run. errorCauses is now surfaced, so a failed write reports the real reason instead of the useless 'Api validation failed: profile'. sendEmail routes through one coercion helper across all four lifecycle tools. update_group's declarative fallback throws rather than silently truncating an extensible group profile. servicenow: attachmentLimit and limit no longer overwrite each other. Neither assignment was scoped to an operation, so all 12 paginated operations could silently return a row count the user never asked for — defeating the block's own design, which gave attachmentLimit a unique id precisely to avoid this. All seven approval states are published by ServiceNow and are now reachable from the filter, with the space-vs-underscore punctuation documented. The five legacy generic tools route through the shared response helpers, and the folder's only 'any' is gone. Block skills now name the semantic operations. * chore(integrations): regenerate catalog and docs artifacts * fix(integrations): disclose MSSQL truncation and keep Okta's poll cursor Three defects the review round found in the audit fixes themselves. MSSQL capped a recordset and then reported it as complete: `executeQuery` computed `truncated`/`truncationReason` but all five statement routes returned only `message`, `rows`, and `rowCount`, so a caller could not tell paging was required. A shared `toRowsResponseBody` now folds the reason into `message` for an agent reading the status line and exposes the two fields for a caller that branches on them. The byte ceiling also admitted a single oversized row as a lone exception, so one `nvarchar(max)` value serialized an unbounded body — the ceiling bounded everything except the case it exists for. A row is now admitted only when it still fits, and the drop is disclosed rather than read as an empty table. Okta's `get_logs` nulled `nextCursor` alongside `hasMore` on an empty polling page. Terminating the loop is right, but the cursor is the resume handle Okta tells callers to persist, so a scheduled workflow that hit one quiet interval restarted from `since` and re-delivered events it had already processed. The two answer different questions and now diverge. Cloudflare's purge block no longer lets the invalid combination be built: the four target fields are hidden once Purge Everything is selected, so the tool's guard is a backstop rather than a reachable hard error. * fix(okta,servicenow): stop sending requests the APIs reject Okta documents `since` and `after` on the System Log as mutually exclusive, so `get_logs` lets the cursor win rather than sending both — the shape a scheduled poll that persists the cursor would otherwise send. Seven boolean query params reached Okta interpolated raw, so an agent tool call supplying `yes` was rejected. Each now routes through `isOktaFlagEnabled`, keeping its existing send-or-omit behavior. A cleared ServiceNow limit/offset/quantity stayed `''` through the block mapper and was appended as a valueless `sysparm_limit=`. The mapper now resolves a blank to undefined, and the tools skip a blank as well. * fix(integrations): mssql guard gaps and Entra query, scope, and output findings MSSQL read-only screen - Screen RENAME, documented T-SQL DDL for Azure Synapse dedicated SQL pools and Analytics Platform System, which are reachable over TDS with exactly the connection fields this block exposes. `SELECT 1 RENAME OBJECT dbo.t TO t2` was a schema change passing an operation advertised as read-only. - Screen the Service Broker family: RECEIVE as a word, and END/MOVE/GET CONVERSATION and SEND ON CONVERSATION as two-token phrases, since END closes every CASE. RECEIVE is a destructive read and END CONVERSATION WITH CLEANUP drops a conversation's messages. MSSQL routes and block - Build the insert statement before connecting, matching update and delete, so a bad identifier answers 400 instead of burning a TLS+login and returning 500. - Declare `truncated`/`truncationReason` on the block, which the tools declare and the routes emit but the block left unreferenceable. Microsoft Entra ID - Pair `$count=true` with `ConsistencyLevel: eventual` conditionally. Graph documents `hasMembersWithLicenseErrors`, `isLicenseReconciliationNeeded`, and `identities/any(i:i/issuer)` as filterable only *without* advanced query parameters, and documents advanced queries as unsupported in Azure AD B2C tenants, so the unconditional pair broke filters that previously worked. When continuing from a nextLink the pairing is read off the link itself. - Request `LicenseAssignment.Read.All` instead of `Directory.Read.All`. The latter was needed by `GET /subscribedSkus` alone, whose permission table names the former as least privileged and does not list the ReadWrite scope we hold. - Enumerate the block's real output keys instead of a single `response` object no tool emits. * fix(splunk,datadog): stop truncating searches and send mute/unmute as query params Splunk run_search: revert the `max_count=1000` default added last pass. It was wrong on both halves. Splunk documents the parameter as "the number of events that can be accessible in any given status bucket. Also, in transforming mode, the maximum number of results to store" — so for a non-transforming oneshot it bounds status buckets, not the response, and for a transforming search (`| stats`, `| timechart`, which is what the block's own skills generate) it capped results at 1000 where Splunk would have stored 10000, silently. The block's `maxCount` placeholder already read `10000`, contradicting the code. Send `max_count` only when the caller sets it and restate the description in Splunk's own terms, matching create_search_job. The real guidance — a oneshot buffers the whole result set, so use Create Search Job + Get Search Results for anything large — moves into the tool description and the search-splunk-logs skill. Datadog mute/unmute: send `scope`, `end`, and `all_scopes` as query parameters. `MuteMonitor` and `UnmuteMonitor` declare no `requestBody` in the authoritative spec (docs.datadoghq.com/resources/json/full_spec_v1.json — the generated datadog-api-client-go v1 schema omits both operations and is a subset, not the authority); all three parameters are `in: query`. Sent as a JSON body they are dropped, so a scoped, time-boxed mute becomes an indefinite mute across every scope and unmute's "all scopes" never applies — answered with a 200 and the full monitor object, so nothing surfaces. Datadog list_monitors: imply `page=0` when a page size is set without a page. Datadog "returns all monitors without a `page_size` limit" when `page` is absent, so Page Size was inert from a control that reads as a bound. `page` is not defaulted when neither is set — that would silently truncate a caller relying on the documented return-everything behavior. Also: - Note in get_fired_alerts that `name=-` returns every saved search's fired alerts and the endpoint documents "Request parameters: None", so there is no count/offset to bound it. - Fix the Splunk block's `messages` output blurb: `[{type, text}]` holds for the search and job-control operations, but get_search_job returns an object. - Generalize the Datadog block's numeric coercion (`datadogPageNumber` → `datadogNumber`) over all 32 bare `Number()` mappings, so a typo or unresolved reference is omitted rather than sent as `NaN`/`null`, and an explicit `0` survives the old truthiness guard. - Disclose create_event's documented 18-hour `date_happened` ceiling, and that send_logs' `ddsource: "custom"` is a Sim default rather than a Datadog one. * chore(integrations): regenerate tool metadata and docs * fix(integrations): resolve confirmed findings from cold block audit Cloudflare: clear purge_cache advanced targets across operations; send action_parameters/ref/logging on rate-limit rule updates; migrate off the deprecated batch zone-settings endpoint; correct MX/URI priority wording; stop coercing blank numerics to 0. CrowdStrike: seed includeHidden to match Falcon's documented default. Microsoft Entra ID: resolve a UPN to an object ID for app role assignment; wrap 21 array outputs in items.properties so nested paths resolve. Okta: route assign_user_role's notification flag through isOktaFlagEnabled. ServiceNow: drop the triage skill's claim of a default limit that does not exist. Splunk: always assign coerced numerics so raw values cannot leak through the executor's raw-input merge. * fix(editor,credential-group): mask secrets outside short-input and stop a per-option abort from failing a shared query config.password only reached the short-input renderer, so eight credential fields rendered in plaintext: private keys on ssh/sftp/pi/kalshi, the Secrets Manager payload, the STS web-identity and SAML assertions, and the Browser Use variables table. long-input, code, and table now honor the flag. Code fields mask through the highlighter because react-simple-code-editor paints its textarea transparent; the table masks every column but the first so key/value rows stay distinguishable. A registry-walking audit test fails both on a password flag sitting on a type that cannot honor it and on any of the eight fields losing its flag. credential-group threaded a per-option AbortSignal into the fetch registered under the workspace-wide credential group list key, so closing one option panel rejected every co-observer with an AbortError that is not a React Query cancellation. The shared fetch now runs on its own lifecycle signal. * fix(mssql,editor): measure the response cap in UTF-8 and stop search from unmasking secrets capRecordset sized rows with JSON.stringify(row).length, which counts UTF-16 code units while the emitted body carries raw UTF-8. CJK is the worst case at 3 bytes per unit, so a recordset admitted as 10 MB serialized to 28 MB. Rows are now measured with Buffer.byteLength, serialized once each, with array punctuation charged exactly and a reserve held back for the response envelope. Workflow search revealed masked credentials without the user touching the field: the search panel keeps focus in its own input and only scrolls the match into view, so typing a guess painted a private key on screen. The index is built client-side from values already in page memory, so this was never a privilege boundary, but masking exists to prevent incidental display and a screenshare-visible reveal defeats it. Focus is now the only reveal, applied through one shared policy across all four renderers. * test(editor): drop PEM-shaped fixtures from the masking tests The masking fixtures carried a literal OPENSSH private key header, which GitGuardian flags as a committed secret even though the body was only the base64 of "openssh-key-v1". The fixtures now use an obvious marker string, and the assertions derive their match text and dot counts from the fixture instead of restating its bytes. * refactor(editor): drop the dead isSearchHighlighted prop No renderer consumed it. The editor computed it at two call sites and sub-block passed a hardcoded false into renderLabel's slot for it, so even the one function that declared a parameter never saw the real value. Its only live effect was in the memo comparator, where an unconsumed value changing forced a re-render for nothing. The name stays in the masking audit's forbidden-inputs list, which guards against a search signal being wired back into a masking decision.	2026-08-16 23:25:16 -07:00