829 Commits

Include renames

Author	SHA1	Message	Date
Waleed	292e59f693	fix(docs): stop publishing unsettable params, fix comment blanking (#7169) ... * fix(docs): stop publishing unsettable params, fix comment blanking generate-docs never read visibility, so every tool param appeared in the public Input table -- including params marked visibility:'hidden', which are shown to neither the user nor the LLM. Several are credential-shaped (idToken, instanceUrl, apiToken, cloudId), so the docs told integrators they could set values they cannot reach. A hidden param is now dropped only when the block declares no subBlock for it, matched on id or canonicalParamId -- a param can be hidden on the tool because the block injects it while the block still renders it as a required field the user types. blankStringsAndComments kept the first and last character of every match. That is right for a quoted string, where both are delimiters, but for a '//' comment the last character is arbitrary source text, so a commented-out '// options: [' left an unbalanced bracket that derailed the subBlock scan. Parsing now throws rather than silently reporting that a block exposes nothing, since that fallback was the destructive one. Also corrects the LinkedIn w_member_social consent-screen description, which read 'Access LinkedIn profile' for a scope that posts on the user's behalf. * fix(docs): keep hidden params the block mapper supplies The carve-out only recognized an identity match between a subBlock id and a tool param, so a block that renames or assembles the value in tools.config.params was invisible to it -- and the row was dropped even though the user types it. Cal.com's attendee (required) is assembled from attendeeName/attendeeEmail/ attendeeTimeZone; JSM's workspaceId comes from assetWorkspaceId; Textract writes parameters.file from a field whose canonicalParamId is 'document', which left the Mistral PDF Parser documenting zero inputs. Collects params written by any accumulator identifier, not just 'result', since the two real mappers use different names. Object keys are collected without proving they are top-level, so a nested key can produce a false keep -- one hard-to-set row is better than hiding a required input. * fix(tools): reject values that cannot be a path segment toGuardedString coerced with String(value), so an object reached the wire as %5Bobject%20Object%5D and a boolean as 'true' -- a doomed request instead of a clean error, on 44 live call sites. Accepts string, bigint, and finite non-exponential numbers; everything else throws a named error. Rejects a number whose decimal text is a rewrite rather than the caller's value: 1e21 stringifies to '1e+21', and an integer past 2^53 has already lost digits. A snowflake cannot be repaired here at all -- JSON.parse destroys it before this runs -- so the doc now says it must arrive as a string, and cites Box folderId (root = 0) instead. Corrects the claim that the parser removes only an exact '.' or '..'; the spec defines 11 removable spellings. The guards are sufficient because encodeURIComponent escapes '%', not because the others cannot occur. * test(oauth): pin the LinkedIn write-scope description Nothing guarded the consent-screen text: utils.test.ts covered only the Bitbucket and Reddit overrides, and the modal test stubs getScopeDescription to identity, so a regression to a read-only label for a posting scope would pass silently. * fix(docs): stop a comment hijacking the id scan, unhang a Firecrawl reference The depth-1 walk copied from the raw source at indices where the blanked copy was at depth 1, so 'id:' inside a string value or a comment landed in the scanned text and won the first match. With the keep-bias that now means a phantom id can retain a param the block never exposes. Matching runs on the blanked text and reads the literal back through a source-index map. No block in the repo trips this today -- verified across all 305 -- so this is a latent fix. Removing the unsettable scrapeOptions row left five Firecrawl Search output descriptions referencing a name that no longer appears on the page. They now describe the response condition instead. Pointing them at 'formats' was not an option: that subBlock is conditioned on scrape/parse/batch_scrape and the search tool declares no such param, so it would have swapped one dangling reference for another. * refactor(tools): drop the unused guard API from this PR safeUrlPath, safeOpaqueUrlSegment and SafeUrlPathOptions had zero call sites -- 469 lines of unused API in a docs-generator change, including an allowEmptySegments flag whose own TSDoc documents a host-takeover footgun ('//evil.com' under new URL(relative, base)). They belong with the ~693 traversal call sites that use them, where they can be reviewed against real usage. What stays is the part with 44 live consumers: safeUrlPathSegment now accepts number and bigint. Staging already rejected every non-string, so this only widens acceptance -- a differential over 87 real call-site values shows 87 identical, 0 differing. Also: encodeURIComponent throws an unnamed URIError on a lone surrogate, which JSON.parse accepts, so a truncated emoji lost the param name the file claims as its invariant. And the exponential rejection told tiny values like 1e-7 they were 'too large' when they round-trip exactly; the rejection is right -- a path segment should not rewrite 0.0000001 into other text -- but the stated ground was not. * fix(docs): abort before writing on a parse failure, unhang three descriptions The guard's own TSDoc said to fail rather than guess, because the fallback strips every hidden param from a page. The caller did the opposite: it caught, recorded, and continued with an empty id set, and the non-zero exit came after every page was already written. A developer who reran, saw red, and missed the scrollback could commit a stripped page. Parse failures are now detected in a dry pass before anything is emitted. The zero-id guard also only fired when the array held a literal '{', so a subBlocks built by a helper call was silently empty, while a subBlocks whose literals only spread ({ ...sb, required: true }) hard-failed the build. It now fires on the destructive case alone -- reporting an unreadable array while leaving legitimate opaque spreads, which 23 blocks rely on, untouched. Three descriptions referenced things the reader can no longer see: Dataverse mandated base64 after its base64 row was removed, Vanta's mimeType described itself as useful only on that removed path, and five Drive actions told the reader to fetch a next page with no input left to accept the token. * chore(tinyfish): use a white block background Matches the dominant convention (97 blocks use #FFFFFF). Regenerates the docs page, the tool metadata, and the deployment catalog, which each carried the previous value. * fix(docs): correct Drive/Firecrawl/Vanta output and param descriptions Google Drive nextPageToken: the discovery doc says the field is *absent* at the end of the list, not empty. Say absent, and name the resource (files/comments/permissions/revisions) per tool. Firecrawl search outputs: restore the per-format gate the v2 OpenAPI states ("HTML content if requested in formats"), and add that Search exposes no input for those formats. Vanta mimeType: the route resolves a content type from storage on every path, so the param is never read. Say so instead of describing it as a fallback. Dataverse: describe the request as sending the bytes as the raw body. * fix(docs): correct the Vanta mimeType and Firecrawl reachability wording Vanta: the base64 branch (route.ts:100) reads params.mimeType as its only content-type source, so "not currently applied" was wrong. Say it applies there and is a fallback on the File branch. Firecrawl: scrapeOptions is declared in the block's inputs map with no subBlock, so it is reachable by a direct tool call. "Exposes no visible input" rather than "exposes no input". * fix(docs): never abort the generator on an unreadable subBlocks array An unreadable `subBlocks` value used to throw, and with no spread base to fall back on the failure was fatal: the pre-scan recorded it and `generateAllBlockDocs` returned false, so `main` exited 1 and nothing was written at all. Nine shipped blocks already use the non-literal form and are saved only because they happen to spread a base — the first block authored as `subBlocks: myFields` without one would brick `generate-docs` and `docs:check` for the whole repository. The author's reason for aborting was sound: an empty `userSettableParamIds` is indistinguishable from "nothing is settable", which strips every hidden param and publishes a wrong page. So the fix is not to treat the failure as empty — it is to represent UNKNOWN distinctly. `extractBlockSuppliedParamIds` now returns `{ ids, mapperIds, parseError }` with `ids: null` for UNKNOWN, that `null` flows through `BlockConfig.userSettableParamIds`, `getToolInfo` and `extractToolInfo`, and the filter site skips filtering entirely when it sees it — restoring the pre-filter behaviour for that one block instead of killing the run. `getToolInfo`'s default is `null` for the same reason: `[]` as a default silently meant "strip everything". The mapper scan now runs before the subBlocks scan, so a spread-inheriting block keeps its mapper's renames when only the subBlocks scan fails. With nothing left that can record a fatal, the dry pre-scan and its reporting are removed. Also fixes a silent blind spot in the mapper scan: both key regexes require a literal `:`, so a mapper returning a shorthand property (`{ file }`) or writing a computed key (`result['file'] = …`) dropped a real user input from the docs with no warning. Shorthand names are read from the depth-1 comma segments of brace-matched regions, which keeps call argument lists from contributing names. Verified byte-identical output: `scripts/generate-docs.ts` and `tool-metadata:generate` reproduce all 302 generated files unchanged, the credential-shaped hidden params stay stripped, and `check:audits` passes. * fix(docs): name the input that gates Firecrawl search scrape output The previous wording ended each description with "for which the Search operation exposes no visible input", a relative clause that attaches ambiguously and never tells the reader what controls the field. Name scrapeOptions and note that it is hidden. * fix(docs): say the Vanta mimeType is ignored for File-input uploads Every return path of downloadServableFileFromStorage yields a non-empty contentType (a literal, getMimeTypeFromExtension's GENERIC_MIME_TYPE fallback, or resolveServableDocBytes' constants/getContentType), so resolved.contentType always wins at route.ts:79-81 and params.mimeType is unreachable on that branch. It is not a fallback; it is ignored. * fix(docs): note the hidden inputs that gate Pulse html and figures output extractFigure and returnHtml are visibility: 'hidden' with no subBlock, and parser.ts:135-144 only forwards them when defined, so neither output can be produced today. Say so on the output rows rather than deleting them, since removing an output field would break saved block references. chunks is left alone: chunking/chunkSize are user-only with real subBlocks. * fix(docs): report a spread-only subBlocks array as unknown, not empty extractUserSettableParamIds answered [] for a subBlocks array whose every element spreads a fields array it cannot follow (NotionV2Block's `[...NotionBlock.subBlocks, ...getTrigger(x).subBlocks]`). [] asserts the block supplies nothing, so the hidden-param filter stripped every hidden param from every tool the block owns - silently, with no parseError and so no warning. That is the exact false-drop the null UNKNOWN state exists to prevent. Return null in that case and propagate it: extractBlockSuppliedParamIds no longer folds it into [], and the block pass no longer collapses it with `supplied.ids ?? []`. A config-level spread base still narrows the filter to its readable fields plus the mapper's renames; with no base the filter is switched off. An array with at least one inline id, a genuinely empty array, and the existing throw/warn paths are unchanged - all 8 warned blocks warn identically and every generated page is byte-identical. Also pin the hidden-param filter on extractToolInfo's source-parsing path, which had no coverage at all: deleting it outright left the suite green. * fix(daytona): stop the lifecycle tools crashing on a non-string sandboxId start/stop/delete echo sandboxId back as the output id when the API returns no body, via params.sandboxId.trim() inside transformResponse - after the request has already gone out. sandboxId is declared type: 'string' but arrives unvalidated, and now that safeUrlPathSegment accepts a numeric id a number builds a URL, sends the DELETE/START/STOP, and only then throws an unnamed TypeError. Both the old and new behaviour fail, so this is not a regression of a working workflow, but for delete_sandbox the side effect is irreversible and the caller cannot tell what happened. Fixed with a shared resolveSandboxId in utils.ts rather than a coercion at each of the three sites: utils.ts already owns every sandbox-id helper, the three tools already import from it, and the reasoning belongs in one place. The encoded value cannot be reused - it is percent-encoded and would be wrong as an output id. Behaviour for a string is unchanged. * docs(url-path): drop the false claim that widening restores prior behaviour The module TSDoc said the number/bigint widening fixed 'a regression for the call sites whose pre-guard form was a bare ${params.id} template that stringified a number fine'. It did not. Every pre-guard form in a42299066d used .trim() (`/v13/deployments/${params.deploymentId.trim()}`, `sandboxId?.trim()`), so a numeric id threw there too - no importer has ever accepted one. The cited examples were also wrong: only Vercel and Daytona import this module, and neither Box nor X does. Replaced with the real motivation - params are declared type: 'string' but nothing enforces it before the guard, and the old coercion-to-'' turned a supplied numeric id into a misleading 'is required'. Two test comments made the same claim ('still stringifies', 'replaced bare ${params.id} templates') and are corrected; no assertion is weakened. * fix(docs): correct output rows that cite inputs Sim does not send mistral_parse: the PR removed the includeImageBase64 input row but left the image_base64 output citing include_image_base64=true, so the page referenced an input it no longer documents. includeImageBase64 is visibility: 'hidden' with no subBlock, mapper or canonicalParamId, so it is annotated the same way Pulse's html and figures were. The sibling rows are a stronger defect: table_format, extract_header and extract_footer appear nowhere in the repo - not as tool params, not in the request body parser.ts builds - so tables/header/footer cited options Sim never sends. Worded accordingly rather than as hidden inputs. pulse structured_output cited 'if schema was provided', but there is no schema or structuredOutput param in the tool, in pulseParseInputSchema, or in the outgoing body, so the field is always null. No output field is deleted - removing one changes the block's output schema and could break saved workflow references. * style: wrap long description literals to satisfy biome Formatting only — regenerating both artifacts produces a byte-identical tree, so no description text changed. * fix(docs): cite the real Mistral options behind tables, header and footer The previous wording was self-contradictory on tables: it described placeholder-referenced table objects and then asserted the list is empty. Mistral's OCR API does expose table_format, extract_header and extract_footer. table_format defaults to inline markdown, so the separate tables list stays empty; extract_header and extract_footer default to false, so neither field is returned. Sim sets none of the three. Name the option and its default in each description instead of asserting an outcome the request body alone does not establish.	2026-08-27 19:50:03 -07:00
Theodore Li	57235de9aa	feat(mcp): add Codex client configuration (#7164) ... * feat(mcp): add Codex client configuration * fix(mcp): keep Codex key creation action	2026-08-27 19:28:58 -07:00
Waleed	e22324f979	chore(docs): refresh product screenshots (#7192) ... * chore(docs): refresh product screenshots * fix(docs): align HubSpot image dimensions	2026-08-27 19:23:50 -07:00
Vikhyath Mondreti	d28e8d7daa	refactor(tools): execute internal operations in process (#7179)	2026-08-27 18:21:25 -07:00
Waleed	1465e94574	feat(tinyfish): add TinyFish web agent, search, and fetch integration (#7177) ... * feat(tinyfish): add TinyFish web agent, search, and fetch integration Adds the TinyFish integration: eight tools across the Agent, Search, and Fetch APIs, a block wiring them, the brand icon, and hosted-key support. Agent runs are metered on the step count TinyFish reports. Search and Fetch are free, so their hosted key costs nothing to run. The async run and its run read/cancel/list companions carry no hosting config — their charge accrues after the request returns and cannot be metered — so they always require the caller's own key. * fix(tinyfish): address review findings - Add tinyfish to PROVIDER_SECTIONS. The sectioned BYOK renderer drops any provider missing from a section, so the key field never rendered. Export PROVIDERS/PROVIDER_SECTIONS and assert they agree, so the next provider to miss a section fails CI instead of vanishing. - Replace the `any` payload params with raw snake_case wire types. This caught two real gaps: `status` could reach the output undefined, and `error` could be null where ToolResponse.error is `string | undefined`. - Reject an already-parsed array output schema, which a `json` block input can produce, and name malformed schema JSON instead of leaking a SyntaxError. - Count Fetch rate-limit usage from the submitted URLs rather than the returned arrays, so a URL in neither array cannot undercount. - Make "List runs" a literal canvas clause; a blank goal filter lists everything. - Regenerate the docs manifest for the new integration page. * chore(byok): audit that every hosted provider is fully wired Adds check:byok-providers. A hosted tool names its provider once in hosting.byokProviderId, but the id must also reach the zod enum, the settings PROVIDERS row, and a PROVIDER_SECTIONS section. Only the union is compiler- enforced; the rest fail silently, which is how TinyFish shipped with no settings row in the first place. Also flags drift between the two BYOKProviderId declarations. Correct the Search and Fetch rate-limit rationale: TinyFish documents both ceilings per API key, not per account, so the numbered key pool does raise the total. Records why a free product is still worth hosting — the endpoints require a key, so hosting is what removes the signup.	2026-08-27 15:18:30 -07:00
Waleed	479ae8c138	fix(cli): seven defects found by live-testing the CLI against staging (#7176) ... * fix(catalog): resolve an unversioned tool id against the visible set `tools get github_comment` answered NOT_FOUND while `github_comment_v2` worked, though the toolId help promises an unversioned name resolves to the newest version. A superseded tool stays in the registry, so `resolveToolId` short-circuits on the exact hit and returns it unchanged; the visibility gate then refuses it because no visible block exposes a v1 tool. 204 base names were unresolvable this way. Resolution now walks the visible set newest-first, the way blocks already do. `resolveToolId` is untouched — execution depends on an exact id returning that exact id, and none of the 5182 visible ids change under the new path. * fix(files): resolve an archived folder path through its active ancestors The archived folder listing built its path map from the archived rows alone, so a folder whose parent is still active came back as its own name. Deleting `a/sub` and restoring `a/sub` therefore disagreed — restore only matched the truncated `sub` — and the path and parentPath fields were wrong. The extra read is taken only for the archived scope; active and all keep their single query, which a test now pins. Restoring by path also stopped guessing. Archiving, recreating and archiving again leaves two archived folders with the same canonical path, and the resolver took the first match, silently restoring the wrong one. It now refuses and names the folder-id form. * fix(v2): answer a folder-list miss with an empty page A parentPath naming no folder returned 404 on the workflow, table and knowledge folder lists, and an empty page on files. The rule the codebase already publishes is the empty page: V2_FOLDER_FILTER_MISS is appended to the folderPath filter on six list surfaces, and resolveFolderPathFilter documents why a list must not become an existence oracle — a 404 claims the collection is missing and breaks a walk when a folder is deleted mid-pagination. Both TSDocs asserted the sibling folder lists already behaved that way. They did not; that premise is corrected here too. Mutations keep every 404. The miss short-circuits before the row query, because an unfiltered parent id lists the whole workspace. * fix(cli): gate activating a deployed version `workflows activate create` switched which version production serves with no confirmation, while `rollback` refused without --yes. They are the same application operation under two transitions, so gating one and not the other was an accident of naming. The destructive-operation classification in the client tests listed activate as non-destructive, which is what kept its sweep from noticing. Moved, so two independent tests now hold the gate. * fix(cli): name the profile in the suggestion configure prints Refusing a root global printed a command to save it — without --profile, so following it verbatim wrote the default profile and left the named one untouched. The neighbouring suggestions in this file already carry the flag. Resolution matches resolveProfile, so SIM_PROFILE is covered too, and the profile name is redacted like the value beside it. * fix(cli): fail a row delete that matched nothing `tables rows batch-delete` exited 0 when none of the named rows existed, while the table equivalent exited 1 on the same shape. Only the id-list selection is checked: a filter answers without a requested count, so the guard self-excludes and an idempotent sweep still exits 0 on its second run. * fix(cli): show the -- escape for an id that opens with a dash Short ids draw from a 64-character alphabet containing one dash, so 1 in 64 open with one and commander reads it as an unknown option. It reaches `audit-logs get` and the custom-tool commands, and the escape was documented nowhere. The hint is appended only for a lone dash followed by two or more characters carrying an uppercase letter or digit — a shape no flag on this surface has — so a misspelt flag keeps commander's own suggestion. * chore: regenerate the API reference and CLI surface * fix(cli): quote a profile name a pasted command would otherwise split The suggestion configure prints is meant to be pasted, and it interpolated the profile name bare. Profile-name validation is creation-only by design — the validator says so, because a hand-written `[profile my stack]` has to keep resolving — so a name carrying whitespace, or a `;` that would end the pasted command and start another, reaches this message unchecked. Names that already satisfy the creation rule stay bare; the rest are single quoted, embedded quotes included. Redaction runs first, so a control character becomes a space and is then quoted rather than splitting the command.	2026-08-27 15:14:48 -07:00
Waleed	e906190bae	fix(cli): refuse what the help already says is invalid (#7157) ... * fix(cli): refuse what the help already says is invalid - validate integer fields against the generated spec's own `kind`, so `--limit 1.5` names the flag and the value instead of surfacing zod's `expected int, received number` - enforce `--limit` >= 1 on the two non-paginated row mutations, which the help already promised and the server already required - refuse a blank `-c/--conversation`: it is falsy, so it was dropped from the body and silently started a new conversation instead of continuing one - refuse a blank `chat` message before the request rather than after - constrain `--recipe` to the recipes the generated body type declares, so regenerating the surface breaks the build if they diverge - announce truncation on `ls` the way `list` already does; the capped answer was printed silently - stop a successful message translation from vetoing itself: the veto now reads the message the server sent, not the rewritten one, which restores `--folder must name an existing folder` and unblocks 100+ operations - drop "destructive" from the `--yes` help line; the gate also covers operations that only add, such as `files unzip` - fix the `logs follow` help, which illustrated `--workflow` with a file id * fix(cli): refuse a fraction the integer parse silently drops Above 2^52 a double's spacing is 1, so Number('4503599627370496.5') is an integer and the safe-integer guard passed it — the API received a value the caller never typed. Read the raw text alongside the parsed number. Digits that are all zero are not a fraction, so 1.0 stays a whole number. Also corrects a chat test comment that described a UUID check the command does not perform; it refuses only a blank -c. * fix(cli): redact the value the root-flag refusal suggests The refusal prints a command for the caller to run, and interpolated the value verbatim. A U+2028 in it split the terminal line, so the tail rendered as a second, plausible-looking suggestion. redact() is what the other twenty messages in this package already use, including one forty lines below.	2026-08-26 22:34:39 -07:00
Waleed	b44b537ab7	docs(self-hosting): correct what a server change clears, and the cert requirement (#7158) ... * docs(self-hosting): correct what a server change clears, and the cert requirement The page said the built-in browser's profile survives a server change. It does not — the teardown clears the browser's saved sessions and the agent's folder grants along with the saved route, because those are capabilities granted to a specific deployment. The page now lists what is cleared and what is kept, says why, and notes that a change which cannot complete is refused rather than half-applied. Adds two things a self-hoster hits in practice. Certificate errors are rejected outright with no "continue anyway", so a private CA that is not in the system trust store will not load however correct the URL is — worth saying, since a private CA is a normal self-host setup. And packaging your own shell needs Xcode 26 or newer, which otherwise fails with an opaque actool error. Also notes that the CLI asks which deployment you mean when a machine has more than one configuration, and states signing/notarization for a self-built shell as a requirement rather than predicting what happens without credentials. * docs(self-hosting): point custom builds at package:mac, not package:share package:share is the "send someone a build to try" path. It passes -c.mac.timestamp=none to skip the per-file round trip to Apple's timestamp authority, and its own docstring notes distribution builds need those timestamps. Apple's notary service requires a secure timestamp, so a build made that way cannot be notarized however many credentials the operator supplies — which is exactly what the section was telling them to do. package:mac inherits notarize and hardenedRuntime from electron-builder.yml and leaves timestamps on, and bun run build honours SIM_DESKTOP_DEFAULT_ORIGIN the same way, so the baked-origin instruction is unchanged. Its artifact path and name differ from the share script's per-channel overrides, so those are corrected too, and the two stacked warnings are merged into one. * docs(self-hosting): spell out that APPLE_API_KEY is a path to the .p8 The variable holds an absolute filesystem path to the App Store Connect key file, not the key material, and @electron/notarize reads it through Node fs so a leading ~ is not expanded — the release workflow carries a comment saying exactly that. Listed alongside the other credentials with no explanation, it reads like somewhere to paste the key, and notarization then fails while every variable looks set.	2026-08-26 22:22:55 -07:00
Waleed	6bbabbbac6	fix(cli): close follow-up gaps (#7137) ... * chore: run the orphaned migration-safety test, and stop publishing real-looking ids One of the eight script tests was reachable from no entrypoint, so it had never run in CI — it passes, it was simply never invoked. This is the second time that hand-maintained list has drifted from the files beside it; the audit runner's own header records the first. The guard against a third is a `check:*` script rather than a test, because the runner derives its list from that namespace and so picks the guard up by name — a test would have had to be hand-added to the very list it guards. A published spec sat outside the generator's manifest and so outside its drift check, and carried six example ids with the texture of real generated ones rather than the pandigital placeholders the rest of the repo uses. The new check globs the directory instead of reading the manifest, since the manifest is what omitted the file. The one pre-existing borderline id is allowlisted with a reason: loosening the threshold to admit it would have hidden one of the six. * fix(cli): close the gaps black-box testing the shipped CLI found `sim profiles <anything>` still exited 0, so a probe reading the exit code to ask whether a command exists was told yes — the one group the earlier guard missed. The exemption was written for commands that are both a group and a leaf, but only `files restore` takes an operand; `profiles` takes none. Registering its listing as a default subcommand puts it back among the pure dispatchers the existing guard already covers, so the guard itself did not need widening. Three commands refuse a workspace API key and said nothing, while their menu siblings said so — reading as though they accept one. They are hand-written, so they never reached the code that appends the note. That note now comes from a helper taking the operation, so a command names the operation it invokes and the two cannot disagree, and a test fails if a hand-written command ever calls a restricted operation without it. A blank numeric value in a request body still became a real zero, the same coercion already fixed for query strings: the guard keyed off the slot when the distinction is the field's declared type. Twenty-one fields across fifteen operations were affected. An empty body string still clears a description. Blank values for the root endpoint, workspace and profile flags fell back to what was configured instead of being refused, and a whitespace workspace was accepted verbatim. A hand-written profile name carrying padding listed as reachable but resolved to defaults rather than erroring. Two schema descriptions named request fields that no flag spells, and a rejected value was echoed unredacted by four messages while their siblings redacted it. A write now re-emits a section header it was not asked to touch byte for byte. The blank-line normalisation around it is left alone: making the writer position-faithful is a change to its model, not a fix. * fix(scripts): match example uuids case-insensitively in the spec audit The pattern only recognised lowercase hex, so an uppercase id in a published spec was never examined and the audit reported success without having looked at it. Matching case-insensitively is not enough on its own: hex is case-insensitive, so a mixed-case id counts `A` and `a` as two digits and reports twenty distinct ones rather than sixteen. That inflated count clears the threshold the texture test uses to recognise a hand-authored placeholder, so a real id could have passed for one. The allowlist is an exact-string lookup and would likewise have missed an uppercase spelling of an entry. Both checks and the lookup now take a normalised id, while the finding still reports the spelling as it appears in the file. * fix(cli): stop a refusal being swallowed, and gate example ids by name A blank root flag was refused everywhere except `profiles`, where the catch that lets a broken profile still list absorbed it and the command exited 0 after printing the table. The refusal now carries its own error class, which is what the listing rethrows on — the two are distinguished by type rather than by matching message text, and a genuinely broken profile still lists. The unknown-profile message redacted the name the caller typed but not the suggestion or the list of configured names beside it, which come from the same file and are equally attacker-influenced once it has been hand-edited. Those are redacted now, as is every other message in these two files that quotes a name read out of the config, and the profile listing flattens the names it renders the way it already flattened the error column. The example-id audit judged a uuid by its digit texture, on the premise that a real one essentially never looks hand-authored. Measured against ten million generated ids, 0.81% of them do — one in 124, where this change alone replaced six. Requiring each digit exactly twice takes that to zero but rejects all fourteen placeholders now in the specs, so it is no cheaper than the alternative. The audit now holds the eighteen ids the specs actually use, which is one file rather than the twenty-seven a reserved format would touch, and a new id fails until someone lists it — which is the review the check exists to force. * fix(scripts): match the uuid sentinels exactly rather than by shape Accepting any id built from at most two distinct hex digits let something through that was never on the approved list. A generated id essentially never has that shape, so the practical risk was small — but this check had just stopped being a shape test and become a list, and a structural exception is the one thing that undoes that. The two ids it exists for are the nil and max sentinels, and both are matched by value now.	2026-08-26 17:53:39 -07:00
Waleed	f02ee99ce2	feat(desktop,cli): support self-hosted desktop installs (#7136) ... * feat(desktop,cli): support self-hosted desktop installs The desktop shell was already origin-agnostic at runtime — navigation, CSP, cookie partition, and the update feed all derive from the configured origin, and every deployment already serves /api/desktop/update/download and its updater manifest. The one thing missing was a way to change that origin: ConfigStore.setOrigin had no IPC channel, menu item, or UI behind it, so a self-hoster installing the signed build was stuck on the baked default. Adds the native server picker (Sim → Server…, plus a "Change server" button on the offline page, since a shell pointed at an unreachable origin lands there with nothing else to click). Its IPC family is gated to bundled file: senders: the surface that repoints the shell must keep working when the current server cannot be reached, and must never be drivable by a page that server serves. A confirmed change relaunches rather than swapping in place — the origin keys the cookie partition, update feed, encrypted per-origin task state, and every live browser view and PTY. Adds `sim-setup desktop`, which resolves the installer from the operator's own deployment, checks that the update feed resolves too, and prints the server URL to paste in. Documents the whole path under self-hosting, including the build-your-own escape hatch for organizations with their own Developer ID. * refactor(desktop,cli): review pass on self-hosted desktop support Two real defects found while auditing the change for hardcoded assumptions. `lastRoute` is a single global setting that carries a workspace id, so it survived an origin change and opened /workspace/<old-id> on the new server. resolveStartRoute cannot rescue that — it discards a route only on a confirmed 403, and a fresh partition draws a 401. Cleared on change, via a named list that is now the documented home for deployment-scoped settings; the agent browser's jar and its known-sites metadata are deliberately kept, together, since changing deployments does not imply the account changed. The offline page's "Check status" sent self-hosters to status.sim.ai, which reports on Sim's deployments and is always green for theirs. Withheld for a non-sim.ai origin, as is the same link in the Help menu, through one isSimCloudOrigin predicate. Hiding it needed `button[hidden]{display:none}`: the page's own `button{display:inline-flex}` is an author rule and outranks the UA `[hidden]`, so the attribute alone left it rendering. The e2e offline test now asserts the whole path, which covers the `server:` local-page IPC gate. Review cleanups: setOrigin no longer rewrites settings when handed the origin it already stores; the picker window installs a permission handler and pre- paints its background like every other window, and its page is theme-aware so that background is not a flash; the CLI reuses httpHealth and the cross-platform openBrowser instead of reimplementing both, skips Compose/Helm discovery when --url makes it dead, and folds two parallel switches into one exhaustive one. Value-flag parsing is now one helper instead of a third copy. * fix(desktop,cli): scope deployment capabilities to their origin Changing the server left two device-global stores in place that grant the INCOMING deployment authority the user only handed the outgoing one: local filesystem grants (directories its agent may read, plus security-scoped bookmarks) and the agent browser's cookie jar (live third-party sessions its agent may drive). Sign-out clears exactly this pair; an origin change is the same boundary, so it now clears it too — awaited before the relaunch, since a quit racing an async clear could leave either behind. browserKnownSites goes with the jar it describes, so Sim is never left believing in sign-ins the profile no longer has. The CLI printed the redirect's filename straight to the terminal. It is read out of a Location the deployment chose, so percent-encoded ANSI or OSC survives decodeURIComponent as real control bytes and could forge CLI output; control characters are stripped and the name is bounded before it reaches the spinner. resolveDeploymentUrl took the first source naming an app URL. A machine with both a local checkout and a real deployment would be probed, printed, and opened at whichever enumerated first, silently — so disagreeing sources are now an error naming each candidate and asking for --url, the way resolveFeatureSetupDestination already refuses ambiguity. * fix(desktop,cli): fail closed on origin change, widen terminal sanitizer The capability teardown could partially fail and still let the shell move. Sequential awaits meant a filesystem-grant rejection skipped the browser-profile clear entirely, and the new origin was already persisted by then, so the incoming deployment inherited whatever survived — and startup restores it. Now the two stores clear independently via allSettled and report which ones survived, and the whole teardown runs BEFORE anything is written. A store that cannot be emptied refuses the change outright and names what it could not clear. Nothing is persisted at that point, so refusing leaves the shell exactly where it was rather than half-applying. Validation moved up front for the same reason: a typo now costs no teardown. The terminal sanitizer matched only C0/DEL/C1 by range, so percent-encoded bidi overrides and isolates survived decodeURIComponent and could still reorder what the reader sees without emitting one control byte. Matched by Unicode class instead — Cc covers the cursor controls, Cf covers the bidi ones. Configuration discovery compared raw strings, so a trailing slash, a default port, a host-case difference, or an ignored path read as two different servers and demanded a --url override to settle an ambiguity that did not exist. Now compared on the parsed origin, which is what the command ends up using. * fix(cli): sanitize the installer name by Unicode group, not by escape list U+2028 and U+2029 are Zl/Zp, so the Cc/Cf filter let them through and a deployment-controlled redirect filename could still forge a status line. Enumerating what to strip had cost a patch per class found — C0 and C1, then the bidi overrides, now the line separators — so this keeps whole groups instead. `C` removes every control, format, surrogate, private-use, and unassigned code point, covering ESC and OSC, the bidi overrides and isolates, zero-width characters, and the BOM; `Z` removes every space, line, and paragraph separator. Separators become a plain space rather than vanishing so a name is not run together at the seam, and runs are collapsed so the result cannot be padded to push text off the line. The regression table now names each class that reached the terminal in an earlier round, so a future bypass says which one came back. * fix(desktop): serialize server changes and report partial teardown honestly Two problems in the same transaction. The picker re-enabled Connect whenever the field changed, including while a request was in flight, so typing and pressing Enter could start a second change that interleaved its teardown and its write with the first — the later write, not a single transition, deciding the next server. The transaction is now serialized in the main process, the way the sign-out coordinator guards its own teardown, since the IPC boundary is reachable regardless of what the page does; the page keeps its button disabled for the whole request so it never asks for something it will only be refused. The stores clear independently, so one can succeed while the other fails. There is nothing to roll back to — a revoked cookie jar and deleted security-scoped bookmarks cannot be un-deleted — and moving anyway would hand the incoming deployment whatever survived. So the change is still refused, but the message no longer names only the failed store as though nothing else had happened: it says some local access may already have been cleared, and that retrying finishes the job. Clearing an already-empty store succeeds, so a retry is safe.	2026-08-26 17:43:11 -07:00
Waleed	c930830310	fix(v2,cli): second audit pass over the v2 API and CLI (#7126) ... * fix(chat): resolve a caller-supplied conversation id through its owner The v2 chat route used the caller-supplied conversationId verbatim, with no existence, owner, or workspace check, against a store keyed by bare text with no owner column. A caller who knew another user's conversation id reached that conversation. Ids now resolve through the same owner-scoped loader the web chat path uses, and anything unresolvable answers one uniform 404 before any lifecycle work runs. Omitting the id mints a server-issued conversation. The contract also accepted any 1-128 character string for a column typed uuid, so a malformed id raised a driver error and rendered 500 while an unknown but well-formed id rendered 404 - a shape oracle, and a 500 on ordinary input. The ownership predicate had no coverage anywhere: the route test mocked the module and the lifecycle test drove a chain mock that ignores its where clause, so deleting the owner condition left both suites green. It is now asserted by composition and by condition count, which is what catches a dropped condition. Also renames the reply's model identifier away from a term the project's own copy rules forbid on a user-facing surface. * fix(v2): conceal workspace absence, and stop archived tables faulting their page Two reads answered a caller more than they were entitled to know. A workspace a caller cannot reach at all returned FORBIDDEN while one that does not exist returned NOT_FOUND, so a workspace-key holder could enumerate which workspace ids exist by diffing the two. Both now answer the same absence, using the concealment policy the billing routes already use. A refusal from inside the workspace - a member whose role is too low - still answers FORBIDDEN, because that caller already knows the workspace exists. Separately, archiving a folder cascades onto its tables but leaves each table pointing at the archived folder row. The archived listing resolved those paths strictly, so one such row faulted the whole page and no cursor could step past it - which also made the ids undiscoverable and left restore unreachable for exactly the tables that need it. The archived scope now resolves leniently to the root, where a restore would place them, matching the shipped workflows behavior. Active listings still fault loudly on a dangling folder. * fix(knowledge): validate upload processing options without stranding live sessions recipe and lang were accepted as free strings up to their length caps, silently discarded, and echoed back nowhere, so a typo was unobservable: uploading with a misspelled recipe returned 200 and quietly used the default. Both are now validated at the boundary and a bad value answers 400 naming what is accepted. The accepted recipe set deliberately includes the sentinel every first-party caller sends today alongside the three real chunker recipes, and the three are derived from the chunker's own union so removing one there is a compile error here rather than a silent 400 in production. The same schema also parses metadata read back off a persisted upload session, so tightening it would have thrown out of resume and complete for any session created before this - a 500 on work that could then never finish. The read-back path now drops a value it no longer recognises instead of rejecting it; the request boundary stays strict. Neither field reaches chunking, so nothing here moves chunk boundaries, embeddings, or search results. * fix(v2): honour a requested stats window, and answer a claimed graph id with a conflict Log statistics accepted a start and an end, filtered the totals by them, and then built the series against wall-clock now. Bucket width was computed over a span the caller never asked for, and every bucket past the requested end was structurally empty - so a bounded historical query returned a wrong-width series with fabricated trailing buckets, under a window label that disagreed with the request. Each edge now honours the bound it was given and keeps its previous derivation when omitted, so an unbounded request is unchanged. Separately, block, edge and subflow ids are global primary keys while the delete that precedes a state replace is scoped to one workflow. An id owned by another workflow survived that delete, the insert violated the key, and because callers pass their own transaction the driver error escaped unclassified as a server fault. The write now refuses such an id up front with a conflict naming it, and re-classifies the same violation if one races past the check, since the lock covers only the workflow being written. The dry run checks the ids a commit would insert and reports the warnings a commit would report, which is what its own contract already promised. * fix(secrets): let a workspace secret change its metadata without resending the value Restoring redaction cost more than removing it. The only way to flip a secret back to redacted was to re-send the plaintext, because the write required a value and omitting it fell into an interactive prompt that cannot run in CI. A workspace secret can now change its description or visibility on its own; the stored value is never re-encrypted or rewritten, a write that names no existing secret answers not-found rather than creating one, and a personal secret still requires a value because it has no other writable field. The path parameter was also one shared schema across the write and the delete, so a single description had to cover both and the delete documented an argument that could create and replace. Split, mirroring the credentials pair. The metadata write is a new update against the credentials table, so its scope is asserted by composition and by condition count: an unscoped update would let one workspace flip another workspace's identically-named secret out of redaction, and the cache invalidation would then carry that flag into the other workspace's runtime catalog. * fix(v2): say what an error means in terms the caller can act on A size-limit refusal collapsed every value under a kilobyte to "0 Bytes", so a 28-byte file over a 27-byte ceiling read "is 0 Bytes, above the 0 Bytes limit" - self-contradictory, and useless for choosing a value that would work. Errors and field descriptions also told callers to invoke raw HTTP endpoints. These strings serve the REST reference and the CLI's own help equally, so they now name the operation and its object rather than a method and a path. A sweep test walks every v2 schema description and holds the line, with the remaining offenders in files this change does not own recorded explicitly rather than left to be rediscovered. Listing the editors of a built-in skill claimed the skill did not exist, while reading the same id succeeded - a well-formed request for a real resource is not malformed, so the list answers an empty roster and only the mutations refuse. Bulk folder deletion recorded only the leaf name in its audit trail while the single delete recorded the full path, leaving two same-named folders under different parents indistinguishable after the fact. Bulk chunk enable, disable and delete each treated an unmatched id differently behind one sentence of documentation. They now follow one rule. A workspace-scoped list refused with the name of a resource the caller never addressed, which reads as an empty workspace rather than an unreachable one. * fix(cli): stop a config value forging a section it was never meant to write The config file is written by joining names and values into INI lines, and nothing checked what was in them. A profile name carrying a newline and a section header wrote a section that merged into a different profile and took over its endpoint - and the next command sent that profile's stored API key there. A workspace value could do the same from the other side, since only the endpoint flag validated its input. The refusal now lives at the writer, the single place untrusted text enters the document, with the flag-level checks kept for the better message. Either alone blocks the forgery; the pair is deliberate. Rejecting rather than escaping, because the format has no escape syntax and these files are hand-edited and read by other tools that would not decode one we invented. The forbidden set covers control characters and the two Unicode line separators, which the previous guard missed - those parse as an unreadable line, so the key silently vanished on read and the next write appended a duplicate while the command reported success. Login also wrote the key before the settings, so a malformed response from the deployment could leave a key on disk with no endpoint beside it, and the next command would send it to the default host. Settings are written first, and the response is checked before anything touches disk. Name validation applies only when creating a profile, so a hand-written one that predates the rule keeps working. * fix(docs): tell the reader which key a command needs, and stop the ids contradicting the CLI Around sixty v2 operations refuse a workspace API key, and the CLI's help said nothing about it - the caller found out from a 403 after the request went out. The restriction is already stated in the API spec, so the generator now reads it from there and the command description carries it. The sentinel sentences are imported from the spec's own constants rather than copied, so a reword cannot silently unmark every command, and the test pins the count as well as named operations because a reword confined to one family would otherwise slip past. The generated reference also rendered an empty default as a sentence pointing at nothing - "Defaults to ." - for every repeatable filter. Omitted now, while false and zero still render, which is the trap that shape of check usually walks into. The hand-written guides used a workflow-shaped id for workflows that the CLI's own help says never names one, and five other families were equally wrong. All of them now match the scheme the CLI declares, consistently per entity across pages, with the shared ones taken from that help text so the two read as one voice. The page documenting every flag was linked from nowhere; both landing links pointed at the overview instead. And the generator's test file was absent from the hand-maintained list CI runs, so its guards never executed. * fix(cli): stop a page-size default capping a destructive filter Every request field named limit inherited the pager's default of 100, but only a cursor-paginated command interprets that flag. The two filter-based row mutations declare no cursor, so the default went onto the wire as a row cap: a filter matching 250 rows deleted 100, exited 0, and said nothing - while the confirmation the user had just answered promised every matching row. The flag's own help offered 0 for everything, which those endpoints reject; the unbounded form is the field being absent. The pager's default now applies only where the pager runs, and the tests pin the omission on the request body rather than in help text. A cap typed alongside an explicit row list was silently ignored; it is now refused on the client, where refusing costs nothing to already-installed versions. Lists also truncated at a hundred with no signal in any format, and the two inventory endpoints that do report truncation had that field dropped on the way out - so a caller reconciling against a clipped list could not tell. One note now goes to stderr while stdout stays a bare array, and a flag raised on a later page survives the fold. Also: a folder whose name contains the separator no longer prints a path that resolves to a different folder; validation errors name the flag the user typed instead of the wire field; an unknown subcommand with --help exits non-zero instead of printing the parent's help; a fractional or negative page size is refused rather than floored; an empty query filter is refused rather than silently returning everything; and the two spellings of the missing-workspace message became one. * fix(cli): gate destructive table imports and fix follow-mode rendering A `tables import --mode replace` empties the table before its first batch, so the only warning was in the describe. It now confirms, and the wording tells the truth per mode: cancelling a replace leaves a prefix of the new file with the originals already gone, while an append re-adds its rows if the file is imported twice. `--yes` skips the gate, and the gate runs before the file is opened. Import and export cancellation carried no describe at all; the import one now confirms, the export one records why it deliberately does not. Follow-mode output truncated cells to whatever the first row happened to measure, so a longer status or workflow name arrived clipped with no signal. Cells now clamp at a shared ceiling and pad to the lock, and the log columns carry width floors so a short first page cannot pin a column narrower than its own values. Interrupting a staged download left the staging directory behind; it is now removed on SIGINT and SIGTERM before the signal is re-raised. `--select-output` without `--follow` selected from a response that does not carry outputs, and said nothing. It is refused client-side, with a separate message for `--async`. Its describe now names what the path addresses. `secrets set` always read a value, even when only metadata flags were passed. Off a TTY that was an immediate refusal, so a metadata-only edit exited 1 in CI for a value it was never asked for; on a TTY it stopped to prompt, and the prompt rejects an empty entry, so there was no way to say "leave the stored value alone" short of re-typing the secret. The read is now skipped and the field omitted, which is what lets a metadata-only edit run unattended. On a TTY, setting only a description no longer prompts. Passing both spellings of the reveal flag is refused rather than silently resolved. Four mandatory hand-authored flags now say so, `billing logs` names its key-type scope, and the dispatch list declares its columns. * fix: close the gaps an adversarial review of this branch found A conflict handler added earlier in this branch was dead code. It read the Postgres error code off the thrown object, but the driver error arrives wrapped with the real one on `cause`, so the check returned false on its first line and the 409 never fired. Its test passed only because it threw a flat shape production never produces. It now reads through the cause chain with the shared helpers, compares the constraint name exactly instead of matching a substring of the SQL, and its test throws the real wrapped error. Resuming a conversation checked its workflow and its workspace but not its type, so a conversation created by the web surface could be continued as a CLI turn. It now refuses through the same uniform 404 as every other mismatch, which closes the same omission on the web posting path. Minting one no longer leaves a blank untitled row at the top of the Chat list. The pre-write check on a minted API key refused fewer characters than the writer does, so a key the check accepted could still fail at the write — after the endpoint beside it was already stored, pairing a new endpoint with the previous key. The two had drifted because the set was spelled three times; there is now one. A description claimed a processed count reported only the chunks that changed. The update returns every row it matched, so re-enabling chunks that were already enabled counts them all. Two OpenAPI sentences promised no conflict detection and no persistence warnings in a dry run, both of which the same branch had just made false. A described window was wrong whenever a start was supplied without an end. Listing the editors of a built-in skill answered a read with a modification refusal on the internal surface. Archived table listings could reach the strict folder projector again through a third scope value the input type still allowed. A metadata-only secret write skipped the guard its personal-scope twin has. The internal document boundary still took the two processing fields as unbounded strings. Truncation was reported only from the response envelope, so a clipped file body, row search and workflow-stats list said nothing. A staged download stopped watching for signals before it finished removing its directory, and cleared every listener for the signal rather than its own. Three tests asserted a contract constant against itself; they now drive rendered help, real argv, or real render output. * chore: regenerate the API reference, CLI surface, and CLI docs The published reference still marked a secret value required and described the delete parameter as one that also creates, the CLI surface still lacked the marker that says which operations refuse a workspace key, and the reference rendered an empty sentence for every repeatable filter whose default is an empty list. * test(cli): use the package's own delay helper in the staging poll The audit bans a hand-rolled setTimeout promise. `sim-cli` does not depend on the shared utils package, and its own idiom is `node:timers/promises`. * fix: act on a second review round, and correct two earlier claims The conflict pre-check read block ids from the wrong side. The writer inserts each block's own `id` field while the check read the record key, and the two can diverge because preparation copies a value under its key without reconciling them. Edges already read the value and subflows are genuinely keyed by the record key, so only blocks were wrong — collecting every family from the values, as first suggested, would have broken subflows instead. A minted API key carrying leading or trailing whitespace passed the pre-write check but failed the writer, leaving the new endpoint on disk beside the previous key. It is refused up front now rather than trimmed: a key is opaque, so trimming would store a value the server never issued and turn a loud failure into an unexplained 401 later. The endpoint normalizer does trim, which is what made a padded `--endpoint` fail only after the browser flow had already minted a key. A metadata-only secret write raced with deletion returned 500, because the follow-up read that only assembles the response body threw an unclassified error; it now reports the same not-found the non-racing miss already gave. An unusable output format in the environment silently printed a table instead of refusing. Two validation messages printed control characters verbatim. A dry run now reports the preparation warnings its own commit path returns. The chat route created a titled conversation and never wrote a message, so it appeared in the Chat list promising content it did not have. Both sides of a successful turn are now persisted; a failed turn still writes nothing, so a question is never stored without its answer. Two claims of mine were wrong. The earlier commit message said `secrets set` sent an empty value that overwrote the stored secret — it did not; the prompt refuses off a TTY and rejects empty on one, so the old behaviour was a clean refusal. And the delay helper commit said this package's idiom is `node:timers/promises`; the package carries its own `sleep`, which is the audit's sanctioned home and has five callers. It uses that now. Also: a test asserting a deadlock stays unclassified could not fail, since every candidate rejects it; it now pins a unique violation carrying no constraint name. Workflow ids spelled with the file prefix are corrected in the remaining fixtures, leaving the genuine file ids alone. * fix: close a credential-misdirection path this branch had opened Making the endpoint normalizer trim handled whitespace around a value but not a control character inside one, and the URL parser removes those from anywhere in its input — so a value that reads as one host could resolve to another, and the profile's key went with it. The flag and environment paths never touch the config writer, so its guard did not cover this. The normalizer now refuses the same character set the writer does, which also keeps the invariant that nothing it blesses can be refused by the write that stores it. Comparing the parsed URL back against its input was the alternative and is wrong: the parser rewrites percent-encoding, case, internationalized hosts and default ports, so legitimate endpoints would be refused. The blank-query guard tested for exactly empty, so a whitespace-only value still reached the wire — as a real zero on a numeric filter, an explicit false on a boolean one, and as an encoded space the server then rejected. It now refuses any value that is blank once trimmed, while a body string keeps its meaning, an explicit zero still sends, and a value with content around its whitespace is passed through untouched rather than trimmed. A graph-id conflict reported 409 on the v2 route and fell through the older persistence wrapper as an unclassified 500. That wrapper now classifies orchestration failures through the cause chain, which also fixes a pre-existing case where a workflow archived between authorization and the locked read reported 500 rather than 404. Persisting a chat turn claimed its row by id alone, so a conversation soft-deleted mid-turn still received the messages and was bumped back up the list. It now requires a live row. A turn whose caller hung up after the model had already answered persisted nothing, though the work was done and billed; it now persists and still reports the connection as closed. An empty workspace id from the login response was read as no workspace at all. A published description still promised a language-tag standard the schema does not enforce. The test asserting that a turn is stored before the final event drained the whole response first, so it held whichever order the code used. It now reads the stream incrementally and fails if the write moves after the event.	2026-08-26 15:05:00 -07:00
Vikhyath Mondreti	db88eca6d4	improvement(billing): replace daily refresh credits with weekly refresh (#7113) ... * improvement(billing): replace daily refresh credits with weekly refresh * fix(billing): scope weekly refresh row seat scaling to organization billing	2026-08-26 12:43:07 -07:00
Waleed	f3ccdfe7de	feat(providers): add GLM-5.3 and GLM-5.3-Flash to Z.ai (#7104)	2026-08-26 10:12:18 -07:00
Waleed	a42299066d	fix(integrations): repair broken endpoints, silent failures, and a path traversal (#7096) ... * fix(integrations): repair broken endpoints, silent failures, and a path traversal - qdrant: search_vector returned the /points/query envelope instead of the points array, so downstream blocks saw an object where an array was declared - elasticsearch: search/count/create_index silently swallowed malformed JSON — count fell back to counting the entire index, search to match_all - serper: only 4 of 6 advertised verticals were mapped; videos and shopping returned empty (still billed) result sets. Replaced the if/else chain with a vertical dispatch table that hard-fails on an unknown type - enrow: find_email flattened the nested info object incorrectly, dropping firstname/lastname, and advertised a linkedin_url the API never returns - linkedin: share_post read postId from an empty body; /v2/ugcPosts returns it in the x-restli-id header - vercel: edge config endpoints moved to /v1/global-config - vercel: encode edgeConfigId so a traversing value cannot escape the base path - sixtyfour: enrich endpoints moved to /people-intelligence and /company-intelligence - langsmith: hard-coded API host consolidated to one constant, added missing non-ok guards, encoded run ids, capped echoed upstream error bodies - daytona: file upload moved to /files/upload-v2 - memory: PUT persisted a bare object where POST and the declared type both use an array Adds 53 tests across the affected tools. * fix(integrations): close path traversal fleet-wide, stop a credential reaching the wire Follows up the review round on this branch. Security: - The previous encodeURIComponent-only guard was incomplete. '.' and '..' are unreserved, so they survive encoding and the URL parser then removes them as dot segments — popping one path segment on a fixed host with the caller's bearer token still attached, including on DELETE. Adds a shared safeUrlPathSegment helper that rejects empty, '.', '..', and any residual path separator, and applies it across every Vercel and Daytona tool that interpolates an LLM-writable id into a request path - langsmith: create_run and create_runs_batch spread the whole params object into the request body, so the LangSmith API key was sent to LangSmith and stored in the run record. Request bodies are now built from an explicit allowlist of run-ingest fields, so an unlisted param cannot reach the wire Correctness: - langsmith: the run-payload normalizer was not idempotent and ran once in request.body and again in transformResponse, so a caller who left Run ID blank got back an id that was never sent. Downstream update_run/create_feedback wired to it would 404 - langsmith: batch patch entries were normalized as if they were new runs, minting ids and overwriting start_time/trace_id/dotted_order - langsmith: the 500-char error cap appended its ellipsis after slicing, so the advertised bound was actually 503 - serper: scholar and patents mapped a date field neither vertical returns, and a test asserted it. Their organic response key is now confirmed against Serper's published per-vertical examples rather than assumed - serper: an unknown vertical derived from the response URL turned a successful response into a thrown error; only a user-supplied type now hard-fails - linkedin: warn when a success status carries no x-restli-id header Also corrects the Vercel endpoint rationale in the PR description: the old /v1/edge-config path still routes and is not scheduled for removal, so the move to /v1/global-config is canonical alignment rather than a break-fix.	2026-08-25 21:04:04 -07:00
Theodore Li	d1786e92e6	fix(custom-blocks): restore self-host entitlement gate (#7098) ... * fix(custom-blocks): restore self-host entitlement gate * chore(helm): bump chart version	2026-08-25 23:45:00 -04:00
Waleed	2d85c0d1e1	fix(cli): correct three descriptions the CLI publishes, and restore --no-recursive (#7093) ... Follow-ups from review of the v0.8.12 release PR, all on surfaces the CLI audit touched. - `credentialId` was one shared schema across PATCH and DELETE, so the disconnect reference offered "update or disconnect" for an operation that cannot update. Split into two, matching the two components OpenAPI already publishes for them. - A boolean query param documents the spellings an HTTP caller may send and closes by calling them the whole accepted set. The CLI renders those fields as bare flags that take no value, leaving the sentence pointing at a list neither `--help` nor the reference ever prints. Stripped for bare flags only; the REST prose and OpenAPI specs are unchanged. - `files list --recursive` became a bare switch in the audit, which removed the only way to send false. The API turns it on by itself as soon as `--search` is set, so a folder search always descended. The twin is back, declared per flag so the one-way toggles do not grow a meaningless negation.	2026-08-25 19:28:06 -07:00
Waleed	4508ec75d2	fix(cli): resolve blockers and majors from a full command-surface audit (#7083) ... * fix(cli): resolve blockers and majors from a full command-surface audit Audit of all 222 CLI leaves against a live deployment, plus fixes for every defect it confirmed. Blockers: - An unrecognized --profile resolved to built-in defaults, so a typo silently targeted production and transmitted the API key there. - sim logs follow sent an undeclared query key and failed on every invocation. - sim workflows run exited 0 on a failed run, so CI reported success. - knowledge connectors documents update matched rows already in the target state, making exclude and restore permanent no-ops. - PDF text layers below the OCR threshold are transcribed by a model and stored verbatim with no record that it happened. Majors include: rollback --version was swallowed by the program-level flag and silently did nothing; nullable string flags could not send null despite their help promising it; six protocol commands discarded excess arguments, dropping files on upload; sim chat crashed with EPIPE when piped to head; tables import dropped malformed CSV rows without reporting them; audit-logs required an organization id no API surface exposed; secrets could not opt out of redaction or read a value from a file; bulk deletes and moves exited 0 having done nothing; and MCP registrations were destroyed by undeploy rather than restored. Adds extraction_method to documents so OCR output is distinguishable from parsed text, and reports the applied scope on billing logs so the two ledger questions are no longer indistinguishable. * fix(mcp): bound MCP restore by server and re-check uniqueness under the lock Two gaps in the archive/restore lifecycle this branch introduced. The candidate query bounded archived rows and deduplicated to one per server afterwards, so several archived generations stacked on one server consumed the whole budget and every other server the workflow had been published on fell out of the result with no warning. Deduplication moves into SQL so the bound applies to servers, preserving most-recently-updated-per-server. The live-registration check ran before the server lock was acquired, so a concurrent tool create could land in between and the restore would un-archive a second live row for the same server and workflow, violating the partial unique index and rolling back the whole deployment. That check now runs under the lock alongside the tool-name, capacity, and metadata-budget checks it belongs with. * fix(review): address round-three review findings across CLI and server Restore now picks candidate servers by recency: DISTINCT ON forces its own key to lead the sort, so bounding on that statement kept the lexicographically lowest server ids and left a workflow's most recently used servers archived. Deduplication and bounding are now separate stages. CLI: a total miss on tables move reported only in notFound exited 0; unsetting a key or removing a profile mutated the first duplicate INI block while reads merged later ones, so the removal appeared to succeed and did nothing; an import that rejected cells but no rows showed a clean progress line; and the --run-id help implied idempotency it does not provide. Server: a run with no recorded output projection let block-name selectors past the new validation; the billing window comparison still fired on a bound that parsed but failed the shared schema; CSV rejection accounting reached only the streaming path, so buffered and synchronous imports still dropped records silently, through to the Copilot tool that reports them; case-insensitive tag name uniqueness now serializes on the knowledge-base row the delete paths already lock; and a failed sync claim reports the lifecycle reason rather than always claiming a sync is in progress. Reverts an over-scrub from the previous commit: workspace-file-imports is consumed only by Copilot, so naming save_upload and glob there is the correct remediation rather than a leak, and the sweep that guards against leaks now exempts it explicitly. Corrects two contract descriptions that promised a bulk tag save would rename or relocate an occupied slot, which it deliberately no longer does. * fix(cli): remove flags a caller cannot use, and correct three that misled Removes surface that should not have shipped: - `files uploads get` is hidden. Its `--upload-token` was required, and the token is minted and consumed inside a single `files upload`, which completes or aborts its session before returning. Nothing in the CLI could produce the value, so the command answered every invocation by asking for something unobtainable. The same flag is dropped from the two table-import commands, where a CLI-created import is already queryable without it. - The `--no-<flag>` companion that sent JSON null is gone. `--no-X` means "send boolean false" on thirty-seven other flags, and one spelling should not carry two meanings. `--description ''` already clears the displayed value, and the help now warns that the literal word null is stored as text rather than suggesting a substitute, because on the OAuth client fields null revokes a stored grant and an empty string does not. - The document extraction method column and its contract field are reverted. Nothing read them, they were null for every existing document, and the name collided with the parser metadata field that already exists. Corrects flags that misled: the workflow move destination is `--to`, matching its two siblings rather than meaning the opposite of `--folder` one command over; `files list --recursive` is a bare flag like the four folder deletes rather than a twelve-alias string; the dispatch row cap takes a count instead of its wire object; `--yes` no longer claims to be required on commands that accept `--dry-run`; cancelling every run on a table is confirm-gated; and the retry-processing negation, which the route rejects, is suppressed. Extends the guard that missed all of this: it swept only `--x-` prefixes over generated commands, so a header spelled without one was invisible to it. It now derives every header name from the operation table and sweeps the assembled program. * fix(mcp): budget MCP restore against the workflow's live server fanout Restore bounded its candidates at the per-workflow server limit counting archived rows only, never subtracting the memberships the workflow already holds live. The fanout validation a few lines later in the same deploy transaction counts live servers against that same limit, so a workflow with both live and archived registrations could restore past it and roll the whole deployment back. The candidate query now bounds on the remaining headroom, and the budget is re-checked under the server locks and spent once per accepted candidate, so a create that lands between the count and the unarchive cannot push it over. Candidates that do not fit are dropped by recency, matching how the set is already selected, and stay archived with a warning naming the workflow, the server, and the reason — restore still never throws inside the deploy transaction. One residual is left open deliberately: a create on a server outside the candidate set is serialized by neither the locks nor the recount. Closing it would need a workflow-level lock, which would change the ordering every other writer here depends on, and the create path runs its own limit check. * fix(mcp): stop restore spending its budget on servers it cannot restore A server can hold both a live registration and archived ones for the same workflow: the partial unique index constrains only the live row. Such a server was counted twice — once shrinking the restore budget, once consuming one of its slots — before the liveness check under the lock skipped it. While the bound was the full server limit that waste was invisible; once the bound became the remaining headroom, every slot spent that way cost a registration that could have been restored. The candidate query now excludes servers the workflow is already live on, so the budget and the candidate set agree. The exclusion sits on the inner stage, before deduplication and the bound, and is a pre-lock optimisation only: the check under the server lock stays authoritative, because the query can go stale between reading and unarchiving. Candidates rejected for a tool-name collision, the per-server cap, or the metadata budget are still not replaced. That case is only knowable under the lock, so replacing it would mean fetching past the bound and locking servers outside the candidate set, widening an ordering every writer here relies on.	2026-08-25 18:43:48 -07:00
Theodore Li	19b8652f35	chore(flags): remove released feature gates (#7087) ... * chore(flags): remove released feature gates * chore(helm): bump chart version	2026-08-25 20:26:40 -04:00
Waleed	98a453d7e6	feat(integrations): add Microsoft Word (#7069) ... * feat(integrations): add Microsoft Word * fix(microsoft-word): guard document edits against concurrent overwrites * fix(microsoft-word): reject non-Word targets, scope SharePoint access, and tighten input bounds * chore(docs): regenerate docs manifest for the Microsoft Word page * fix(microsoft-word): strip XML-forbidden control characters from generated documents * fix(microsoft-word): fail closed when a document reports no version to compare	2026-08-25 11:38:20 -07:00
Waleed	ed60fbaa4e	feat(integrations): add Semrush (#7068) ... * feat(integrations): add Semrush Adds the Semrush SEO API as a block with 44 operations across overview, domain, subdomain, URL, keyword, comparison, and backlink reports. Reports answer as delimited CSV, so the shared decoder maps each header cell back to the export column it was requested as rather than reading by position: the API may return fewer columns than were asked for, and two columns can render under the same header label. * fix(integrations): sync docs manifest for the Semrush page * fix(integrations): address Semrush review findings - Only the API key stays user-only; report selectors (target scope, limit, offset, date, sort, filter) are user-or-llm so an agent can set them - Hold the row limit at one row: a positive fraction floored to zero and sent display_limit=0 - Locate Domain vs. Domain metric columns by their own headers, so a dropped position column shortens the compared-domain run instead of shifting competition, search volume, and CPC onto the wrong values - Describe competitor and domain-list metrics as belonging to the row's domain, not to the target - Describe paid and URL traffic cost as an estimated cost, matching the Traffic Cost header those reports return, not the organic Traffic Cost (%) - Drop the newest-first claim from history outputs, which order by display_sort - Replace the erased any casts in the request tests with a typed helper	2026-08-25 11:17:58 -07:00
Theodore Li	4c0bd944dc	feat(slack): launch v2 triggers and backfill custom bots (#6873) ... * feat(slack): launch v2 triggers and backfill custom bots * fix(slack): propagate legacy webhook dispatch failures * fix(slack): continue shared legacy webhook fanout * fix(slack): acknowledge filtered webhook deliveries * fix(slack): finalize custom bot migration rollout * fix(slack): dedupe migrated bots per workflow * fix(slack): harden custom bot rollout * fix(slack): acknowledge permanently ignored deliveries * fix(slack): retry failed webhook deliveries	2026-08-25 14:11:02 -04:00
WaleedandTheodore Li	656840a1b1	feat(api): make the platform operable headless over v2 (#6912) ... * feat(v2): download run output files by API key Adds GET /api/v2/workflows/{id}/runs/{runId}/files/{fileId}, closing the async-run loop for headless callers. A run's output carries UserFile URLs pointing at /api/files/serve/..., which rejects x-api-key outright, so an async run that produces a file previously had no byte path out for an API key at all. The file is addressed by the id the run reported and resolved against the run's own recorded execution data, from which the storage key is read. The request never supplies a storage key, so the endpoint cannot be aimed at bytes the run did not produce. Resolution deliberately reads the materialized-but-undisplayed recording, because the display projection strips exactly the `key`/`context` fields a byte read needs. Also hardens normalizeStartFile to derive a file's storage key only from a validated internal serve URL, discarding any caller-supplied `key`/`context`. A workspace API key has no human subject, so the executor resolves its actor to the workspace billing owner (preprocessing.ts -> resolveSystemBillingAttribution); verifyFileAccess then authorizes a workspace-context key as that owner, whose reach is not bounded by the key's workspace. Accepting an attacker-authorable key made that substitution exploitable as a confused deputy. Normalization is all-or-nothing, so a forged file now drops the whole files input. * feat(workflows): one graph-write door, principal-derived audit source, and v2 authoring endpoints Extract replaceWorkflowNormalizedState as the single persistence primitive for a workflow graph replace and route both the internal editor save and the Copilot edit tool through it, so neither can skip state preparation, the row lock, the lastSynced stamp, or custom-tool extraction by choosing a different entry point. Derive the audit source from the acting principal instead of hardcoding 'copilot', then widen workflows.variables.apply_operations and workflows.bulk.move to every principal kind. Add GET/PUT /api/v2/workflows/{id}/state, POST /operations, /duplicate, /restore, PATCH /variables, and POST /api/v2/workflows/move over surface-neutral application use cases; move the edit engine to lib/workflows/editing. * test(workflows): cover the graph-write primitive, audit source, and the v2 authoring surface Pin the two-doors fix (preparation runs, the row is locked, custom-tool extraction is post-commit and best-effort) and the false-audit fix (a session principal writes source: 'session', a delegated one writes its service). Both were verified to fail with the fix reverted. Add the application matrix for replaceWorkflowState, applyWorkflowOperations, readWorkflowGraph, and restoreWorkflow — role floor, principal-kind rejection before canonical load, asserted-scope concealment, lock, validation, atomic conflict, plan gate, and audit-then-notify ordering — plus route tests for every new endpoint. * test(workflows): pin the internal graph-write door and the v2 list scope Characterize saveWorkflowNormalizedState's statuses, messages, and notification after the persistence extraction, and cover the new scope filter on GET /api/v2/workflows including a cursor replayed under a different scope. * feat(api): add v2 block, tool, connector-type, and enrichment catalogs Adds six read endpoints under /api/v2 that publish Sim's code-defined catalogs: GET /blocks, GET /blocks/{blockId}, GET /tools, GET /tools/{toolId}, GET /connector-types, and GET /enrichments. These read like static reference data and are not. What a caller may place is decided per workspace by its permission-group integration allowlist, per organization by which unreleased blocks have been revealed, per deployment by ALLOWED_INTEGRATIONS, and per workspace again by the workflows it has deployed as blocks. So all six are plain defineWorkspaceOperation reads at minimumRole 'read' with workspaceApiKey 'allow' — the exact policy of credentials.providers.list — and every response keeps Cache-Control: private, no-store, because an unrevealed preview block's existence must not leak across organizations through a shared cache. Trigger blocks ride as ?capability=trigger rather than a second endpoint, and workspace custom blocks ride inside /blocks discriminated by `source`, so "what may I place?" stays a one-call question. The block projection is extracted out of the Copilot get_blocks_metadata tool and rewritten onto @/tools/metadata and @/tools/metadata-outputs. That cuts the tool's own @/tools/registry edge as a side effect: its module graph drops from 6,756 to 1,318, and the new routes land at 1,673-1,734, next to the shipped /v2/credentials/providers baseline of 1,668. Supporting changes: - scripts/sync-tool-metadata.ts derives hostedApiKey ('always' | 'conditional' | 'none') from each tool's `hosting`. The config itself stays excluded because it holds closures, but "does Sim host the key" is a first-order authoring question, so the answer is emitted. - getCopilotToolDescription takes hostedApiKey as an option instead of reading `hosting` off the tool, so both an executable ToolConfig and the generated metadata can answer it through one shared derivation. - principalUserId / allowedIntegrationTypes move out of lib/credentials/application/provider-catalog.ts into lib/integrations/principal-scope.server.ts. Two copies of the workspace integration gate would diverge first on the workspace-key path, which has no user for permission groups to key on. - scripts/check-tool-registry-boundary.ts walked page.tsx/layout.tsx under app/workspace only, so a route importing the executable registry passed green. It now walks a list of entry sources, seeded with the four catalog route subtrees and the shared projection barrel. Routes are covered per subtree rather than wholesale because 122 of ~1,130 route files legitimately execute tools. Registry sweeps parse every block, tool, connector type, and enrichment through its published response schema and compare against the wire round-trip. They caught a real drift while being written: an operation's inputs were typed as a union of the tool-param and block-input shapes, and the union resolved to whichever member matched first, silently dropping a block input's `schema`. * docs(api): stop publishing a 413 GET /workflows/{id}/state cannot emit * feat(v2): read upload-session state Adds GET /api/v2/files/uploads/{uploadId}. Only DELETE was exported, so a caller that lost track of a transfer could abort it but could not ask whether the session was still alive, already finalized, or failed — the resume story was missing. Runs on a new files.upload.read operation at minimumRole 'read' rather than reusing uploadCancel, which is a 'write': asking about a session must not require permission to destroy it. The GET is a control leg like every other, so it carries the signed upload token and re-authorizes the caller's present workspace permission through reauthorizeWorkspaceUploadPurpose instead of resolving the session on its id alone. * fix(api): reconcile v2 catalog and workflow-authoring integration Merging the catalog and workflow-authoring branches surfaced four issues that neither produced in isolation. - Route and OpenAPI counters were bumped to the same value on both branches, so git merged them as one change while the merged tree holds the sum. Corrects the route ratchet to 1142 and the workflows document to 29 operations (152 total), then regenerates the OpenAPI documents and the CLI surface from the reconciled contracts. - The seven new workflow operations were published in the spec but absent from the workflow API reference groups, which `check:openapi` rejects. - `route-policies.ts` reached `WorkflowOperationsNotAppliedError` through `apply-workflow-operations`, dragging the edit engine — and its diff and comparison dependencies, which reach a client OAuth hook — into every route that uses the shared workflow error policies. The class moves to its own leaf module, mirroring `WorkflowImportError`, and each importer now takes it from there. - The operations route test shadowed that class inside its module mock, so `instanceof` matched a fake and the assertion pinned a message the production class never emits. It now uses the real class and asserts the real message. * feat(v2): extract ZIP archives over the public API Adds POST /api/v2/files/{fileId}/extract and widens files.extract_archive from principalKinds ['session'] / workspaceApiKey 'deny' to admit personal and workspace API keys at the unchanged 'write' role. The widening is an authorization change, so the justification lives in the operation's TSDoc: extraction grants no capability an API key lacks, since every file it writes could be created one at a time through files.create and files.upload.create, both already 'allow' at the same role. It only collapses many calls into one. The previous ['session'] restriction read as an artifact of the UI having been the only caller. Delegated services stay out — no copilot or executor caller exists and admitting one is a separate decision. The response is counts plus the destination folderPath, never the extracted files: a large archive would otherwise materialize thousands of objects into one body. Callers page GET /api/v2/files?folderPath=... instead. The use case returns the internal display path and the adapter projects it to a v2 path, keeping the use case surface-neutral. * feat(v2): extract file text over the public API Adds GET /api/v2/files/{fileId}/text. Text extraction previously sat behind checkInternalAuth on /api/files/parse, a route that also mixes in external-URL fetching, execution-file upload, and multi-file aggregation, so it could not be reused. The parse call is lifted into a thin application use case instead. Runs on the existing files.read_content operation unchanged — it is already workspaceApiKey 'allow' at the read role, and turning bytes it already authorizes into text grants no further reach. `degraded` is a required, non-optional boolean on the response. The legacy doc and ppt parsers deliberately return best-effort or placeholder content rather than throwing, so an omittable flag would let a client that never checks it treat guessed text as extracted text. It is reported honestly rather than converted into an error, because the parsers' behaviour is deliberate and characterization-tested. The read is bounded on its input at 25 MiB before extraction rather than on its output after, given the parsers' documented DoS history; a caller may lower the ceiling but never raise it. * feat(v2): restore archived folders and list the archived set DELETE /api/v2/files/folders archives recursively, so a recursive delete was unrecoverable over the API: the archived files stayed visible through GET /api/v2/files?scope=archived, but nothing could rebuild the folder structure. Adds POST /api/v2/files/folders/restore, path-addressed like the rest of the v2 folder family, and a `scope` selector on the folder list so a caller can find the archived path to hand it. `scope` extends the files folder-list query rather than the shared v2ListFoldersQuerySchema: only workspace files have an archived folder set, so adding it to the shared schema would give tables, workflows, and knowledge a parameter they ignore. GET /api/v2/files/folders is a FULL_SET_LIST, not paged, so no cursor binding changes — list-pagination.test.ts passes unchanged. Restore resolves the archived folder from its path by scanning the archived set rather than walking the live tree, which by definition does not contain the folder being restored. The folder-restored analytics hook now reports the folder actually restored rather than the requested selector, which carries no id on a path-addressed surface. * feat(v2): bulk-download a file selection as a zip Adds GET /api/v2/files/bulk-download, an adapter over the existing downloadWorkspaceFileItems use case and its internal binary route. Path collision: a static segment beside [fileId] permanently shadows a file whose id equals it, and workspaceFileIdSchema does accept [A-Za-z0-9_-]+. Rather than invent a new shape, this follows the existing bulk-delete sibling: the hyphenated form cannot be produced by either minted id shape (UUID v4 or wf_<shortId>), so the shadowed id is unreachable in practice. Documented on the contract so the reasoning is not lost. Folders are addressed by path, matching the rest of the v2 file surface. The paths resolve against the folder set the selection already loads, so it costs no extra query, and a path matching no folder is rejected rather than silently dropped — a misspelled folder must not yield a zip of whatever else was selected. The empty-selection and folder-count guards now account for folderPaths, which a path-only selection would otherwise have tripped. Selections are comma-separated only: v2 rejects a query parameter sent more than once, so a repeated-parameter form would never reach the schema. Pinned by a test so the contract cannot advertise a form the boundary rejects. * feat(v2): expose run output files and optional inline bytes on the runs read GET /api/v2/workflows/{id}/runs/{runId} now reports the files a run produced, each with the downloadPath that fetches its bytes, and can inline them as base64 on request. Gated by includeOutput, matching `output`'s nullability: a caller that did not ask for output does not receive a file list it did not request. The async execute request's rejection of includeFileBase64 is deliberately left alone — at submit time the run has not happened, so there is nothing to inline; reading a finished run is the first moment the question means anything. Inlining is capped per file at the executor's 16 MiB inline ceiling, which a caller may lower but never raise. A file above it answers 413 naming that file's downloadPath, so the caller is told exactly how to get the bytes rather than being left stuck. The descriptor deliberately omits the storage key — files are addressed by id and the key is re-derived from the run's recording — and omits an expiry, which the recording does not carry and which would be fabricated if published. The route becomes headSafe: false, since inlining reads object storage. The builder enforces that this requires the use case to expose authorize(), so HEAD still answers from a real authorization rather than from authentication alone. * feat(v2): permanently delete an archived file DELETE /api/v2/files/{fileId} only archives — the OpenAPI says its stored bytes are never removed — so there was no way to actually destroy a file over the API. Adds the repository primitive, application use case, operation, and DELETE /api/v2/files/{fileId}/permanent. A distinct path rather than a flag on the ordinary delete: a query parameter that turns a recoverable archive into an irreversible destruction is set by accident, and the two acts carry different minimum roles, which one route declaration cannot express. The file must already be archived; a live file answers 409 naming the archive step, so no single request can turn a live file into lost bytes. minimumRole 'admin', which forces workspaceApiKey 'deny' since the workspace-key ceiling is 'write' — the desired policy anyway: unattended credentials should not destroy bytes. Row first, then object. The two legs commit independently, so one can survive a crash between them: deleting the row first leaves at most an orphaned object for the storage sweep, while the reverse would leave a live row pointing at bytes that no longer exist — a file that lists and opens but can never be read. A failed object delete is therefore reported as objectDeleted: false rather than thrown, because the request has genuinely succeeded once the row is gone. Both directions are pinned by failure-injection tests, verified to fail when the order is reversed. Audited as a distinct FILE_PERMANENTLY_DELETED action, not a reuse of FILE_DELETED, which records the recoverable archive step. * feat(api): v2 log analytics, itemized cost, filters, and sortable query Adds the aggregate and rich-read halves of the public logs surface, and fixes three defects the existing reads carry. Aggregate analytics. `GET /api/v2/logs/stats` returns time-bucketed run counts, success rate, error count, mean latency, and the window bounds, per workflow and for the workspace. The first-party route was a raw handler with inline SQL and inline aggregation, so it is split into a repository (`lib/logs/stats-queries.ts`), a pure aggregator (`lib/logs/stats.ts`), and an application use case. That route keeps its legacy authorization — it answers a caller without workspace access with a zeroed 200, where v2 conceals the workspace as a 404 — and consumes only the two surface-neutral halves. `segmentCount` had no `.int()`, `.min()`, or `.max()`, so `0` divided by zero and `1e9` allocated two billion-element arrays: both caller-reachable 500s. Bounded on both contracts. `workflows` is capped, with the workspace totals still computed from every workflow and the cut reported as `workflowsTruncated`. Detail reads gain the itemized `cost.items` ledger (`null` and `[]` are distinct answers and both reachable) and `workflowInput`, restoring a v1→v2 regression. The list gains `workflowName` and `status` filters, and `includeJobRuns`, which unions Chat and Sim-agent job runs into the sequence behind a new `kind` discriminator — without it a job run is indistinguishable from a run whose workflow was deleted. A filter no job row can answer drops the branch outright rather than meaning two things across the union. `POST /api/v2/logs/query` carries the additional sort columns. `GET /logs` is untouched: its single `order` param rests on there being exactly one sortable column, and both escapes from that are ruled out, so the rich read gets its own endpoint — the split the table surface already ships. It uses the shared keyset scheme with the two nullable sort columns read through a sentinel, since a keyset cannot compare against null. `folderPaths` now covers a folder's whole subtree on the public path, as it already did everywhere else; it previously omitted every nested run with no error. The path strings did not change, so a folder-scope version is stamped into the cursor and in-flight tokens restart rather than silently skipping rows. Also fixes `folderName`, which ILIKEd `workflow.name` — a copy of the clause above it — and so searched workflow names instead of folders. `buildLogSortCursorCondition`'s `IS NULL` disjunct is documented and pinned: under `NULLS LAST` the null block is only reachable through it, so removing it as a duplicate-row fix makes those runs unpageable. Ratchets: route count 1142 -> 1144; logs OpenAPI operations 2 -> 4; total operations 152 -> 154. * feat(api): v2 tables run state, dispatch polling, batch update, bulk, archive Closes the headless gaps on the v2 tables surface. - Per-cell run state is now readable through an opt-in `includeRunState` on `GET /rows`, `POST /query`, and `GET /rows/{rowId}`. The default projection is byte-identical; a page whose sidecar outgrows its byte budget is a 413 rather than a silent truncation. - Run dispatches are addressable: `GET /tables/dispatches/{dispatchId}` publishes the column's full four-state domain so polling a finished run is not a 500, and `GET /tables/{tableId}/dispatches` lists what is in flight. - `POST /rows/batch-update` takes one distinct patch per row. Its transaction moved out of the Copilot-only module into a surface-neutral use case both surfaces now call. - `GET .../enrichment/{groupId}` publishes the provider cascade, cost, and timing behind one enrichment cell. - `POST /tables/bulk-move` and `/bulk-delete` reach the existing bulk use cases, which now accept folders by canonical path and resolve them inside the application layer. - `DELETE` is recoverable: `scope=archived` on the table list plus `POST /tables/{tableId}/restore`. * feat(api): expose knowledge chunks, tag writes, archive/restore on v2 Closes the knowledge cluster's remaining public-surface gaps. Chunks: list/read/create/update/delete/bulk under `/api/v2/knowledge/{id}/documents/{documentId}/chunks`. `queryChunks` gains an `id` tiebreaker on every sort so the list pages on a keyset rather than an offset — `tokenCount` and `enabled` are both non-unique, so a page boundary inside a run of equal values used to repeat or drop the tied rows. The internal offset caller is unchanged; the two positioning schemes share one read. Tag definitions: create, update, delete, next-slot, usage, and the document-scoped save and cleanup. Without them a caller could write a tag value into a slot with no definition and then had no way to name it, so tag-filtered retrieval was unbuildable end-to-end. `v2KnowledgeTagSchema` gains `id`, without which PATCH and DELETE are unaddressable. The document-scoped DELETE is pinned to `action: 'cleanup'`: the domain's `'all'` deletes the whole knowledge base's tag vocabulary from a document path. Archive/restore: `GET /api/v2/knowledge/archived` as a sibling route rather than a `scope` param — the two reads bind different operations and a v2 route declares one — plus `POST /api/v2/knowledge/{id}/restore`. `knowledge.restore` is a new workspace operation carrying `delete`'s policy, since an operation's inverse must not be harder to reach; the internal session route now delegates its workspace branch to the shared use case and keeps only the legacy personal one. Also: `POST .../documents/from-workspace-files` surfaces `addWorkspaceFiles`, so a file already in workspace storage no longer has to be re-uploaded byte-for-byte to be indexed; the `chunkingConfig` write widens to the first-party five-key schema with its refines and separator bounds, while the response stays `.catchall` so a legacy JSONB row cannot 500; and `CONNECTOR_MANAGED_RESOURCE_READ_ONLY` joins `FORBIDDEN_DETAIL_CODES` now that the bare 403 on connector-managed chunk writes is wire-reachable. Document upsert is deliberately not included. * feat(api): add v2 credential rotation and a gate-exempt capabilities endpoint PATCH /api/v2/credentials/{credentialId} rotates service-account secret material or renames a credential in place, preserving the credential id so existing workflow, deployment, paused-run, connector, and webhook references keep working. Re-posting to POST /api/v2/credentials answers 409, and delete-and-recreate mints a new id, so rotation previously had no door. The route is adapter-only: updateWorkspaceCredentialUseCase already owned the rotation, its audit projection, and credentials.update. It gains one additive assertedWorkspaceId field for the v2 workspace assertion, and the per-principal credential-type table that deleteCredentialUseCase already applied is lifted into requireManageableCredentialType so both operations share it. Without it a personal API key could rename an env_workspace row and toV2Credential's throw would surface as a caller-reachable 500. CredentialProviderOperationError now maps to 503 with Retry-After when the provider is unreachable, instead of the 400 its OrchestrationError('validation') base projected. A transient outage rendered as a permanent input error invites a caller to revoke a working credential. GET /api/v2/meta reports the calling key's rollout cohort, type, and expiry. It is the one route declaring the new typed gate: 'exempt' option, because the rollout gate and the unknown-path catch-all answer byte-identical 404s and a gated /api/v2/meta could never resolve that ambiguity. Authentication still runs first, so the only fact disclosed is one about the caller's own credential. * feat(api): publish deployment lifecycle and workflow-MCP v2 surfaces Adds the four deployment-lifecycle operations v2 was missing, and the workflow-as-MCP publishing surface, both as adapters over application use cases that already existed. Deployment lifecycle: - PATCH /api/v2/workflows/{id}/versions/{version} relabels a version. Deliberately not the internal route's body-shape dispatch between "rename" and "promote to live". - POST .../versions/{version}/activate promotes a version. Same use case as rollback under a different transition, on its own path because the two mean opposite things to a caller. - POST .../versions/{version}/revert overwrites the draft. Accepts the literal `active` alongside a version number. - PATCH /api/v2/workflows/{id}/deployment toggles unauthenticated public execution. `workflows.public_api.update` widens from session-only to session plus personal API key: it is an admin-role change the same accountable human may make from a script. Workspace keys stay denied. Its EE refusal now carries PUBLIC_SHARING_NOT_ALLOWED instead of a bare forbidden. Workflow MCP servers: - /api/v2/workflow-mcp-servers list, create, update, delete, plus publish and unpublish of a workflow as a tool. Named apart from /api/v2/mcp-servers, which registers the external servers Sim calls. - The six mcp_servers.workflow_deployments operations widen from ['delegated'] to admit sessions and personal API keys; roles and the workspace-key denial are unchanged. - The server list gains keyset pagination, matching its external sibling, since nothing caps how many a workspace publishes. - Server, tool, and workflow reads move out of the use case into lib/mcp/queries. Route ratchet 1150 -> 1160; OpenAPI operations 161 -> 171. * feat(api): extract chat deployments and publish the v2 surface Chat deployment was a shipped module with no public API and two authorization systems: `lib/workflows/application/chat-deployments.ts` had deploy/undeploy extracted, but only Copilot used them — the REST routes reimplemented workflow authorization inline, and `PATCH /api/chat/manage/[id]` additionally owned password encryption, the auth-type field-clearing matrix, identifier uniqueness, the redeploy-gating protocol with two 409s, a raw db.update, and a manual recordAudit. New `lib/chat-deployments` domain: - `chat_deployments.list/read/update/delete`, keyed on the deployment whose workspace is derived by joining its workflow. Creation stays `workflows.chat.deploy`, which is keyed on the workflow. - The PATCH extraction, including the field-clearing matrix and the asynchronous-cutover invariant the route had hand-mirrored from `performChatDeploy`. - One `buildChatDeploymentUrl`, replacing three constructions that had already drifted onto two different host helpers. There is no chat subdomain, so nothing publishes a host. - Repository reads moved out of the use cases into `lib/chat-deployments/queries`. Internal routes are now adapters over those use cases. `GET /api/chat` is deliberately not migrated: it scopes by `chat.userId` while every other chat operation authorizes by workspace admin, and reconciling the two is a product decision. `PATCH` keeps its 400 for an identifier collision through a typed `ChatIdentifierInUseError`; v2 reports the 409 the condition actually is. v2 surface at `/api/v2/chat-deployments`: list, create, read, update, delete. Workspace-scoped, keyset-paged, and a stored password is never readable — reads carry `hasPassword` only, and the session-only reveal endpoint deliberately has no v2 counterpart. Also: an email- or SSO-gated chat with an empty allow-list is now refused in the use case rather than only at the internal boundary, since it is unenterable; and the doc comment on `processHostedKeyCost` claiming a `usageLog` write is corrected — no such write exists. Route ratchet 1160 -> 1165; OpenAPI operations 171 -> 176. * fix(api): close three review findings, two of them caller-reachable - Run output files are filtered to keys under the run's own execution prefix. The recording they came from is not a trustworthy key source: the start block copies every caller-supplied input field verbatim into its output and `collectUserFilesById` accepts anything carrying the `UserFile` shape, so a caller could name any storage key and have the download and base64 paths — neither of which authorizes per file — serve it back. - `getBlock` reads own keys only. `BLOCK_REGISTRY` is an object literal, so `constructor`, `toString` and friends returned inherited functions that every consumer then treated as a block, turning a path segment into a 500. `getToolMetadata` already guarded this way. - A folder-scoped log page no longer unions in every job run in the workspace. The guard read `filters.folderIds`, which the public surface never sets — it carries the folder filter in `folderScope` — so the page contradicted the contract's promise that job runs are dropped whenever a filter they cannot answer is set. Also: the log cursor stamps `includeJobRuns` only when it is on, so its `.default(false)` no longer puts a constant in every fingerprint and rejects cursors minted before it existed; and the `folderName` subquery is scoped to the workspace and to workflow folders instead of scanning the whole `folder` table. * fix(api): close two more review findings, one an authorization bypass - `workflows.operations.apply` no longer admits a workspace API key. The use case authorizes against three per-user policies — the EE permission config, block visibility, and credential reachability — and all three take a human subject. An actorless key has none, and both substitutes fail open: attributing to the workspace billing owner evaluates the batch as the least-restricted account in the workspace, and passing no user makes `getUserPermissionConfig` return `null`, which every caller reads as unrestricted. Either way a workspace constrained by an allowlist was edited as though it were not. Personal keys keep the capability, so headless editing is unaffected for a credential that names a human. - `GET /workflows/{id}/state` reads its variables through `parseWorkflowVariables`, and the stored variable response schema drops the two assertions the column cannot honour. The column has carried a JSON string and a legacy array as well as the current record, the realtime `variable.add` op types `type` as `z.any()`, and the parser writes `name` through verbatim — so the input bounds on the read turned a stored workflow into a 500 on the endpoint that opens it. The write schema keeps them, which is where they can still be honoured. - `GET /workflows?scope=archived` projects folder paths tolerantly. Archiving a folder cascades onto the workflows inside it but leaves their `folderId` dangling — which is why restore has to null it — so the strict projector threw a bare `Error` and took the whole page down with no cursor able to step past the row. * fix(files): bind Start-block file keys to the executing workspace The Start block derived a file's storage key by parsing the caller's own `url`, which `isInternalFileUrl` matches on any host and `extractStorageKey` returns verbatim — so a request body could name any tenant's bytes. The key is now accepted only when its own layout names the workspace the execution runs in, and every file is dropped when the execution carries no workspace. Also bounds `includeFileBase64` with an aggregate response ceiling and a worker pool instead of an unbounded `Promise.all`, scopes the bulk download's authorization resource to the workspace when folder paths are requested, makes the folder-restore selector mutually exclusive at the type level, and names the bound in the `maxBytes` validation message. * fix(api): close v2 log review findings Cursor scope: `scope` on the workflow and table lists carries `.default('active')`, so it entered every fingerprint as a constant and refused every cursor minted before the param existed — with the "cursor does not match the requested filters" 400, which is actively misleading for a caller that changed nothing. Both now stamp the default as absent, so only a caller who asked for `archived` gets a new sequence. Dashboard stats: `maxWorkflows` capped the response, not the allocation. Segment series are now densified after the cut instead of before, so returning 200 series no longer materializes one `segmentCount`-length array per workflow in the window. The aggregate still sums every workflow, now from the sparse per-workflow maps. Cost keyset: `cost_total` is an unconstrained `numeric`, so its anchor travelled through `Number()` and was compared back at full precision — rows differing beyond float64 collapsed onto one anchor. Adds `decimalKey`, which carries the digit string and binds it `::numeric`. Run detail: `cost_total` is a backfilled projection, so a run predating the backfill reported `cost: null` even with a real ledger, making `items` unreachable for exactly the runs the ledger explains. Falls back to the ledger total. Also caps the log folder-path index reads at MAX_FOLDERS_PER_WORKSPACE like every other reader, publishing the folder-tree 413 on the four log operations; reverts a dead `status` widening in `v2CommaListSchema`; drops an unread `executionData` select; corrects the segment-count and searchLogs prose; and replaces the sort-cursor SQL-text assertions with a two-page walk over a fixture with a null block. * fix(api): close knowledge v2 review findings - widen knowledge.list_archived to the delete/restore policy so a workspace API key can discover what it may restore - escape LIKE wildcards on the now-public chunk search - derive tag slot capacity from TAG_SLOT_CONFIG per field type - type updateKnowledgeBase's chunkingConfig as ChunkingConfig and project every declared field explicitly - attribute a restore to the calling surface instead of a literal 'api' - gate 'knowledge chunks batch-update' behind --yes, since it can delete - present the tag-cleanup action from the parsed request rather than faulting on the domain result after the delete committed - unbind asserted-scope workspaceId from the nested knowledge cursors, matching the table-row lists - add executed-SQL coverage for the chunk keyset * fix(catalog): close the catalog and registry-boundary review findings The module-graph ratchet treated an entry with no baseline row as informational, so the six catalog routes and the projection barrel were unratcheted while the summary still read "within their module-count baseline". An unbaselined entry now fails --check, the summary counts only what was actually compared, and the baseline is re-recorded. The Copilot block-metadata tool — the reason the shared projection exists, 6,756 modules down to 1,321 — was in no guarded subtree. It is now an entry source and a catalog boundary root. Catalog behaviour: - hostedApiKey is gated on the deployment, so a self-hosted install reports none instead of promising 127 tools' keys it will never supply - block detail resolves an unversioned base type to its newest version and projects through the viewer's visibility, so it can no longer 404 a block the list contains or name it differently - offset-cursor ordering compares code units rather than the process locale - projections copy every array they publish instead of handing out the registries' own - an options function returning a thenable throws rather than silently widening the providers-store substitution across the event loop - a throwing block projection costs the Copilot tool one block, not all of them - the trigger-kind log returns to debug: chat/manual/api are entry-point kinds, not authoring defects Also sweeps the custom-block detail branch against its response schema, guards each projection module rather than the dead barrel over them, and drops a provably dead branch in processHostedKeyCost. * fix(workflows): close v2 workflow-authoring review findings - Read a blockless draft back as an empty graph. `PUT /state` of `{ blocks: {}, edges: [] }` — the contract's own published example — deletes every block row, and the loader answers `null` for a blockless workflow, so the following `GET /state` answered 404 while the list endpoint still showed the workflow. Existence is the workflow row's to decide; the null is now projected as an empty graph. - Rewrite the `readWorkflowGraph` authorization test so it can fail. It called `authorize?.()` and asserted only a negative, so deleting `authorize` or replacing it with a no-op both passed — the invariant the head-safe `HEAD` path depends on. - Route `setWorkflowBlockEnabled` through `replaceWorkflowNormalizedState`, the same door the other two graph writes use, instead of writing the normalized tables itself without state preparation or custom-tool extraction. - Count applied operations directly. Enablement refusals landed in the same skipped-item array and were subtracted from the operation count, which `Math.max(applied, 0)` then masked when it went negative. - Give a `disabled_ancestor` refusal its own member of the published skip enum instead of reporting it as `block_locked`. - Refuse an `atomic` batch whose credential or hosted API key would be stripped, and carry the dropped inputs in the 409 details. - Publish the whole lint report — `sources`, `sinks`, `orphanBlocks`, `emptyOutgoingPorts`, `invalidBranchPorts`, `invalidConnectionTargets`, `fieldIssues`, and the `kind` discriminator on unresolved references — rather than only free-text reference prose. - Stop reporting unresolved lint references as `inputValidationErrors`. `collectUnresolvedReferences` is read-only, so those values stay persisted; they were double-reported, and falsely as dropped inputs. - Replace two unfalsifiable negative-principal tests, which used a principal kind `Exclude`d from `PrincipalKind`, with a reachable one. - Nits: drop a stranded TSDoc block; assert the membership predicate in the selector-validator admin test; make the HEAD test assert a representation; assert the sanitized graph is what `replaceWorkflowState` writes; add a route test rejecting `baseGraph` in a v2 body; carry `principalAuditSource` on restore/duplicate/moveBulk audit; unify the two `base64MaxBytes` ceilings on `MAX_INLINE_MATERIALIZATION_BYTES`. * fix(api): close seven v2 review findings, one an authorization bypass Raise mcp_servers.workflow_deployments.update_server to admin: its body carries isPublic, and a public server executes with no Sim credential, so a write member could remove authentication from every workflow it publishes. create_server already grants the same visibility at admin. Pin the widened operations in registry tests — the six workflow-MCP ones, the four workflow widenings, and files.extract_archive. Reject secret fields on a credential that has no rotatable secret instead of dropping them behind a 200, classify a non-transient provider 4xx as caller error rather than a retryable outage, and reconcile a provider outage to 503 with Retry-After on all three surfaces. Give /v2/meta a declarative principal policy through a new defineOperation factory, carry the key expiry on the auth context instead of reading the api_key table from the application layer, and make the impossible principal branch an invariant error rather than a codeless 403. Enforce the rollout-gate exemption at definition time, against the one contract path it is reserved for, and remove the gate parameter from the exported admission helper so the builder is its only door. * fix(tables): bound the run-state sidecar, and close six v2 review findings Enforces the 2 MiB run-state ceiling INSIDE the sidecar drain rather than over its materialized result, refuses the unbounded query form paired with it, and normalizes the two stored blobs the v2 surface publishes from bare `as` casts. - The run-state budget now travels into `loadExecutionsByRow`, which drains row ids in bounded chunks and refuses before fetching the next one. The post-hoc `requireBoundedRunState` walk is gone: it measured a spike that had already happened, and re-serialized every entry to do it. - Both row reads that accept `includeRunState` cap the page at `V2_MAX_RUN_STATE_ROW_LIMIT`, and `POST /tables/{id}/query` additionally refuses the flag paired with `limit: 0`. - `runState.status` and the enrichment cascade blob are projected onto the published shape before presentation; both were caller-reachable 500s on a well-formed read. - `sim tables bulk-delete` now gates behind `--yes`, and the CLI sweep that should have caught it covers destructive non-DELETE forms. - `POST /tables/{id}/restore` is idempotent (200, no audit) like its knowledge sibling, and bulk folder selection deduplicates after resolution. - The batch-update backstop keeps the looser Copilot ceiling and says so in TSDoc: it is a backstop no surface reaches, because the contracts stop a v2 caller at 1000 and the Copilot tool stops itself at 5000. Each caller sees the bound that actually applies to it; neither surface's cap moved. * fix(chat-deployments): close v2 chat review findings Fixes the chat-deployments slice of the v2 review, several of which are regressions the application-operation extraction introduced. - Stop a `500` on schemaless JSONB: the response now declares a stored shape without bounds and `toV2ChatDeployment` projects `customizations`, `outputConfigs`, and `allowedEmails` onto it. The request schemas keep `.strict()` and their bounds. - Restore the specific validation message on `POST /api/chat` and `PATCH /api/chat/manage/[id]`, and the deleted test that pinned it. - Restore `chat_deployments.read` to workspace `admin`; the detail read serves the visitor gate. - Narrow the list projection so `chat_deployments.list` can stay a `read` operation reachable by a workspace API key: `allowedEmails`, `hasPassword`, and `customizations` are gone from the list entry and available only from the admin-gated detail read. Serialized field by field so a field added to the detail shape cannot reach the list by default. - Classify create-path failures: `performChatDeploy` carries an `errorCode`, so an in-flight deployment is a `409` and an invariant failure a `500` instead of every refusal being a `400`. - Delete the callerless `GET /api/chat`, which served the encrypted password column with no response contract. - Propagate undeploy infrastructure failures instead of concealing them as `404`, and return `ChatDeploymentView` from both delete paths. - Name `CHAT_AUTH_MODE_NOT_PERMITTED` on the create path. - Guard `getBaseUrl` inside `buildChatDeploymentUrl`, which otherwise throws on a self-host with no `NEXT_PUBLIC_APP_URL`. - Correct the published allow-list claim: a replacement `allowedEmails` is applied after the auth-type clear, so it does survive. - Assert `workspaceId` on the v2 detail routes and reconcile the concealment TSDoc with what the error policy actually renders. - Move `resolveActiveWorkspaceApplicationContext` to the workspaces domain so chat-deployments no longer imports workflow application code. * test(credentials): pin the reconciled provider-outage status The internal route alone answered 502 where the v2 surface, the shared status helper and `PROVIDER_OUTAGE_CODES`' own TSDoc all say 503. The test pinned the divergence; it now pins the reconciliation, including the `Retry-After` a 503 carries. Corrects a stale comment that still named 502 as the value callers see. * fix(executor): restore cloud-storage Start files, dropped by the key rule The ownership check accepted a key only when it could be parsed out of an internal `/api/files/serve/...` URL. But the server-side uploader for run inputs returns a *presigned cloud* URL whenever object storage is configured, whose path is the bucket key — so every chat-deployment attachment, API `files[]` payload and generic-webhook file field resolved no key, and because normalization is all-or-nothing the entire `files` input was dropped with no error. It passed locally and under vitest only because the uploader falls back to an internal URL when no object storage is configured, which is exactly why no test caught it. The test is ownership, not provenance: a key is accepted when its own layout names the executing workspace, whether it arrives directly or is parsed out of the URL. Neither field has to be trusted, since both are caller-authored and both are held to the same check. A payload whose key and URL disagree is refused rather than resolved in the caller's favour — a genuine uploader writes the two consistently, so only a forged pairing is turned away. `context` is now derived from the accepted key rather than read from the payload or the URL's `?context=`, so an owned key can no longer be labelled with a bucket its bytes do not live in — the hardening the previous comment claimed but did not perform. * fix(api): close four defects the fix pass introduced - `resolveLatest` built a `RegExp` from the caller's block id and read the registry with a bare lookup, so the catalog detail route — moved onto it by the version-alias fix — routed around the `ownBlock` guard added for exactly this. `GET /api/v2/blocks/%5B` was a `SyntaxError` 500 and `.../constructor` an inherited function. Matched by string comparison now, the way `tools/tool-ids.ts` resolves the same convention, and read through `ownBlock`. - The run-state byte budget was applied inside `queryRows` rather than at the callers that publish it, so the first-party table grid — which reads run state at five times the row limit and publishes no ceiling — turned a large page into a hard failure, with an error naming a parameter it does not expose. The budget is now an explicit option the public reads pass and internal callers omit. - Three graph-write CLI commands shipped ungated because the sweep meant to catch them matched only the names already enumerated, so it could never fail. It now forces every non-`GET` operation into a destructive or non-destructive list, and the three carry confirmations. - Unbinding `workspaceId` from the knowledge-documents cursor was right on the merits and wrong in effect: the value is constant per sequence, so removing it changed the fingerprint and refused every cursor already in flight. Restored there; the chunks list is new in the same change and keeps the cleaner reading. * fix(api): resolve a detail read to a version the viewer can see `getLatestBlockForViewer` took the newest version and then hid it, which inverted the contradiction it was written to close: `slack_v2` and `table_v2` are preview-gated while their v1 deliberately stays in the toolbar, so an unrevealed viewer got a `404` on a detail read for a type `GET /api/v2/blocks` was listing in the same breath. It now walks versions newest-first and answers with the first one visible to that viewer. Also: - The chat password guard ran after `performFullDeploy`, so a request that could never succeed burned a real workflow deployment version and then answered 400. Its two sibling gate guards already refuse ahead of the deploy; this one now does too. - The Copilot sub-block serializer published the registry's own `options` and `dependsOn` arrays by reference. Pre-existing, but the catalog projection this parallels copies every array it publishes precisely because they are process-global and shared by every request. * fix(api): restore the locked read-modify-write and the password validator Two findings verified as real regressions against staging, out of ten checked — the rest were pre-existing, latent, or false. `setWorkflowBlockEnabled` read the graph outside the row lock and wrote it back inside a later transaction. The editor's own save takes that same lock, so an autosave committing in the window was silently discarded: this operation writes a whole graph, not a delta. The persistence primitive now accepts a reader that runs after the lock is taken, and the toggle re-reads and re-decides there. Its lock predicate is also scoped to the workspace and to a live row again, so a workflow archived mid-flight is refused rather than written. The v2 chat-deployment contracts inlined their own password rule twice instead of using `chatDeploymentPasswordSchema`, losing the refusal of a whitespace-only password — which the internal contract rejects precisely because it strands the deployment behind a password the visitor form will not submit. Both sites use the canonical validator now. * fix(api): one folder projection, one dynamic-provider list, honest 413s - `toV2Folder` existed twice, and the second copy had been written without the name/path invariant — so a row the list read refuses loudly would have been served with a mismatched pair by the restore read. One definition, guard included. - The catalog projection restated `DYNAMIC_MODEL_PROVIDERS` and had drifted by one member. Derived from the canonical list instead. - The tables reads documented a `413` for run state that they cannot emit — the budget became opt-in, and the row limit is the bound now — so the claim is removed rather than declared. The workflow run read has the opposite problem: it genuinely emits one, on a single file *or* the run's inlined total, and declared neither. Now declared, and the sentence covers both. - Reclassifies the operations staging added into the destructive sweep, so the triage stays exhaustive. * fix(v2): classify storage and uniqueness failures, drop permanent file delete - remove the permanent file-delete endpoint; the platform offers no such action in the UI, and its manager wrote outside a transaction with no storage accounting - extract a generated document's text from its compiled artifact rather than its generation source, matching the download path; a `.pdf` source was a 500 and a `.docx` source returned generator JavaScript as clean content - report a run file whose object retention has already swept as 404 rather than 500, on both the inline base64 read and the download stream - report a knowledge tag that loses at a unique index as 409, naming whether the slot or the display name is taken - gate `workflows versions revert` behind a CLI confirm; it overwrites the draft graph and was classified non-destructive * feat(v2): report lint from both graph writes and add dry-run previews - `PUT /workflows/{id}/state` now returns the same `lint` report as `POST /operations`; an agent authoring a graph from scratch needs the findings at least as much as one editing incrementally - extract the report into one shared builder so the two writes cannot drift, and one shared presenter so the wire shape is identical - skip the credential/tool reference pass when the caller has no human subject, rather than resolving it against the workspace billing owner: that would misreport what the workflow can reach and disclose another person's grants. `lint.notes` says when it was skipped - add `?dryRun=true` to both graph writes: validates and lints, persists nothing, records no audit, notifies nobody. A query param, not a body field, since the body of a PUT is the resource itself - CLI: a dry run no longer demands `--yes`; requiring confirmation to preview a change teaches callers to pass `--yes` reflexively - CLI: name the graph commands for their verbs — `workflows state get`, `workflows state replace`, `workflows operations apply` — instead of the derived `state list` / `state update` / `operations create` - document when to use `rollback` vs `versions/{version}/activate` on both * chore(docs): sync generated docs manifest for the new CLI pages * feat(v2): add the missing workflow-MCP reads and align bulk naming - add `GET /workflow-mcp-servers/{serverId}` and `GET /workflow-mcp-servers/{serverId}/tools`. The resource could be PATCHed and DELETEd but never read, and its tools could be published and unpublished but never listed — the server list reports tool names only, so nothing published the `workflowId` that addresses a tool for deletion. Both mirror `mcp-servers` beside them, and carry that family's workspace-API-key denial rather than the wider `mcp_servers.read` policy - rename `POST /tables/bulk-move` to `POST /tables/move`, so tables matches the shipped `files` resource exactly (`move` + `bulk-delete`) - name the CLI commands for their operations instead of the derived `... create`: `tables move`, `workflows move`, `tables bulk-delete`, and `tables rows update-each` for the per-row batch, which sits beside the existing filter-based `tables rows batch-update` `POST /tables/{id}/rows/batch-update` keeps its name: a distinct payload per resource is precisely AIP-234 BatchUpdate, and `bulk-` would have collided one word away from the filter form. * fix(v2): correct documented statuses and a caller-reachable 500 - duplicating a workflow into a locked destination folder answered 500: `FolderLockedError` is a plain Error carrying `status = 423`, which the v2 error policy does not classify. Converted to OrchestrationError('locked') at the application boundary, matching the bulk-move path - restore workflow promised a 413 for an oversized folder tree that its response list never published; the cap is real, so the status now is too - move workflows and apply variables documented 409/423 they cannot emit: every per-item lock and conflict is reported in `failed`, not thrown - bulk download and delete knowledge tag can both 409 and did not say so; cleanup tag definitions cannot and did say so - apply workflow operations denies workspace API keys but never documented it - the dry-run responses are not byte-identical to a committed write: `needsRedeployment` describes the pre-write state and persistence warnings cannot appear. Reworded rather than overclaimed - read file text and get file upload are head-safe, so the "HEAD skips the effect" sentence did not apply to them * fix(v2): guard tag field-type changes and publish the 415 every body route can return - `PATCH /knowledge/{id}/tags/{tagId}` accepted a `fieldType` incompatible with the slot the tag already occupies. Slots are enumerated per field type, so a text tag could be relabelled `number` and every later read would interpret its values as the wrong type. Create checked this; update now runs the same two checks - derive `415` from the contract the way `413` already is: the JSON builder answers UNSUPPORTED_MEDIA_TYPE for any body under a content type it cannot read, so all 100 body routes could return a status none of them published - give version activation its own result component instead of publishing it as `RollbackResult`; the shipped rollback keeps that name - correct descriptions that promised behaviour the code does not have: a `processingStatus` field never returned, a `gmail_send` resolution example that short-circuits, bucket widths that overflow the window, a bulk tag save that relocates rather than overwrites, and per-server tool names actually gathered under a page-wide budget - drop 409 from three knowledge and upload reads that cannot emit it * docs(v2): correct the upload transfer contract and 16 other published claims The upload transfer step was documented as Sim's own data plane on every deployment: "success is 204" and "a failure is the v2 error envelope". That holds only when Sim stores objects itself. With object storage configured the URL is the provider's presigned URL, so S3 and GCS answer 200 and Azure 201, and a failure is the provider's XML — a client written to the old text reads a successful cloud upload as a failure. Also states that part ETags do not need retaining: completion takes no body because Sim lists the parts from the provider itself. Other corrections, all to shipped descriptions rather than behaviour: - DELETE table and bulk-delete files archive rather than erase, and neither said so; bulk delete also cannot emit the 409 it declared - complete knowledge upload published a 402 only the create leg can raise - billing status conceals a foreign workspace id as 404, not the 403 its TSDoc and description both claimed - audit entries null a folder's resourceId and strip folder ids from metadata at every level; neither redaction was documented - details=full adds the workflow summary to workflow runs only, never to job runs; GET /logs folderPaths covers a subtree like its two siblings; getLog now carries the retention sentence - list secrets returns description too, and the logs and resources documents described only part of what they serve * improvement(api): consolidate the v2 surface and close seven defects Endpoint consolidation: - Fold POST /logs/query into GET /logs; add sortBy/sortOrder, cap the comma lists, and move the list onto the shared keyset codec - Fold GET /knowledge/archived into GET /knowledge?scope=archived, matching files, tables, and workflows - Re-home chat deployments as a singleton under the workflow they belong to; keep the workspace-scoped discovery list - Move the tag-definition writes off the document path onto /knowledge/{id}/tags, where they already acted - Nest the table export and dispatch reads under their parent table Defects: - Publish isPublicApi on the deployment read; it was write-only, so a workflow could be opened to unauthenticated execution unauditably - Stop publishing raw storage keys and an unusable URL in log files - Classify a chat-identifier unique violation as 409 rather than 500 - Fall back to the root path instead of throwing when a knowledge base's folder is archived - Cap bulk-download at the ceiling it actually enforces - Normalize variables through one helper on both graph write paths - Fix a folder-name log filter that matched workflow names Naming and gaps: - Rename /files/{id}/extract to /unarchive, /rows/find to /rows/search, /rows/batch-update to /rows/bulk-update, /columns/run to /dispatches - Type the last six generic [id] path segments - Add table folder restore and id-addressed dispatch cancel * chore(audits): record the v2 catalog routes in the boundary baseline * fix(api): accept a null chat password and correct three published claims - performChatDeploy validated `password: null` as a password, so the replace-shaped chat PUT answered 400 for every mode that owns no password — public (the default), email, and sso. The declared payload type has always allowed null, and the stored value is cleared by authType regardless, so null needs no validation of its own. The route test could not catch it: it mocks the orchestration module and pinned the exact null the real guard refused. - Redirect the two docs slugs this branch retired that were genuinely published: findTableRows and runTableColumns. - The table folder restore described an idempotent no-op for an already active folder; it answers 404. Say so, and say where the path comes from, since the tables folder list cannot yet report archived folders. - Name the customizations exception to the chat PUT's replace semantics. - A cursor-binding case used status=error, which is a level and not a status, so it failed contract validation and never reached the cursor check. Use an accepted value and pin the reason, not just the status. * chore(cli): classify the new v2 chat operation as non-destructive * feat(cli): expose canonical resource URLs * fix(api): close three caller-reachable failures found by the final probe - GET /files/{fileId}/text called parseBuffer unguarded, and parseBuffer signals every failure as a bare Error that no v2 policy classifies. A zero-byte upload or a mislabelled archive was an unhandled 500. Empty bytes now answer empty text — a zero-length file has no text — and unparseable bytes answer 409, matching the rendered-artifact resolver. - GET /workflows/{workflowId}/state asserted write-side bounds over stored data. workflow_blocks.name and .type are bare text() and the realtime rename op accepts z.string(), so a block renamed past 255 characters made the workflow unreadable, and unrepairable, over v2. The read shape now takes the same input/stored split the variable schema already had. Stored subflow conditions are coerced in the loader beside the existing numeric guards. - GET /knowledge stamped scope into the cursor fingerprint unconditionally. scope defaults to active and is new on that list, so every cursor the deployed build handed out would have been refused with a message saying the caller changed a filter they never sent. Its siblings already carry the guard and the comment. Also: POST /workflow-mcp-servers answers 201 like every other v2 create; GET /logs/stats reuses the log list's entry ceilings; the execute and resume routes install the media-type-aware 415 they publish; and a rationale citing an endpoint that never reached the wire is corrected. * fix(tables): carry the dispatch terminal timestamps through the stale sweep Staging's abandoned-dispatch recovery builds its own `DispatchRow`, and this branch had added `completedAt`/`cancelledAt` to that shape for the id-addressed dispatch read and cancel. The merge was textually clean and left the new mapping short two fields. * fix(workflows): deny workspace API keys on the graph replace `PUT /workflows/{workflowId}/state` stores blocks and their tool wiring wholesale, but the policies deciding which of those a member may add — the EE permission config and block visibility — take a human subject. A workspace API key has none, and both substitutes fail open: the billing owner is a different, typically less-constrained person, and passing no user makes the permission lookup return null, which every caller reads as unrestricted. That made the replace a second graph-write door storing what its sibling `POST /workflows/{workflowId}/operations` refuses, which denies workspace keys for exactly this reason. Both doors now agree. Personal keys keep the capability, so headless authoring is unaffected for a credential that names a human. * chore(api): drop the enrichment catalog endpoint GET /api/v2/enrichments listed the code-defined table enrichments. The per-row enrichment run detail stays; only the catalog read goes. Removes the route, contract, response and query schemas, the semantic operation, the use case, the projection module, its registry-boundary entry, and the CLI command. The "not found" and "blank search" cases it covered are repointed at the connector-type sibling so the shared behaviour stays tested rather than deleted with it. * improvement(api): make the workflow operations endpoint self-describing Two things stood between this endpoint and a caller who has only the published spec. The accepted `params` keys existed only in the edit engine's source. The spec said "the accepted keys depend on the target block type", so a caller reading it could create a nameless empty block and nothing more — while the Copilot tool catalog, over the same engine, has always spelled out the envelope. That guidance now lives in the contract, shared by the add, edit, and insert_into_subflow parameter schemas so the two surfaces cannot describe one engine differently: `inputs` keyed by sub-block id, `retry`/`triggerMode`/`advancedMode` beside it rather than inside it, `connections` keyed by source handle, and `removeEdges` for dropping one edge without restating the rest. A `block_id` that is not already a UUID is replaced with a minted one, and the mapping was computed and then dropped. A caller could not reference the block it had just created except by re-reading the graph and matching on name. The engine now returns it and the response publishes it as `mintedBlockIds`, with the in-batch versus cross-request rule stated in the operation description. * fix(api): close the pre-merge scan's blocking findings Docs and public wire, all of it permanent surface once released. - Two published tag groups, Catalog and Meta, had no sidebar entry in any locale, so six operations shipped unbrowsable. Added to all six, and a test now fails when a published tag has no entry. - deleteWorkflowChatDeployment pointed callers at DELETE on /deployment; the undeploy verb is on /deploy. - PUT /state still promised workspace API keys a degraded lint pass after the operation began rejecting them outright. It also lived in a single-quoted string, so the shared clause would not have interpolated. - POST /tables/move took targetFolderPath as nullable-but-required, which rendered the CLI flag as a required `<json|@file>`: `--to /Archive` failed to parse and omitting it failed outright, while the docs said "omit for root". Now optional and a plain string, matching POST /files/move; the route supplies the null the use case wants. - Table dispatch status and row run state published `cancelled` beside imports, exports, and job state publishing `canceled`, and the note explaining the split was wrong about its own sibling. Both new schemas now publish `canceled`; the stored column is unchanged and mapped at the presenter. The shipped `cancelled` count field is left alone. - applyWorkflowVariables can answer 423 and both workflow-MCP deletes can answer 409; none declared it. Hand-assembled error lists replaced with the shared sets. - /files/{fileId}/unarchive became /unzip. `extract` reads as "extract text" and `unarchive` reads as the inverse of restore on a resource where archived means soft-deleted; unzip collides with neither and is what the implementation calls itself. * improvement(cli,docs): finish the naming and clear the enrichment leftovers - Four single-record GETs derived to `list` while returning one thing: meta, a workflow's chat deployment, log stats, and file text. Renamed to `meta status`, `workflows chat status`, `logs stats`, `files read`, matching the `workflows deployment status` correction already in this branch. None of the old spellings shipped. - Dropped five `renamedFrom` aliases pointing at spellings that never existed, each of which built a hidden command and a permanent deprecation warning for argv nobody could have typed. `tables rows find` keeps its alias — that one really shipped. - Removed the enrichment catalog from three prose sites left behind when the endpoint went: the resources spec description, its Catalog tag, and the contract and pagination-test comments. - Documented the six new command groups in the CLI index table. That page is a guide page, so the docs staleness check cannot flag it. - Published the `customizations` exception to the chat replace semantics. It was in the route TSDoc and invisible to every caller reading the spec, which is where the claim "Replace, not merge" is made. * improvement(cli): use batch- for the tables bulk delete, matching its siblings The CLI renames a bulk form only when it would collide with its singular sibling, and uses AWS's `batch-` prefix when it does — `files batch-delete`, `tables rows batch-delete`, `knowledge chunks batch-update`. `tables delete` exists, so `tables bulk-delete` was that same rename reaching for the other word, and the only `bulk-` command on the surface. No `bulk-` CLI command has ever shipped, so this costs nothing now and would be permanent later. `files bulk-download` keeps its name: there is no `files download` to collide with, and it is one archive rather than N operations. Its config block now says why it exists at all, since the command is never built — the builder skips non-JSON response modes, but the contract sweeps still read the entry and require the folder-path field to be marked. * fix(workflows): allow operations on blockless drafts * feat(workflows): add manual and run-from-block execution * fix(cli): update workflow run description test * fix(tests): align fixtures with current contracts * fix(api): derive chat activity from workflow deployment --------- Co-authored-by: Theodore Li <theo@sim.ai>	2026-08-25 04:19:34 -04:00
Vikhyath Mondreti	0a5b3801ea	feat(secrets): let workspace secrets opt out of redaction (#7045) ... * feat(secrets): let workspace secrets opt out of redaction * fix(secrets): certify no sandbox exemptions once the registry is incomplete * feat(secrets): carry visible secret values on the v2 list and document visibility * fix(secrets): read visible values by own property so prototype-named secrets cannot poison the list	2026-08-24 13:46:57 -07:00
Waleed	edf07ec3cd	fix(vllm): support LM Studio endpoints (#7036) ... * fix(vllm): support LM Studio endpoints * fix(vllm): validate compatible base URLs * fix(vllm): guard discovery URL validation	2026-08-24 10:58:07 -07:00
Vikhyath MondretiandClaude Opus 5	81ff24a2fc	improvement(secrets): gate Copilot code mounting at use level (#7004) ... * improvement(secrets): gate Copilot code mounting at use level Mounting a saved secret into Copilot code required credential-admin on that key, while a workflow Function block resolves the same secret for the same person at use level through getPersonalAndWorkspaceEnv. Copilot reaches that path itself — edit_workflow plus run_workflow — so the admin bar contained nothing. It redirected a Credential Member through a detour that mutates a persisted workflow, while the direct path is ephemeral and files a usage row. The inconsistency was also internal to Copilot: the secret names advertised to the model come from getAccessibleEnvCredentials and getPersonalAndWorkspaceEnv, both role-agnostic, so Copilot listed every secret the caller could use and then refused to mount all but the admin ones. Widen the workspace and shared-personal predicates to any active grant, and drop the matching role filter from the query. Workspace write is still required, revoked and pending grants are still refused, and a caller with no grant still gets nothing. The view gate stays where Copilot cannot route around it: values remain masked under Settings, and See usage remains admin-only, so a member's use is recorded for whoever can rotate the key. Model-egress projection is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(secrets): stop implying Personal secrets are shareable The Copilot code-execution paragraph listed "any secret shared with you as a Credential Member or Credential Admin" among what mounts, which reads as though a Personal secret can be shared. It cannot through any product surface: CredentialMembersSection renders only for workspace secrets and OAuth credentials, and the personal-credential sync only ever grants the owner. Narrow the sentence to Workspace grants. The comparison table's "Only you can use" row for Personal was correct and is left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-22 18:33:04 -07:00
Theodore Li	8104c33bba	Fix knowledge connector sync follow-up (#6927) ... * Fix knowledge connector sync follow-up * Fix connector sync pause race * fix(knowledge): surface connector sync dispatch failures * fix(knowledge): make connector sync recovery durable * fix(knowledge): deduplicate connector sync dispatches * fix(knowledge): preserve pending connector syncs * fix(knowledge): lock connector sync snapshot	2026-08-22 20:50:48 -04:00
3 people	c26529a82e	feat(bitbucket): add repository webhook triggers (#6934) ... * feat(bitbucket): add repository webhook triggers * fix(bitbucket): harden webhook trigger delivery * fix(bitbucket): address final trigger review * chore(bitbucket): address review conventions * fix(bitbucket): harden triggers and connector sync --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-08-22 15:23:35 -07:00
Vikhyath MondretiandClaude Opus 5	71129cd112	feat(custom-blocks): let a publisher decide whether their block's runs reach consumer traces (#6950) ... * feat(custom-blocks): let a publisher decide whether their block's runs reach consumer traces Joining a custom block's child run into its caller's trace shipped on by default, gated at read time by whether the person reading could already open the source workspace. That gate is doing the wrong job: a custom block's whole point is that consumers need no access to the source, so the check refuses exactly the readers the feature exists for, and it makes the answer depend on who is looking rather than on what the block's owner agreed to publish. The decision moves to the party whose data it is. `custom_block.trace_child_runs` is set by the publisher in Settings, applies org-wide, and is the entire policy — nothing downstream re-checks a caller. `getCustomBlockAuthority` already resolves per invocation and is the one lookup both the canvas handler and the Agent-tool runner pass through, so one column covers both surfaces and no consumer input can assert it. It defaults to FALSE. With the viewer check gone, an opted-in block publishes the source workflow's block names, inputs, outputs, and prompts to anyone who can read a consuming workflow's log. That is the same boundary curated outputs and redacted errors hold, so it opens by an affirmative act of the publisher or not at all — never as the residue of a column default on rows nobody revisited. Closed means the handle is withheld outright rather than persisted behind a flag: with no `childExecutionId` there is nothing for a reader, a migration, or a later refactor to join. What replaces it is a `_childTraceDisabled` marker, because a boundary span with no children renders exactly like a leaf block and an untraced run would otherwise read as one that did nothing. The consumer-facing failure `ref` is untouched either way — it is the only thing that makes an untraced failure reportable. Custom blocks invoked as Agent tools now join too. The child's handle already reached the agent's persisted `toolCalls[].result` (`postProcessToolOutput` strips only `__`-prefixed keys); nothing lifted it onto the tool span. Both span builders lift and strip it, and `hydrateChildTraces` needs no change — its boundary walk already recurses. The same handle is stripped from the model-facing copy of the tool result in `executeProviderTool`, the single point where the raw and model copies diverge: an opaque execution id in a tool result reads to a model like data the tool returned. The live SSE stream keeps one condition beyond the policy: an identified consumer. Not an authorization check — no workspace query — but chat deployments and the public API leave `liveTraceViewerUserId` unset because their consumer may be anonymous, and opting into org-wide tracing is not consent to stream a publisher's raw agent tokens to the internet. Copilot deliberately cannot set the field; exposing a team's internals org-wide is a human decision, not one an agent makes while publishing on their behalf. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(custom-blocks): read the publisher's trace policy at read time, not from the handle's presence Treating a persisted `childExecutionId` as proof of publisher consent is only true for handles this PR's writer produced. Every handle written before it meant something else — "a child ran; authorize the reader" — and the rows carrying them outlive the migration, so removing the reader check turned them into an open door: a consumer could open an old parent log and receive the source workflow's block names, inputs, outputs, and prompts from a block whose publisher never opted in. `hydrateChildTraces` now resolves the policy live, per boundary, from `custom_block.trace_child_runs`. The child log row's `workflowId` is the key — publish enforces one block per workflow — which also covers an Agent-tool boundary, whose span carries no block type to look up. A workflow with no block row (never published, or since deleted) has no publisher left to consent and stays shut, as does a failed policy read. This is not redundant with the write-time withholding. The handler still emits no handle for a block that was closed when the run executed, so such a run stays closed forever even if the block is opened later; this check decides whether the runs that DO carry a handle may still be shown. Turning the policy off therefore also closes what is already recorded, which is what a governance switch has to do to mean anything. Reported by Greptile on #6950. Also drops `any` from the trace-policy tests: outputs read through `Record<string, unknown>` (the handler's declared return does not name these internal keys) and failures narrow through `ChildWorkflowError.isChildWorkflowError`, which pins the failure type as well as its fields. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(logs): sum the child-trace drop counters from the struct, not a hand-listed set `totalDropped` re-listed four of the five counters, so a read whose only drops were policy refusals computed zero and skipped the log entirely. That is the commonest drop there is now — every handle written before the publisher policy existed refuses at that gate — so the one signal telling an operator the live check is closing joins went silent exactly when it started mattering. Summed from the struct instead. A hand-maintained list beside a struct is stale the moment a field is added, which is precisely how `policyClosed` was left out. Reported by Cursor Bugbot on #6950. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(db): renumber the custom-block trace migration around a 0299 collision Staging landed its own 0299 (`table_run_dispatches.heartbeat_at`) while this branch was open. The two migrations are independent — different tables, no shared statement — so only the number and drizzle's snapshot chain collided. Regenerated rather than hand-merged: a drizzle snapshot is a full-schema dump whose `prevId` links it to its parent, so editing one by hand to sit after a migration it was not generated against is how the chain silently stops matching the database. Staging's 0299 and its snapshot are taken verbatim; this is 0300, generated against them, and its SQL is byte-identical to what it replaced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-21 18:21:08 -07:00
Siddharth Ganesan	58aa6379e0	feat(cli): add chat command (#6937) ... * feat(cli): add chat command * fix(cli): harden chat command execution	2026-08-21 12:59:08 -07:00
Waleed	dbbe99e473	fix(integrations): validation pass over Crunchbase, PitchBook, and CB Insights (#6925) ... * fix(crunchbase): widen tier-gated collection allowlists and cap the deleted feed The deleted-entity, autocomplete, and fields-metadata allowlists each held only the collections the narrowest package tier publishes, so requests valid on a richer package were rejected locally before any request went out. An Advanced Financials key could not read the funding-round deletion feed at all. - deleted-entity collections: 9 -> the 14-collection union across all tiers - autocomplete and fields-metadata: 14 -> all 43 collections - clamp the deleted feed to its documented max of 25, not Search's 1000 - offer "All collections" so the cross-collection feed stays reachable - name the richer-tier card additions instead of presenting the base set as exhaustive Also rewrites a test that asserted the broken behavior and tightens a substring URL assertion that passed on the value it was meant to reject. * fix(pitchbook): stop a rejected API key reaching block output and logs PitchBook's 401 body echoes the submitted key back inside `message`. No PitchBook tool declared an `errorExtractor`, so the failure fell through to the generic chain, whose first entry returns `data.message` verbatim — putting the credential in the block error, the run log, and any agent context reading the failure. The existing scrubber sat in `transformResponse`, which never runs on a non-ok response. - add a `pitchbook-errors` extractor that replaces the unauthorized message with a fixed string, and wire it through all 91 tools - the extractor returns undefined unless the body carries a `message`, so a foreign 401 on the shared fallback chain is never labelled a PitchBook failure - correct `investor_preferences.preferredIndustry` to the shape the API returns - make `company_industries.emergingSpaces` opaque; its item shape is undocumented - reject a non-list of article ids instead of throwing a bare TypeError * fix(cbinsights): reject malformed input instead of silently rescoping a billed query CB Insights is metered, so a filter that fails to parse must fail the request — dropping it does not narrow the result, it charges for a query the caller never asked for. - reject an unrecognized boolean rather than dropping it, which had been widening a VC-backed firmographics search - reject a non-numeric limit instead of falling back to the endpoint default - reject non-text filter entries instead of stringifying them to "[object Object]" - accept only asc/desc for sort direction; a typo had returned the bottom of a metered result set as though it were the top - treat a whitespace-only numeric bound as unset, not as zero - drop `totalHits`/`totalHitsRelation` from list business relationships; that endpoint reports no total, so both were permanently null - trim `nextPageToken`, matching the id fields Also moves the token cache onto `lru-cache` per the in-process caching rule, replacing hand-rolled TTL arithmetic and a manual prune. * chore(harmonic): drop the team-key help text from the credential descriptor * chore(tools): regenerate tool metadata for the validation fixes * fix(tools): redact the retained error body, not just the message Scrubbing the extracted message left the raw provider body reachable: `createTransformedErrorFromErrorInfo` attaches `errorInfo.data` to the thrown error and the executor surfaces it on the failed tool's `output.data`, so a PitchBook key rejected with an echoing 401 still reached block output and agent tool results via `output.data.message`. - add an optional `redactData` to the error-extractor contract, so an extractor that exists because a provider echoes a credential can replace the body too - retain `redactErrorData(errorInfo, extractorId)` in place of the raw body - PitchBook replaces only the unauthorized body; every other failure is untouched - cover the executor path itself, since asserting on the redactor directly still passes when nothing is wired to it	2026-08-20 22:22:28 -07:00
Theodore Li	5b28da1989	fix(tables): accept plain row query predicates (#6916) ... * fix(tables): accept plain row query predicates * fix(cli): show table predicate group syntax	2026-08-20 20:48:47 -04:00
3 people	42f6287911	feat(byok): add organization-wide key inheritance (#6834) ... * feat(byok): add organization key management * feat(byok): inherit organization keys at runtime * feat(byok): add organization scope to BYOK settings * fix(byok): refresh org key state after mutations * fix(byok): hide stale inherited status badges * chore(db): drop colliding byok migration ahead of staging merge Staging independently claimed 0293. Remove ours so the merge is clean; it is regenerated at the next free index right after. * chore(db): regenerate byok migration at 0296 Staging claimed 0293-0295 during the merge; the regenerated SQL is byte-identical to the dropped 0293. * docs(byok): document organization scope, precedence, and the full provider list The BYOK section described workspace-scoped keys only. Add the organization scope, its Enterprise requirement, the per-provider precedence rule, what an entitlement lapse does, and the Pi sandbox exposure. Refresh the provider table from the settings page, which had drifted from 14 to 34 entries. * feat(byok): open organization keys to every organization plan Organization BYOK was gated on Enterprise, but an organization is the only thing that can hold the keys, so every plan that can own an organization should qualify — Pro for Teams, Max for Teams, and Enterprise. Add checkOrgPlan/resolveOrganizationPlan beside the Enterprise pair rather than widening checkEnterprisePlan, so the Enterprise-only gates (Access Control, whitelabeling) are untouched, and restore resolveOrganizationEnterprisePlan to module-private now that BYOK no longer needs it. * perf(byok): cache the organization entitlement, not the key material getBYOKKey runs once per agent block and once per hosted-capable tool call, so a loop over N items resolved N times — and each organization-inheriting resolution paid three sequential billing queries on top of the two key reads. Split the two reads by staleness tolerance. Key rows stay fresh, because revocation must be immediate. The entitlement is a billing gate that tolerates bounded staleness in the harmless direction (a lapsed organization keeps using its own key for <=60s), so cache it per organization with an in-flight share so concurrent blocks issue one query set. The management surfaces keep reading it fresh, so an organization that just upgraded is never told otherwise. Also run the block check and subscription read in parallel inside resolveOrganizationPlan, and carry the resolved scope on BYOKKeyResult so a log line can say whether a run used the workspace's key or an inherited one. * feat(byok): let workspaces store the Z.ai and Cohere keys the runtime reads Both ids were already in the BYOK contract enum and both are resolved at execution time — getApiKeyWithBYOK reaches 'zai' (GLM models are in the hosted catalog, so the BYOK branch runs), and 'cohere' backs both the Embeddings block and Knowledge Base reranking — but neither appeared in the settings list, so there was no way to store the key either path looks for. Cohere had no icon; add one from the official multi-color mark so it stays legible on a light and a dark page. Cohere's embed-v4.0 is kbEligible:false, so the description says 'Embeddings and Knowledge Base reranking' rather than claiming KB embeddings. * improvement(byok): shorten the workspace scope chip to 'Workspace' It sits beside 'Organization', so the scope reads from the pair; 'This' only added width. * fix(byok): do not cache a billing outage as an unentitled organization resolveOrganizationPlan maps a failed billing read to false, which is indistinguishable from a real plan lapse. The entitlement cache stored that, so one transient outage held the gate shut for the full TTL and every inheriting run silently fell back to a metered hosted key — and the cache's rejection path, which exists to prevent exactly this, was unreachable. Give the resolver the onError option its neighbours already have and let the cached read ask for 'throw', so a failure stays out of the cache and the next resolution retries. Behavior for the call that saw the error is unchanged: getBYOKKey still fails closed. Reported by Cursor Bugbot. * fix(byok): propagate the subscription read's failure too The previous commit threaded onError through resolveOrganizationPlan's own catch, but getOrganizationSubscriptionUsable soft-fails to null on its own, so a failed subscription read still arrived as an ordinary 'no usable subscription' and returned a successful false — which the entitlement cache then stored for the full TTL. Thread the option into that call as well. Test it at the billing layer rather than the cache layer: the entitlement test mocks resolveOrganizationPlan wholesale, so it could never have caught this. Verified the new test fails against the previous commit. Reported by Cursor Bugbot. * refactor(byok): cache the entitlement with LRUCache, like copilot entitlements The hand-rolled version reinvented three things the codebase already has a canonical answer for. lru-cache is a declared dependency of apps/sim and lib/copilot/entitlements.ts already caches an entitlement with it — by storing the in-flight Promise, which is what makes concurrent callers collapse onto one resolution with no in-flight bookkeeping at all. TTL and the size bound come from the library. That removes the second Map, the manual eviction (and its interaction with an in-flight entry), and the dead value-while-refreshing state: 23 executable lines. The one thing the library does not cover is dropping a rejected promise so a billing outage is not cached for the TTL, which is kept and pinned by a test that fails without it. TTL expiry is no longer re-tested — that is the library's behavior, not ours, and lru-cache reads its clock at module load so faking timers never moved it. * refactor(byok): coalesce the entitlement read with the shared singleflight lib/concurrency/singleflight.ts is the codebase's coalescing primitive and oauth/credential-service.ts already pairs it with a read-through cache. Adopting that shape fixes a case caching the promise directly did not: a *hung* billing read wedged every caller for the full 60s TTL, where coalesceLocally evicts and rejects at its settle deadline. It also removes the hand-rolled rejection eviction — the cache is written only on the success path, so an outage leaves no entry by construction. The cache now holds booleans, which introduces the one trap worth a test: a truthiness check would read a cached false as a miss and re-query billing on every resolution for lapsed organizations. Pinned. * fix(byok): keep an abandoned entitlement producer from writing the cache coalesceLocally does not cancel a producer it timed out — its docstring says so explicitly — so writing the cache from inside the producer let a late billing result overwrite a fresher answer a retry had already cached, and hold it for a full TTL. Move the write onto the value the caller actually received. A caller that timed out throws before reaching it, so an abandoned producer now resolves into nothing. The test reproduces the overwrite and fails against the previous shape. Reported by Cursor Bugbot. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>	2026-08-20 17:45:28 -07:00
3 people	ea70f8dcf1	feat(harmonic): add contact workflow integration (#6902) ... * feat(harmonic): add contact workflow integration * fix(harmonic): sync docs manifest * fix(harmonic): address integration review findings * feat(harmonic): add the missing people endpoints and fix two error paths Extends the integration from 4 to 13 tools, covering every non-deprecated people-scoped Harmonic endpoint, and repairs two defects found by validating the existing tools against Harmonic's OpenAPI and API reference. New tools: - Enrich Person (POST /persons) — the only path from a LinkedIn URL or email a workflow already holds to a Harmonic contact. - Get Person, Get Company Employees — account-based sourcing; employees returns URNs that chain into Batch Get People. - Saved-search net-new results and their acknowledgement, so a monitor stops reprocessing the entire result set on every poll. - Bulk email enrichment: submit, poll, and quota, plus Get Enrichment Status. Fixes: - The error extractor dropped Harmonic's string and object `detail` envelopes. A tool that names an extractor gets no fallback chain, so every FastAPI abort surfaced as "Request failed with status 403". The enrichment 404 also carries the scheduled `enrichment_urn`, which was being discarded — that URN is the only handle on the job, so it is now kept in the message. - The saved-search selector failed the whole dropdown instead of degrading: the response cap was half the sibling value on an endpoint that is unpaginated and returns every saved search with its full query object, and the option ceiling threw rather than truncating. Raised to 1MB and switched to truncate-and-warn, matching the other data-driven selectors. Clearing net-new results now requires an explicit scope. Harmonic treats an absent `entity_urns` as "clear everything", so an empty field would have silently discarded the backlog. Scope deliberately excludes company-side, deal, typeahead, network, and Scout streaming endpoints, and every endpoint retiring on 2026-11-05. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-08-20 17:12:57 -07:00
Waleed	865f8173ab	feat(affinity): add Affinity CRM integration (#6908) ... * feat(affinity): add Affinity CRM integration Adds the Affinity v2 API as a block with 70 tools, covering 86 of the 87 documented endpoints. Only Send Feedback is omitted — it reports product feedback to Affinity rather than doing workflow work. Endpoint families that differ only by an entity segment are one tool with an entityType param, so companies/persons field, list, row, and relationship reads, the company/person merge endpoints, and entity notes each collapse into a single operation. * chore(affinity): regenerate the docs manifest for the new integration page	2026-08-20 16:27:29 -07:00
Theodore Li	a99f61bee9	feat(api): add v2 resource management endpoints (#6900) ... * feat(api): add v2 resource management endpoints * fix(cli): gate destructive v2 commands * fix(test): update v2 request-slice count * feat(cli): add shared workspace profiles	2026-08-20 18:55:57 -04:00
Waleed	a27f376164	feat(connectors): add Bitbucket, Databricks, Google Chat, and Workday Help KB connectors (#6895) ... * feat(connectors): add Bitbucket, Databricks, Google Chat, and Workday Help KB connectors Adds four knowledge base connectors, closing the gap where Sim shipped tool blocks for these services but could not index their content. - Bitbucket: repository source files and pull request descriptions over the existing Bitbucket OAuth credential - Databricks: notebooks (Workspace API) and saved SQL queries, PAT auth - Google Chat: spaces indexed as message transcripts, new google-chat OAuth service under the shared Google client - Workday Help: knowledge article versions via the public helpArticle/v1 API * fix(connectors): second-pass validation fixes and test coverage Adversarial re-validation of all four connectors plus a combined-change regression audit. - bitbucket: stop declaring incremental sync (deletion reconciliation is disabled for incremental runs, so deleted files were never removed on the default code configuration); drop a wasted listing round-trip after the frontier drains; add 33 tests - databricks: reject an explicit maxDocuments of 0, which meant unlimited; add 34 tests - google-chat: correct the sender displayName documentation (user auth populates only name and type) and emit second-precision RFC-3339 in the message filter; 15 -> 24 tests - workday: fix a crash when maxVersions is persisted as a number; refuse a configuration whose status filter Workday did not honor; cap the unresolved-name error; 18 -> 27 tests - document the Google Chat service-account omission - docs: list all four connectors and correct the connector count * fix(connectors): index Google Chat spaces with no messages in the window Review round 1. - orderBy takes a full ordering expression, not a bare direction. The reference documents the default as `createTime ASC`, so send `createTime DESC`; a bare `DESC` either 400s every hydration or is ignored, which would make the cap keep the oldest traffic and the later reverse render the transcript backwards. - getDocument no longer returns null when the message window is empty. A space with no messages is still a live space, and null is the "document is gone" signal the engine treats as last-known-good: returning it dropped spaces whose only prose is their description or guidelines, and left a stale transcript indexed after a space was cleared or lookbackDays was tightened past every message. - The transcript header is omitted when no message contributed text. * fix(connectors): only flag a Bitbucket listing capped when the cap withheld something Review round 2. takeIndexableWithinCap reports capReached as soon as the running total equals maxItems, which is also true of a listing that ended at exactly that count. Setting listingCapped there suppressed deletion reconciliation for a complete listing, so upstream-deleted files and pull requests could stay in the knowledge base indefinitely. applyMaxItemsCap now takes whether Bitbucket had more content beyond the page -- a next link, or directories still queued on the frontier -- and flags the listing only when the cap actually withheld something, matching the Databricks, Google Chat, and Workday connectors. * fix(connectors): keep the Bitbucket cap flag set when it skips the pull request phase Review round 3. Fixes a regression from 622a21bd9a. maxItems is shared across the code and pull request phases, so a code walk that ends with exactly maxItems documents and no next link or frontier stops pagination before the pull request phase runs. Scoping listingCapped to "this phase had more" left the flag unset in that case, and the engine then treated the run as a complete enumeration and could hard-delete previously indexed pr:* documents that were never listed. The cap flag now asks whether anything the connector was configured to list remains unlisted -- including a later phase the cap is about to stop us reaching.	2026-08-20 14:46:29 -07:00
3 people	d9cfd7c68e	improvement(mothership): v0.9 (#6815) ... * checkpoint * Checkpoint * dot fixes * Make async tool resume delivery recoverable * Support split table tools and option recovery * Harden VFS mutation handling * feat(platform): platform subagent support — docs corpus VFS, search_docs, account context Squash of the feat/platform-agent branch (sim side): mounts the Sim docs corpus in the copilot VFS, wires search_docs and retires the legacy docs search tools, and syncs the generated tool catalog and trace contracts for the platform subagent. * Align Copilot tools and resource handling * Expand workflow log query support * checkpoint * Port desktop-improvements-0 desktop and browser-agent work * fix(desktop): keep browser-agent input alive through live-SPA re-renders * Harden workflow sanitization and Slack setup * feat(desktop): coordinate clicks, caret insertion, and drag for the browser agent * Revert subagent group eager auto-collapse * fix(chat): keep sends FIFO across the streaming-to-idle drain gap * Add the steering backend surface for mid-turn sends * Sync generated contracts for async subagent orchestration Pulls the mothership tool catalog (wait_agents / tail_agent / steer_agent / interrupt_agent), trace spans (chat.async_subagent.*, chat.orchestrate.*), and trace attributes (copilot.async_subagent.*) into the generated TS contracts. * Add display titles for the async subagent orchestration tools wait_agents / tail_agent / steer_agent / interrupt_agent get natural-language running titles (naming the agent id being waited on, tailed, steered, or stopped) and a Steering→Steered completed-verb rewrite. * Show orchestrator-chosen subagent names on agent groups A subagent_start whose payload data carries a name (the orchestrator's new name trigger parameter) now labels the agent group with that mission name — the agent-type icon stays. The name flows through the live stream path, the turn model (AgentNode.displayName) and its serialize/rebuild round-trip, and persisted transcripts (PersistedContentBlock.name), so reloads keep the label. * Improve Copilot error handling and logging * Backfill the subagent display name from the second start event The dispatch-time subagent_start fires before the trigger args (and therefore the name parameter) have streamed; the phase-3 start re-announces the lane with the name. The block builder was dropping that duplicate wholesale, losing the name on streaming providers — now it backfills subagentName onto the existing block instead. (The home turn-model path already reconciled this case.) * Support Slack bot connection flow * Harden Copilot error and VFS handling * Harden VFS resource operations * Show 'Waiting for the first of N agents' for mode-any waits The wait_agents title ignored the mode argument, so an any-mode wait over three agents read 'Waiting for 3 agents' while the model narrated waiting for the first — contradicting the transcript. * Collapsed-by-default agent cards with live intent status lines Subagents now narrate their work through <intent>3-5 words</intent> tags (a fleet-wide prompt protocol on the mothership side). The turn model streams each subagent's text through a split-safe tag parser: complete tags update the agent's currentIntent and disappear from the prose, tags split across deltas are carried until their close arrives, and a tag that never closes flushes back as plain text. The agent card renders as one line — display name (or agent label) plus the latest intent, replaced inline as the agent shifts gears — and never auto-expands; expanding to the full tool log is a deliberate click. Only an outstanding permission prompt or a browser hand-back forces a group open. Intents persist on the subagent block (and through the legacy persisted- message paths) so reloads keep the last status, and a renamed reinvocation now takes the latest name instead of pinning the first. * Add the internal in-band tool execution route for live mothership turns POST /api/copilot/tools/execute (INTERNAL_API_SECRET, Go→Sim) runs one sim-server tool through the same server tool router the resume driver uses and returns the result synchronously — no checkpoint. This is what lets background (async) subagents write files/tables/knowledge, and lets the main lane keep streaming (instead of checkpoint-pausing and killing every background run) while async agents are live. * Persist resource side effects for in-band tool execution Files/tables created through the internal execute route now register on the chat's resources exactly like the resume driver's executions — the route runs the same handleResourceSideEffects pass (persistence only; an out-of-band route has no live event sink, so mid-turn chip pushes are a follow-up). * Extract intents from group text on every path, sync and async The turn-model intent filter only fires for span-scoped subagent lanes, but this surface also delivers subagent text through the legacy block path — so <intent> tags flowed through unparsed and rendered as prose rows. Groups now extract intents from their accumulated text at append time: the last complete tag becomes the card's status line and every complete tag is stripped from the rendered prose. Covers span-scoped, legacy, and persisted-reload paths for both synchronous and background delegations. * Fall back to the live tool title for the agent card status line Persisted data proved tool-first subagents (grok search agents) emit zero prose, so intent tags never stream no matter what the prompt says. The collapsed card now always narrates: the agent's own <intent> tag when present, else the latest tool's display title while the lane is live. * Catch subagent <intent> tags in the server relay The relay's subagent text handler now runs the split-safe intent extraction as chunks stream: the latest complete tag is stamped onto the lane's persisted subagent block (subagentIntent) and stripped from the stored prose, so live, persisted, and replayed views all agree. Per-lane carry handles tags split across chunks; a never-closing tag flushes back as plain text. * Drop the tool-title fallback: the status line is the agent's intent With the intent protocol now injected into every spawn's task message, agents open with an <intent> tag; the card shows that narration or nothing. * Replace intents with live tool-title status lines on agent cards Intent parsing is fully removed (turn model, relay handler, persistence fields, group extraction). The collapsed card's status is the latest tool call in its RUNNING phrasing — never the completed rewrite, which stays in the expanded log. Parallel tools show the most recently started still-running title with a +N for concurrent siblings; between rounds the last title stays frozen; a closed lane shows the bare name. Nested agent cards compute their own status recursively from their own items. * Keep the main Sim lane live-expanded; collapse only real subagent cards The mothership group is the turn's own narration, not a delegation card — collapsing it hid main-lane text and tools until manual expand, which read as mis-ordered streaming while async subagents interleaved. It keeps the original live-expand behavior and no status suffix. * Persist subagent lane lifecycle blocks from the span handler Lane-scoped span events route to the span handler, which only recorded trace side effects — no subagent start block was ever persisted (verified: a seven-agent run stored 104 blocks with zero starts). Grouping then fell back to keying lane content by agent NAME, so a respawned agent of the same type merged invisibly into the first one's card until it resolved. The handler now persists the start block (spanId-keyed and deduped, carrying the display name) and stamps endedAt on close, giving every invocation its own card. * Name agents in orchestration titles; '+ n more' overflow format wait/tail/steer/interrupt titles humanize the slugified agent ids back to their display names ('Waiting for the first of Digest Workflow Build + 4 more'), and the agent card's parallel-tool suffix uses the same '+ n more' format. * Harden in-band tool execution and resources * Route in-band execution through the comprehensive tool dispatcher The internal execute route used the bare server-tool router, which rejects VFS tools with 'Unknown server tool: read/glob/grep' — so nearly every background agent's first discovery call failed (102 in-band calls in one run, dozens rejected). It now uses the relay's executeTool dispatcher: registered handlers (VFS, function execute) with permission checks and param normalization, falling back to the app tool router — the same surface foreground execution gets. * Harden chat stream transition handling * Harden VFS provenance and resource writes * Standardize tool environment references * Harden browser panel and chat cleanup * Descriptive, user-language tool titles across the board House rules applied everywhere: use every argument the call carries, never name internal machinery, and never lead with Getting (the Got rewrite is deleted so it cannot return). - Deployments name the workflow: Deploying {workflow} as API/chat app/MCP tool - Workflow reads name the part: Reading {workflow} meta/state/deployment/notes; generic reads always name the file (Reading {leaf}), never bare Reading file - Block runs name block and workflow: Running {block} in {workflow}, Running from {block} in {workflow}, Running {workflow} until {block}, and Enabling/Disabling {block} in {workflow} - The six split-table tools get per-operation verbs (Adding column {name}, Updating rows, Wiring automation, Creating view {name}) instead of a wall of Querying table - The manage quartet drops X-action system-speak for gerunds - get_* internal names become user language (Checking run settings, Tracing block inputs, Reading the deployed version); web_fetch says Fetching - Scheduled-task titles removed entirely (feature deleted from the Go catalog) - New verb rewrites: Fetched, Traced, Wired, Configured, Looked, Rotated * Deploying {workflow} as chat, not as chat app * Loader gerunds; mv names both ends; mkdir names the folder search_integration_tools -> Finding the right integration; load_integration_tool -> Loading {integration} tools; load_skill -> Loading skill {name}; run_enrichment -> Looking up {subject}. mv prefers the model's phrasing, else reads 'Moving {files} to {destination}'; mkdir reads 'Creating folder {name}' from the path. * Overflow counts read '+ n', dropping 'more' * Unify workspace find and search * Scale desktop title bar with page zoom * Serialize account and organization truth into the copilot VFS Workspace standing, membership, billing, org role, access-control restrictions, published-block provenance, and fork topology were reachable only through three parameterless tools (or not at all). They are ambient read-only facts, so they belong in the VFS where they are greppable, cost no tool round-trip, and every agent that can read gets them — the same move that retired get_blocks_and_tools and list_user_workflows. Adds account/{workspace,workspaces,members,billing}.json (always mounted) and organization/{organization,access-control,custom-blocks,forks}.json (only when the workspace is org-hosted). Every file projects an existing use case or util after getOrMaterializeVFS's access assert — no new queries, no new authorization. One relation per file, cross-referenced by id-and-name stub, so overlapping facts cannot disagree. Volatile content (billing, access control, forks) is lazy, so numbers are read-time fresh and unasked-for reads cost nothing. Projection follows the viewer: member emails are admin-only, fork detail requires workspace admin on a forking-enabled org, and the whole organization/ namespace is absent for a personal workspace — which is itself the answer. Retires get_account_billing, get_enterprise_context, and list_user_workspaces along with their handlers; display titles stay for transcript replay. * Fix insert_text refusing an editable field focused inside a frame describeFocusedEditable descended shadow roots but not frames, while activeElementReadback descends both. Focus inside a same-origin frame therefore surfaced to the first as the FRAME element — not an input, not contentEditable, not a canvas, no textbox role — so it fell through to 'not-editable' and insert_text refused a field that press_key had just typed a character into. Two functions answering 'what is focused' with different answers is the bug; the descent loops now match exactly. The refusal also names what actually held focus (tag, role, contenteditable). A bare 'not-editable' gave the agent nothing to act on, so it guessed at the cause — a real run spent twenty rounds on the wrong theory and had to be stopped by the user. * Keep retired browser takeover renderable in history The tool is gone from the catalog, so its generated constant went with it and every path that referenced it stopped compiling. Deleting those paths instead would have silently downgraded every past transcript containing a takeover card to a generic tool row, and dropped the no-timeout budget that an in-flight takeover still needs while a rolling deploy finishes. retired-tools.ts gives the literal a documented home that says what it is and why it survives its tool. * Follow the agent into a tab it opened to work in browser_open_tab created the page with activate: false, so the agent worked in a tab the user could not see while the panel sat on a page where nothing was happening. The panel now follows a tab the agent deliberately opened. Scoped to that tool only. A page spawning its own tab (popup, target=_blank) is the site grabbing the view rather than the agent choosing a workspace, and stays in the background as before — two existing tests pin that and caught the first version of this change, which moved both. A tab the user claimed still wins over both: the work starts in the background instead of pulling the page out from under them mid-read. * Make the browser tools agree with each other An audit of the module found the frame-descent bug was one instance of a pattern: six independent definitions of 'is this editable' and seven of 'what is focused', disagreeing with each other. A tool refusing what its sibling accepts on identical page state is invisible at runtime — the agent follows a snapshot that says one thing into a tool that says another. - browser_type now accepts role="textbox" like browser_insert_text does. The snapshot advertises those elements as [textbox] with a ref, so refusing them meant rejecting exactly what the outline told the model to type into. Both the native and synthetic paths, and their descendant scans. - pressKeyOnPage descends shadow roots and frames like every other focus reader. It was dispatching synthetic keys at the shadow host or <iframe> element, where they bubble but never reach the editor, while reporting success — and contradicting the activeElement reported beside it. - not-editable and ambiguous-editable name what was found: the element's tag and role, and the candidate fields. Both had the data and discarded it, which is what turns one blocked step into twenty rounds of guessing. - obstructedAfterNavigation requires a dialog that ARRIVED with the navigation. It compared against nothing, so every SPA route change under a persistent role=dialog reported a successful click as obstructed. The test that covered this asserted the false positive; it now pins both directions. - browser_insert_text observes the top document when typing inside a frame, like every other input tool. A submit that navigates the top page was invisible to its frame-scoped observation. * Let hover actually see what it mounted Four independent defects made browser_hover blind to the most common thing a hover produces — a row's action bar — so it reported no effect on a hover that worked, and the agent fell back to clicking pixels off screenshots. - The popup scan matched only role=tooltip/menu/listbox. Slack's message shortcuts bar is a labelled toolbar/group, so it registered as nothing at all. Added toolbar, menubar, labelled group, and [popover]. - The baseline was captured BEFORE prepareElementSurface scrolled the target into view, so scrollChanged was always set by the tool's own probe. That pinned every unproductive hover to 'background DOM churn' instead of the honest 'nothing happened', and hid scrolling the hover really caused. Re-baselined once the scroll settles and before the pointer moves. - The MutationObserver attached only on the first observation, while the roots list is rebuilt every call and grows as shadow roots mount. Components that appeared later were never observed, so their DOM changes raised no revision. Roots are now observed as they show up. - observationTruncated was computed and never read, so a scan capped at 12k nodes reported 'nothing appeared' with the same confidence as a complete one — and portalled overlays live at the end of <body>, exactly what the cap drops. Hover now says the page was too large to scan and to confirm visually. * Stop the browser agent acting on the wrong element, and say why it refused Four findings from the module audit, the first of which could silently do the wrong thing rather than merely fail. - A ref whose node is gone is re-adopted by structural resemblance, matching on ORIGIN only so a pushState between snapshot and act does not kill every ref. That leniency also let a ref to a row control in one view rebind to the identical control in a view the app had since navigated to — acting on the wrong message, signalled by nothing louder than recovered: true. Adoption now requires the same path; a view swap reports the ref stale, and the caller re-snapshots. Revalidating a still-connected node stays lenient, because that is literally the node the model chose. - A hit INSIDE the requested element is its own nested control, not an overlay. Both produced 'covered by X — close or move the overlay', advice that cannot be followed because there is nothing to close. Nested hits now say so and point at retargeting. - browser_click_at, browser_insert_text, and browser_drag listed targetChanged in their effect formulas, but none passes an elementId, so no targetState is ever captured and the term was always false — coverage that read as real. Removed, with a test pinning the dependency. - The seven effect formulas are deliberately NOT collapsed into one predicate: drag must trust domChanged where others must not, hover must ignore field and focus changes, click counts focus only for editables. Forcing one would make each tool wrong differently. The differences are now documented in one place next to the shared computation, so divergence is a declared policy rather than an accident. * Let edit_workflow configure block retries * Updates * Always focus the resource the agent is working on, and its browser tab The resource panel had a carve-out: an already-open browser session declined to replace another selection and only got an attention marker, so agent browser work happened off-screen. The panel now follows the agent to whatever it touches — browser included — and an event can still opt out explicitly. The browser panel also follows the agent BETWEEN tabs: the store already tracked automationTabId (and the strip marked it), but the visible tab never changed. It now switches when the agent's target tab changes, so watching the agent never means hunting for the tab it moved to. Keyed on the target changing rather than on it being set, so a user who browses elsewhere mid-run is only pulled along when the agent itself moves. * Never paint a browser snapshot at stale geometry (the modal-open flash) Opening a modal locks scroll, which removes the window scrollbar and reflows the panel — so a capture taken before the lock describes a rect the panel no longer occupies. The handshake painted that frame anyway and only then retried, so the replacement landed visibly offset from the page it stands in for: the flash. A capture is now checked against the host's live rect before it is painted; a mismatched frame is skipped and re-captured at the settled layout instead (modal retries go 2 -> 3 to absorb the extra settle). * Name the workflow in deployment and workflow-scoped tool titles 'Checked deployment status' never said which workflow — nor did the deployed- state read, run settings, block outputs/inputs, redeploy, promote, or the global-variable write. These tools carry a workflowId (often defaulting to the current workflow), so only the client can resolve a name: the enrichment layer now resolves it for the whole workflow-scoped family and passes it as workflowName, which every workflow title already reads. Titles: Checking {workflow} deployment status, Reading deployed {workflow}, Checking {workflow} run settings, Reading {workflow} block outputs, Tracing {workflow} block inputs, Redeploying {workflow}, Promoting {workflow} version {n} to live, and 'Adding workflow variable {name} in {workflow}' — each falling back to its unnamed form when no workflow resolves. * Name the block that ran; never fall back to a raw block id run_block and set_block_enabled carry only a blockId, so their titles would have printed an opaque UUID ('Running 7f3a2b91-… in Invoice Sync'). The enrichment layer now resolves blockId against the workflow store the same way it already did for run_from_block's startBlockId, and the base titles no longer accept an id as a name — an unresolved block reads 'Running block' rather than a UUID. * Add the missing Removing -> Removed rewrite The table work introduced 'Removing automation'/'Removing enrichment' with no past form, so those rows kept their present tense after completing. * Name the target resource in the remaining tool titles Table tools keep their operands nested under args and identify the table by id, so their rows said 'Adding rows' with no hint where: enrichment now lifts the nested args and resolves tableId against the cached workspace table list, giving 'Adding rows to Runtimes', 'Adding column status in Runtimes', 'Reading views of Runtimes'. Also: a block-schema read names the block instead of the file ('Loading Slack', 'Loading Google Sheets tips'); browser type/insert show the text they send, middle-ellipsized; downloads name the file; library-docs searches name the library and query; knowledge-base searches include the query; generated media names its output file; and diff_workflows, list_deployment_versions, and publish_custom_block joined the workflow-name enrichment set. * Bubble nested agents' tool calls into the parent's status line The collapsed status only scanned a group's OWN tool items and skipped nested agent groups, so a parent that had delegated froze on its last own tool while its child did the actual work — the line described nothing that was running. Status now walks the whole subtree: any tool at any depth counts, the most recently started running one is shown, and the rest become the same '+ n' overflow. With nothing running it falls back to the last tool at any depth, so an idle parent still reflects where its subtree got to. * Align nested tool call status rows * Revert branch-local KB connector error-message edits Restores apps/sim/connectors/ to staging state. Two copilot-focused commits on this branch (3a9fc1c5a5, 24b24e5f8e) drove by nine KB source connectors (airtable, confluence, discord, github, gitlab, google-drive, microsoft-teams, notion, slack) to enrich credential-validation error messages. The env-reference resolution machinery those errors supported is kept; the product connector surface stays unchanged in this branch so the staging promotion remains scoped to copilot work. * Fix the failing audits: NUL escape and route-count ratchet check:source-text: resource-vfs.ts used a raw NUL byte as its folder-index key separator (comment and template literal), which makes git treat the file as binary and hide it from review. Written as the '\u0000' escape — the runtime string is identical. check:api-validation:strict: baseline 1120 -> 1122 for the two routes added on this branch since the last bump. * Add Force Reload (Cmd+Shift+R) to desktop View menu * Route Force Reload through focused-resource boundary; regen docs manifest * Align the title-bar surface audit with the zoom rework 'Scale desktop title bar with page zoom' (833d2f126d) changed the CSS contract in two ways its audit test still pinned the old shape of: the lane vars gained a max() floor around the platform env() terms so page zoom cannot shrink the lane below the OS-drawn lights, and the control square became fixed px — CSS px already scale under zoom — leaving only the centering offset derived from the lane height. The test required the bare env() prefix and calc() on all three control vars, so it failed the commit that implemented its own regression comment. Pins now assert the env() term inside the clamp (still platform-derived, the test's actual intent) and split the control vars: offset must stay computed, size and icon are explicit constants. * Budget the two structurally slow tests explicitly sso-trust imports the whole Better Auth module graph (2.5s on an idle machine) and events.attribution scans call sites across the repo. Under a fully-parallel uncached run on a loaded machine both blow the default timeout while passing in isolation and on CI — a verdict decided by machine load, not by the code. 30s budgets make a loaded local run mean what it says. * Carry the cross-service trace id in sim log lines * Improve nested tool status presentation * Refresh secret schemas and shared test mocks * Open the deployed graph of org-published blocks, read-only, org-wide A consuming workspace could see a published block's interface but never what it does: the backing workflow lives in the publishing workspace, and other workspaces are nameable, not readable. Publishing a block org-wide is the act of sharing it, so the graph it executes is now readable from any org workspace — the DEPLOYED graph, not the publishing workspace's live editor state, so nothing in-progress leaks and what you read is what runs. The namespace also adopts the root's index/detail split: custom-blocks.json slims to names with a detail pointer, organization/custom-blocks/{type}.json carries provenance plus the deployed graph (loaded lazily through the cached loadDeployedWorkflowState; credential ids and env references inside it belong to the publishing workspace and say so), and organization/README.md is the namespace guide WORKSPACE.md is at the root — files, usage, the in-depth block inventory, and forks.json documented only when actually mounted. The block list moves to materialize time (same indexed query the components pass already runs) because the README, the index, and the per-block key-view entries all need it; only the graph stays lazy. * Withhold the deployed graph from external collaborators Workspace access and org membership are different grants: an external collaborator can open the workspace and use the published block, so the names index and the interface schema stay visible to them — but the deployed graph is org implementation internals, and their detail files now simply do not exist. isHostOrganizationMember is the viewer bit the host context already resolves for exactly this distinction. * Fill out the organization namespace: workspaces, permission groups, credential groups Three more read-only files, all lazily loaded — the paths appear in the key view so glob discovers them, but no query runs until a read — and all gated by registration, so an unpermitted viewer's file simply does not exist: - workspaces.json (org members): the org's full workspace map with the viewer's access flag and fork parentage — account/workspaces.json only ever showed what the viewer can reach. Inaccessible workspaces stay nameable, not readable. - permission-groups.json (org admins): every group with member count, targeted workspaces, and the restrictions its config activates. access-control.json remains the per-viewer binding. The queries are lifted into lib/permission-groups/queries.ts because their only prior home was inline drizzle in the route handlers, which the VFS cannot import. - credential-groups.json (entitlement-gated): per-option configuration readiness and enrollment progress — the two facts that decide whether a credential_group workflow will do anything at runtime. The note teaches the contract that bit the audit: an active group with zero completed enrollments yields an empty loop, not an error. Enrollee emails are workspace-admin-only, matching the settings page; counts come from the first enrollment page and say so when truncated. The README documents each file only when mounted for this viewer. * Stop reporting a click that navigated as a failed click The field case: 'Begin Assessment' submits a form. The navigation tears the origin document down while the press completes, so everything after dispatch — the CDP call's own completion, the synthetic dispatch's return value, the postcondition reads — fails against a destroyed context, and a maximally successful click was reported 'Failed clicking element'. The agent's own follow-up investigation in the transcript diagnosed exactly this. navigationRescue detects it at the driver level, where the navigation epoch and URL survive the renderer teardown: when the page provably navigated since dispatch began, a dispatch-path failure becomes a success carrying navigatedDuringDispatch and a note explaining why no postconditions exist. Applied to all four click dispatch paths (unframed CDP, framed native, synthetic in-page, click_at). The soft path had the same blindness: with the after-state unreadable, urlChanged computed false and a navigating click reported 'no observable change'. navigatedByDriver now folds into navigated/effectObserved, which also keeps the new-dialog obstruction check meaningful on real navigations. * Re-hide the browser under a modal after main loses the occlusion lease The punch-through: a modal opens, the native view hides behind a painted snapshot, and the renderer records applied: true. Then one heartbeat commit is skipped — renderer jank past the 2.5s bounds-lease TTL is enough — and main expires the lease, resetting panelOccluded on its side. The next heartbeat finds the modal marker still present and calls setDesired(true), but the lease's dedupe sees applied === desired and sends nothing; the bounds commit that follows lays out an unoccluded native view above the open modal, and no later event ever re-hides it. The comment on this branch already claimed it 'reasserts the lease' — the dedupe made that claim false exactly when the lease had been lost. While the occlusion marker is present, each heartbeat now drops the applied belief (assumeRevealed) before setDesired, so the reassert is a real, forced, idempotent hide IPC — one per second while a modal covers the browser — and any main-side lease loss self-heals within a heartbeat. * Make a stale ref name which of its five causes fired A field run burned five snapshot->click cycles on a STATIC landing page, every one refused with the same sentence — 'the page changed since the last snapshot' — and the agent reasonably concluded the page was regenerating its DOM. It was not; the resolver was refusing, and the message could not say why. Five distinct conditions produced that one string: an id missing from the registry, a connected node whose identity drifted, the view-changed adoption gate, no confident replacement, and a replacement tie. The resolver now stamps the reason (with the drifted node's current identity, or the from->to paths for the view gate) and every stale producer carries it into the driver message. Same pattern as not-editable: a refusal that names its cause costs one round; an opaque one costs a loop and a wrong theory in the bug report. * Pulse throttling off when the browser view is revealed, so it actually paints The blank-page report: a navigation completes while the view is hidden, the page 'finishes loading', and the panel shows white until the user re-navigates by hand. invalidate() on reveal was already there but recomposites the LAST frame — and the last frame is blank, because background throttling suspended the rAF the page's SPA paints its first frame from. The reveal now pulses throttling off (forcing the renderer to produce a real frame), invalidates, and hands the policy back to the session a second later through reassertTabThrottling, which preserves the automation-tab exemption. * Let the agent browser join meetings and finish passkey sign-ins Camera and microphone: 'media' joins the agent partition's allowlist, but every grant is gated on the macOS grant first — asked via systemPreferences.askForMediaAccess so the system prompt appears on first use, and answered from getMediaAccessStatus on checks — so System Settings stays the real authority and a page can never hold a grant the OS refused. Granting site permission without the OS grant produced the misleading NotReadableError Google Meet showed. Packaging gains the camera entitlement and usage string (macOS kills the process on prompt without one) and the mic string now covers meetings. Passkeys: WebAuthn itself is Chromium-native and nothing in our handlers blocks it — USB security keys need no permission at all. The hybrid transport (passkey on a nearby phone via QR) rides Bluetooth, which signed builds silently lacked: the bluetooth entitlement and usage string enable it. iCloud-Keychain platform passkeys remain outside what an entitlement here can grant — Apple restricts that to approved browsers. * Compile and preview Sim-styled pages * Wait for the batched prepare intent instead of instantly failing apply_file_edit The model batches prepare_file_edit and apply_file_edit into one round and the Go loop runs same-round tools concurrently, so apply could reach the executor before its prepare staged the intent. The instant no-intent error cost a model retry round and flashed 'Failed creating …' on the shared file row before the retry succeeded. The apply handler now polls briefly (10s cap) for the intent; a truly missing prepare still errors at the deadline. * Store page source, render docs-styled documents on view The pdf model for agent pages: the .html file keeps the markdown-shaped source (frontmatter + prose + sim: fences) and every surface renders the docs-styled document on demand — preview panel, /api/files/serve, public shares, and downloads all call the same pure compiler, now shared in lib/workspace-files. The docs chrome is reproduced from the real fumadocs source: the left sidebar's exact pill metrics, the clerk TOC with its animated scroll indicator, divider-style tables; cards and stats left the vocabulary. Table cells and kv values render inline markdown, and sim:workflow/table/knowledge/file links resolve to real workspace routes, bridged out of the sandboxed preview to the app router. Hand-written imitations of rendered output are rejected at apply_file_edit with a steer back to source, and a streaming page hides its source behind the live rendered preview (batched ~2s) the way a generating pdf hides its script. * Stagger the page rails so the resource panel keeps the docs sidebar Rails were gated at 1100px of iframe width — the chat resource panel never reaches that, so pages rendered single-column there. The rails now stagger the way the docs do on a laptop: >=640px keeps the section sidebar (240px) beside the content, >=1060px restores the full three-column frame with the clerk TOC, and only a truly narrow pane collapses to one column. * Fix in-page anchors escaping the preview; add the docs' toggle, code frames, pagination, and images Clicking a TOC or section link (or pressing Enter in the section filter, which clicks one) navigated the sandboxed frame off about:srcdoc in Electron and landed on a cookie-less sign-in page — the shell now intercepts every '#' anchor and scrolls directly. The page chrome gains the docs' exact theme toggle (emcn Sun/Moon, 30px rounded-lg, top right), framed code blocks with language label and copy button, and footer previous/next cards from prev/next frontmatter. Workspace images () compile to /api/files/view and the preview host inlines them as blob: URLs so the cookie-less frame can render them; sim:accordion joins the vocabulary as the faq component with title keys. * Size the section sidebar to its content so it appears at panel widths The left rail was a fixed 240/300px column, so it only earned its place once the pane was wide. fit-content caps at the docs width but shrinks to the longest section title (150px floor for the pills), and the two-column tier now starts at 560px instead of 640. * Let the clerk TOC join at 860px by sizing it to its content Same move as the section sidebar: the TOC column fits its longest link (capped at the docs' 268px, 150px floor), so the full three-column frame starts at 860px of pane width instead of 1060. * Open external links from pages in a new tab The preview bootstrap cancelled every non-anchor click, so an external link (the Sim docs, a vendor page) did nothing. External http(s) links now compile with target=_blank rel=noopener for the standalone and share surfaces, and the sandboxed preview bridges the click to the host, which window.opens a new tab — same channel the workspace deep links use. * Lock Sim pages to the rendered view via an internal record type The record's contentType is stamped text/x-sim-page when apply_file_edit detects page source — the file stays .html to the user (serving and downloads still emit text/html), but every surface now knows what the file holds before content loads. The viewer forces the rendered view for these files at every moment: the first streamed chunk (whose frontmatter is still partial) no longer flashes raw source, the gaps between an agent's tool calls no longer flip back to raw HTML, and both toggle surfaces (the Files toolbar and the resource-panel tabs) stop offering a code view for them. Mid-stream compiles run lenient — a fence still being written is malformed by definition, so its skip-notice callout is suppressed until the stream settles. * Honor the model's declared page type; default copilot .html to a page An explicit contentType on create_empty_file always wins (the skill now declares text/x-sim-page for pages, text/html for bespoke raw pages); with no declaration a copilot-created .html defaults to the page type. The first apply_file_edit still re-confirms from the actual content, and the category map knows the internal mime explicitly instead of falling through to the extension. * Default undeclared .html back to plain text/html A file is a Sim page only when the model declares it at creation or the first written content proves it — never by extension alone. * Match the docs' PageFooter for page navigation The invented bordered cards with Previous/Next labels are replaced by the docs' actual footer: the destination name with a 14px emcn chevron on a flex-1 hover pill (rounded-lg, px-3 py-3, --surface-active), next right-justified, and a spacer holding the empty half — verified against apps/docs/components/docs-layout/page-footer.tsx. * Scroll the rails invisibly, like the docs The sticky TOC box is overflow-y auto, and the clerk track's absolutely positioned SVGs could tip it a few pixels into overflow — Chromium then painted a full scrollbar beside the rail. Rails now hide their scrollbar chrome entirely (scrollbar-width none + webkit display none), matching how the docs scroll their sidebar and TOC. * Send sim:file links to the Files page, like a markdown link A workspace-file link in a page now navigates exactly as one tagged in a .md does: an in-app SPA push to /workspace/{ws}/files/{id} (the Files page with the file open). The fullscreen /view route stays reserved for the standalone surface; image refs keep the /api/files/view byte route. * Inline workspace images after the page compiles, not before The blob substitution ran on the raw source, where the compiled /api/files/view src it looks for does not exist yet — so the sandboxed cookie-less frame fetched every image itself and got 401s (broken image icons). The substitution now runs on the built document, covering compiled pages, legacy stored-compiled pages, and bespoke HTML alike. * Highlight the section you are AT in the left rail, not the last one visible The rail's current-section pick walked every heading visible in the viewport and kept the last h2 — so clicking a section landed correctly but highlighted whichever later section peeked in from below. Current is now the last h2 at or above the top reading line (matching the 72px scroll-padding a clicked anchor settles at), falling back to the first visible section when everything is below the line. * Stop the TOC jittering sideways as the highlight moves Active TOC links step from weight 430 to 470, and the rail is a fit-content column — every active-section change re-measured the longest link and shifted the rail a pixel or two side to side. Each link now carries a hidden zero-height ghost of itself at the active weight, so the column always occupies its bold width and the highlight moves without the layout moving. * Absolutize links and images in served page documents A downloaded page must behave like a downloaded .md whose links are absolute: clicking a workflow reference opens Sim in the browser at that workflow. The standalone renderer (plain download, fullscreen viewer, shares) now compiles sim: links and workspace image refs against getBaseUrl(); in-app surfaces keep relative paths and SPA navigation. * Drop the eyebrow; add the docs' top page controls The docs have no eyebrow line, so compiled pages no longer render one (old sources still parse; the field is ignored). The title row gains the docs' top controls: Copy page (copies the page text) and prev/next chevrons wired to the same neighbors as the footer cards, disabled-dim when a side is missing. * Read kv keys and table first columns as labels, the docs way kv keys dropped the blanket monospace — they render as the docs' row labels (500, primary, sans), with backticks in the source opting a code-like key (a path, an env var) into the inline-code chip; keys now run through inline markdown to make that work. Table body first columns pick up the same label treatment the docs tables show. * Platform font and emcn chrome for pages Pages live inside the app, and the platform — emcn, every workspace surface — renders the system stack, not the docs' Inter webfont; Inter made pages read as foreign next to the app around them. The face is now the platform stack with weights on the platform scale (400/500/600), the Inter delivery machinery (page-font.ts, the preview data-URI fetch, the public woff2) is gone, and the section filter wears emcn ChipInput's exact chrome (30px rounded-lg, --surface-5 fill flipping to --surface-4 in dark, --border, 14px, no focus ring). The docs' geometry — layout, spacing, rails, tables, code frames — is unchanged. * Color-only active state in the TOC — width can never move again Two attempts at reserving the bold width (a hidden ghost, then freezing measured rail widths) each traded one artifact for another: the ghost did not stop the fit-content column re-measuring, and the freeze made a long link wrap to two lines when it gained weight. The root cause was letting the active state change a width-affecting property at all: the active TOC link now shifts color only (muted to primary — the clerk thumb already carries the emphasis), the search input is a plain text field (the native search clear button is not emcn chrome), and the ghost/freeze machinery is gone. * Drop the Copy page control; keep the top chevrons The title-row actions keep only the previous/next chevrons (rendered when the page has neighbors); the Copy page button is gone. * Set-level sidebar: docs-style groups with the current page expanded Multi-page sets can now carry the whole set's sidebar. nav frontmatter (groups of labelled page links, identical on every page of the set) compiles into hidden set-nav markup whose sim: links resolve like any other; the shell lifts it into the left rail as muted group labels over page links, recognises the current page by title, and nests that page's section list beneath it — the docs sidebar's exact shape. Pages without nav keep the plain section list. * Center the content column at the docs' measure On wide panes the 1fr center cell stretched, so content hugged the left rail with dead space before the TOC. The main column now caps at the docs' ~760px measure and centers in its cell, matching how the docs balance a wide viewport. * Docs Steps, code-tab groups, and API method chips Three docs components join the vocabulary: sim:steps renders the numbered timeline (muted circle markers, hairline connector, title and content per step); sim:tabs renders the docs' grouped code block (mono tab chips, one pane at a time, the icon copy control targeting the visible pane); and a METHOD prefix on a set-nav page entry renders the API-reference chip — sidebar entries only, on the platform badge tokens (blue and purple added to the mirrors and the live bridge). Also: preview images switched from blob: to data: URIs — blob URLs are origin-bound and the sandboxed frame's origin is opaque, so Chromium refused to render them — and the page view-lock is sticky per file so a patch stream cannot flash raw source. * Downloaded pages carry their images Absolute URLs made LINKS survive a download, but an embedded image request from a downloaded file is cross-site and carries no session cookie, so images 401ed outside the app. The standalone renderer now inlines every workspace image the page references as a data: URI at serve time — like a pdf carrying its images — capped at 8MB per image, restricted to the page's own workspace, falling back to the URL reference on any miss. Applies to serve, download, and public shares. * Dead-center the content column between equal gutters The rails are content-sized and unequal, so the old grid (fit-content / 1fr / fit-content) skewed the middle cell toward whichever rail was narrower. The wide tier now uses the docs' geometry: a fixed 760px content column centered between two equal flexible gutters, the sidebar hugging the container's left edge and the TOC its right — rail widths can no longer move the content. * Defer title-bar history state out of currententrychange dispatch The Navigation API fires currententrychange synchronously from the history mutation that caused it, which can originate inside another component's useInsertionEffect (style libraries navigating during commit) — setState there trips React's 'useInsertionEffect must not schedule updates'. The arrow-state sync now defers to a microtask, flushing after the commit unwinds, with a disposal guard. * Show the section sidebar only when there are sections to list Fewer than two sections (and no set nav): the left rail and its filter disappear and the reserved left gutter collapses — the content column leads the container with the TOC trailing. Two or more sections, or a multi-page set, keep the full centered docs frame. * Execute the page shell in a DOM harness jsdom runs the real shell against compiled pages and asserts the layout decisions: both rails on a many-section page, only the left rail dropped on a one-section page. * Medium panes keep the TOC, not the sidebar Between 560 and 860px the frame showed the section sidebar and hid the clerk TOC — backwards by our own reasoning, since the sidebar is the redundant list on a single page. The TOC now survives at medium widths and the sidebar joins only on wide panes. * Sidebar is for doc sets only; stagger the rails like the docs The left rail now exists only for multi-page sets — on a lone page it just repeated the TOC. A set opens its sidebar at 560px with the TOC joining on wide panes (the docs stagger); a lone page waits until 700px and then shows the TOC alone. Set-sidebar spacing tightens to the docs values, with the current page's nested sections styled as small muted entries behind a hairline instead of full chips. * Center the lone-page content and TOC as a pair On a page with no sidebar the content column stretched while the TOC hugged the far edge, leaving a field of dead space between them. The content now caps at reading width with the TOC directly beside it and the pair centered in the pane, and the TOC waits until 800px so narrow panes stay single-column a while longer. * Equalize grid-template specificity across the rail tiers The 560px tier selects .art-cols:not(.no-side-nav) at (0,2,0), so the 860px tier's bare .art-cols template at (0,1,0) could never win and a set page on a wide pane kept the 2-column template — wrapping the TOC to the next grid row, bottom-left. The wide template now carries the same :not() guard, the 860 block's duplicate of the 800px no-side-nav rules is gone, and a test pins every template rule to equal specificity so a future tier can't silently lose the cascade again. * extract_doc_assets: pull a reference deck's assets into the workspace Sim-side handler for the new file-agent tool: given an uploaded .pptx or .docx, unzip it (OOXML is a zip), parse theme1.xml into theme.json (color scheme as hex, major/minor fonts, slide size from presentation.xml) and write every ppt|word/media file into a "<Name> assets" folder with original bytes and real content types. Re-runs overwrite the set in place. Pure extractor unit-tested against in-test-built packages; display label "Extracting assets from <file>". * extract_doc_assets learns .pdf via the doc sandbox PDFs have no zip structure or declared theme, so extraction runs in the same vetted sandbox that compiles and renders documents: poppler's pdfimages dumps every embedded image in its native format (masks filtered via -list), pdfplumber contributes each image's placement rects in page points plus the document's font names, and rendered pages are sampled into an explicitly-inferred color palette. theme.json for a pdf carries fonts, page size and count, the inferred palette, and a per-asset placement map. * Pages have one navigation rail; compile errors go to the agent The left sidebar leaves the renderer and the DSL: the shell builds only the content column and the clerk TOC (pair centered at 800px, one bare .art-cols selector per tier so the cascade cannot invert), the filter box goes with it, and nav frontmatter is tolerated but no longer rendered — sidebar METHOD chips and the set-nav markup are gone. Malformed sim: blocks no longer render a reader-facing "block was skipped" card: the block is omitted and the failure is reported as a diagnostic that apply_file_edit appends to its result, so the authoring agent sees exactly which fence to fix. The lenient flag existed only to suppress those cards mid-stream and is removed. The steps timeline connector now derives its position from the marker size, so it stays centered under the number circles. * Sync tool catalog: extract_doc_assets accepts pdf * Asset extraction yields the rebuild recipe, not just the parts pptx: theme.json now maps every image to its slide-by-slide placements (slide rels resolve rIds to media names; each pic frame's EMU offset and extent convert to inches) plus the slide count. pdf: a second layout.json is written — per page, the text blocks with content, position, font, size, and fill color; the filled rects (backgrounds and scrims); and rect-over-image overlay detection with coverage, which is the "image opacity" effect decks fake with a tinted rect. Stream alpha is unrecoverable, so overlays name the color and the rendered page remains the reference for strength. * Split shared-baseline text runs into separate blocks Two text boxes sitting at the same height merged into one wide line; a gap much wider than a space now starts a new block, so columns and label/value pairs land as distinct entries in layout.json. * Extract faithful document layout recipes * Add .chart files: live interactive ECharts docs, static or table-backed * Size charts by width-driven aspect, not panel height; separate title and legend * Map table chart rows from storage column ids to display names * Inject table rows as datasetIndex 0 so specs can transform; stagger array legends * Give .chart files their own bar-chart icon * Chart table sources gain groupBy/aggregate/pivot shaping; renderer-owned chrome * sim:chart page fence: ECharts SSR to themed inline SVG; shared option builder * sim:chart hydrated embeds: inline or .chart file refs, live table reads per serve * Finish extensionless pages and document staging * Render live charts without server hydration * Cover active-theme page token overrides * Keep artifact tokens synced with the app theme * Use tabs for multi-page Sim docs * Preserve dollar-prefixed tool credentials * Build chart specs from validated fields * Sync new integration docs into Copilot manifest * Add in-document tabs to Sim pages * Rebuild page TOC on tab changes * Keep page tabs with docs chrome * Stabilize tabbed page layout * Regenerate docs manifest for staging's Modal docs page * Share one divider between the bar and the chrome tab row * Pin the keyless OCR path in the unreadable-document test --------- Co-authored-by: Justin Blumencranz <96924014+j15z@users.noreply.github.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>	2026-08-20 14:05:46 -07:00
Vikhyath MondretiandClaude Opus 5	97c1688c49	feat(modal): add Modal Labs integration (#6896) ... * feat(modal): add Modal Labs integration Modal has no public REST control plane — the Python/JS/Go SDKs all speak gRPC — so this covers the two surfaces that are reachable over HTTP: deployed Web Functions/Servers, and the OpenAI-compatible Endpoints API. Three operations: call a deployed function with proxy-token auth, generate a chat completion on an Endpoint, and list the models a token can reach. Auth sends the token pair as Modal-Key/Modal-Secret rather than the combined bearer form, so a Web Function that validates its own bearer token keeps the Authorization header free. Both URL fields require https since Modal terminates TLS everywhere, and a cleartext URL would leak the token. Chat completion declares request.modelInput so the system prompt and user message project to canonical placeholders before egress. Call Function deliberately does not — a Web Function runs arbitrary user code, and nothing proves its body reaches a model. /v1/models fields beyond `id` are inferred from OpenAI compatibility rather than printed in Modal's docs, so they are marked optional. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(modal): type the wire payloads and default chat to the shared endpoint Chat Completion required an endpoint URL and passed a blank one straight into modalOpenAiUrl, which throws — while List Models already fell back to the shared inference host and the generate-on-modal-endpoint skill tells agents to leave the field empty for Shared Endpoints. Skill-driven chat calls against the shared host failed instead of using that default. Chat now falls back the same way and the block field is no longer required. Replaces every `any` in the Modal tools with declared wire types for the OpenAI-compatible /v1 payloads. Fields stay optional because the shape comes from whichever inference engine backs the endpoint, so the readers keep their defensive `??` guards — the types exist so a future change to that mapping fails the compiler instead of shipping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-20 13:12:34 -07:00
Theodore Li	4214a891f4	fix(setup): publish unscoped setup package (#6886) ... * fix(setup): publish unscoped setup package * fix(setup): strip renamed status command	2026-08-20 02:11:06 -04:00
Waleed	f6a9f0dd87	fix(integrations): white CB Insights tile and a borderless Crunchbase mark (#6884) ... CB Insights moves from a dark navy tile to white, matching Jira, Confluence, and Bitbucket. Its icon carries its own fills, so it stays legible on the lighter tile. The Crunchbase icon drops the white rounded-square plate and its border, leaving just the `cb` mark on `currentColor` so the block's bgColor supplies the tile. The viewBox is retargeted to the glyph's true curve extrema with padding that keeps it at the same optical weight as the surrounding brand marks.	2026-08-19 22:34:17 -07:00
3 people	e3a4874ece	feat(integrations): add Bitbucket Cloud (#6860) ... * feat(integrations): add Bitbucket Cloud * fix(bitbucket): enforce selector workspace slugs * fix(bitbucket): overfetch small pipeline log tails * fix(bitbucket): harden provider edge cases * fix(bitbucket): accept provider diff redirect specs * fix(bitbucket): stop advanced-field leakage and harden log, status, and selector paths Splits the `closeSourceBranch` advanced subBlock into per-operation ids. Advanced fields serialize without evaluating their condition, so a value set on Create Pull Request reached Merge Pull Request and closed the source branch unprompted. Also: - read step logs through the byte-capped server transport and map an empty-log 416 to an empty result, keeping a genuine 416 an error - trim a step log's partial leading line after the character cap rather than before, and never return an empty log when the retained window held content - surface Bitbucket's `error.detail` alongside `error.message` - treat commit-status `key`/`state` as nullable so one malformed row cannot drop a page - match repository `full_name` case-insensitively and reject dot segments in a workspace slug before the outbound request - type `reviewerAccountIds` as the comma-separated string it is - trim optional Bitbucket query strings; correct the token lifetime to two hours --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-08-19 20:32:44 -07:00
Waleed	9f346765fe	feat(granola): complete API coverage, note triggers, and connector validation (#6880) ... * feat(granola): complete API coverage, note triggers, and validation fixes Granola's public API exposes nine endpoints; Sim implemented three. Adds the remaining six and wires the new programmatic webhook-endpoint lifecycle into a managed trigger. Tools (6 new, 9 total): - get_transcript, list_audit_events - create/list/update/delete_webhook_endpoint Triggers: note.generated, note.edited, note.access_granted, plus an all-events trigger. The provider handler registers the Granola endpoint on deploy and deletes it on undeploy, scoped to the trigger's own event names, and verifies every delivery with the Standard Webhooks HMAC-SHA256 signature Granola returns on creation. event_id is the idempotency key, which Granola reuses across retries. Validation fixes to the shipped tools: - get_note dropped speaker.attribution ("me"/"them"); now surfaced - a 413 on get_note now explains that the transcript is too large inline and points at get_transcript, instead of surfacing a bare status code - note IDs are URL-encoded rather than interpolated raw - base URL, auth headers, and status-aware error handling are shared runtime helpers; params/outputs stay literal per file so the docs generator still reads them Tests cover signature verification (including replay and body-tamper rejection), event matching, subscription create/delete, and the block/tool contract — plus a guard that ids shared between the tool and trigger surfaces seed the same default, since block state is keyed by id and last-wins. The knowledge-base connector was validated against the spec and needed no changes. * fix(granola): correct array output schemas, listing-truncation signal, and docs Findings from validation passes over the tools, trigger, and connector. Tools — array outputs were declared as `type: 'json'` with `properties`, which describes an object, not an array. Agents and the output picker therefore saw `notes.title` instead of `notes[i].title`. All 15 array outputs (including the pre-existing three tools) now use `type: 'array'` with `items`, matching the 2000+ other tool files. The audit event `data` field stays `json`; it is genuinely free-form per the spec. Connector — `hasMore` was ANDed with the cursor, so a `hasMore: true` response with no cursor was reported as a complete listing. The sync engine treats exactly that shape as truncated and sets `listingTruncated` to block deletion reconciliation; masking it meant a partial first page could be taken for the whole corpus and reconciliation would hard-delete every note past it. Granola would have to violate its own contract to emit that shape, but the engine already handles it and the connector was hiding the signal. Also aligns mimeType with the `.txt`/text-plain bytes the engine actually writes (it was the only connector of 101 claiming text/markdown). Trigger — the setup instructions named a Granola settings path that does not exist; the help center says Settings > Connectors > API keys in the desktop app. Both list parsers now split commas inside array entries, so an array-wrapped free-text value cannot be sent as one malformed identifier. Block — `id`, `events`, and `hasMore` are produced by several operations but their descriptions named only one, unlike `folders` which already documented both meanings. Adds connector tests pinning all four listingCapped quadrants and the truncation signal, and tool tests for the list parser and the PATCH body's per-field "omit means unchanged" semantics. * fix(granola): clean up webhook endpoints created by a failed registration Raised independently by both reviewers. The registration service only rolls external state back when createSubscription *returns* — its rollback is guarded on `preparedProviderConfig`, so a handler that throws is assumed to have left nothing behind. Granola's handler broke that contract: when Granola accepted the POST but the success body was missing `id` or `signing_secret` (including a body that failed to parse and became `{}`), it threw with the endpoint already live. Nothing then recorded an external id, so undeploy could not remove it, and Granola kept delivering to a callback whose signature could never be verified — duplicating on every deploy retry. The handler now removes what it created before rethrowing, matching the pattern grain's multi-hook create already uses. It deletes by id when Granola returned one, and otherwise recovers the endpoint by matching the callback URL, which also covers a connection that fails after the request reached Granola. Endpoints whose URL was redacted to its origin are never matched — that comparison could delete another workflow's endpoint on the same host. Cleanup is best effort and never masks the original failure. A non-2xx is left alone, since no endpoint was created. Also folds the delete call shared with deleteSubscription into one helper. * fix(granola): never recover an orphaned endpoint by callback URL The previous commit's URL-based recovery was unsafe. A redeploy reuses the live registration's `path`, so the candidate and the currently serving endpoint share a callback URL — listing by that URL and deleting every match would remove the live deployment's endpoint and silently stop a working trigger, which is worse than the leak it was trying to prevent. Cleanup is now keyed solely on the id Granola returned. When the success body carries no id there is no way to tell the candidate's endpoint from the live one, so it is left in place: a leaked endpoint produces unverifiable deliveries that Granola disables on its own, whereas deleting the wrong one takes down live traffic with no signal. The 2xx-missing-signing-secret case this originally fixed still cleans up, since that response does carry an id. Adds a test asserting no lookup or delete is attempted when the response has no id, so URL matching cannot be reintroduced unnoticed.	2026-08-19 19:02:27 -07:00
Waleed	f17938c09e	feat(cbinsights): add CB Insights API v2 integration (#6879) ... * feat(cbinsights): add CB Insights API v2 integration Covers every non-streaming v2 endpoint across 25 tools: free organization lookup, firmographics search, funding rounds and cap tables, investments, portfolio exits, business relationships, management and board, the Mosaic / Commercial Maturity / Exit Probability outlooks and their histories, funding windows, revenue, strategy maps, Scouting Reports, ChatCBI, and RAG context. CB Insights authorizes by client-credential exchange rather than a static key, so the tools run through directExecution: the shared executor trades the credentials for a bearer token, caches it briefly, and re-authorizes once on a 401 — the token lifetime is undocumented, so expiry is discovered rather than predicted. ChatCBI and RAG declare request.modelInput so an activated Sim secret in the message is projected to its canonical label before reaching a third party's model. directExecution still runs projectToolModelInputParams, so the two are compatible. The two streaming endpoints are deliberately excluded; they deliver incremental JSON chunks and their non-streaming counterparts return the same content in one piece. * fix(cbinsights): reject malformed ID lists and bound the token cache - Reject an organization ID list containing an invalid entry instead of dropping it. Silently filtering meant a typo ran the request against a narrower set — spending credits on the wrong organizations, or quietly widening a filtered search — and still reported success. - Apply the same rule to the optional firmographics ID filters, where a dropped filter broadens the search rather than narrowing it. - Bound the process-wide token cache so a long-lived worker serving many CB Insights accounts does not grow with the cumulative number of accounts seen. Expired entries are swept on write, then the oldest evicted. * fix(cbinsights): stop paging and blank input bypassing the search guards - Measure the firmographics empty-search guard against the filters alone. limit, nextPageToken, and sort were in the same object, so a request carrying only paging slipped past it and issued an unfiltered search over the whole database — which still spends credits. - Reject a mistyped numeric bound instead of dropping it. A bad headcount, funding, or valuation filter silently widened the search, the same failure mode already fixed for ID lists. - Treat an empty comma segment identically on the required and optional paths. A trailing or doubled comma is a separator artifact that cannot change which records are requested, so both paths now discard it; every other malformed entry is still rejected. * fix(cbinsights): accept only plain decimal organization IDs Number reads "0x10" as 16 and "1e2" as 100, so either notation resolved to a real but unintended organization and the request spent credits on it. Both the path-scoped and the bulk validators now require a plain run of digits, and use Number.isSafeInteger so an ID past the precision limit cannot round to a neighbouring one. * fix(cbinsights): bound a numeric organization ID to the safe-integer range The string path already required a safe integer; the numeric path still used Number.isInteger, which accepts a value past the precision limit. JSON parsing has already rounded such a value, so the request would target a different organization than the caller supplied.	2026-08-19 18:31:14 -07:00
Waleed	a9cf760c0f	feat(pitchbook): add PitchBook integration (#6876)	2026-08-19 18:03:25 -07:00
Waleed	40aa8ad5eb	feat(crunchbase): add Crunchbase Data API integration (#6875) ... * feat(crunchbase): add Crunchbase Data API integration Covers the v4 Data API end to end: dedicated search and lookup operations for organizations, people, funding rounds, and acquisitions, plus generic collection-parameterized search and lookup reaching the remaining 39 collections, single-card paging, autocomplete, the deleted-entity feed, and fields metadata. Adds a crunchbase-errors extractor: the API answers failures with a bare JSON array, which no existing extractor reads, so an auth or predicate failure would have reported only its HTTP status. * fix(crunchbase): honor card paging limits and cursor exclusivity - Cap a card page at the documented 100-item maximum instead of Search's 1000, which the shared Limit field made easy to carry over - Always request the card's identifier so a narrowed cardFieldIds cannot return a full page with a null cursor and stall a paging loop - Reject the mutually-exclusive afterId/beforeId pair on the card and deleted-entity endpoints, not just on search - Report an unexpected card shape as empty rather than wrapping the envelope as a one-row page	2026-08-19 17:23:20 -07:00
Theodore Li	1372977d07	feat(setup): publish standalone self-hosting package (#6849) ... * feat(setup): publish standalone self-hosting package * fix(setup): refresh discovered compose installs * improvement(setup): unify repository command * fix(setup): harden standalone package launch * Update README.md * fix(setup): isolate standalone compose installs * fix(setup): restore default stopped installs	2026-08-19 20:04:05 -04:00
Waleed	4f9d5f33b0	improvement(search): search every folder, and document real API error bodies (#6861) ... * improvement(search): search every folder, and document real API error bodies Search on Files, Tables, and Knowledge was ANDed with the open folder, so a query only ever matched that folder's direct children — and the query was not cleared when you entered a folder, filtering the folder you just opened down to the same matches. A non-empty query now searches the whole workspace, a Location column names each result's folder, and opening a folder ends the search. Also gives GET /api/v2/files a `recursive` flag, and replaces the single shared OpenAPI error example — which showed `BAD_REQUEST` under every status tab — with one real body per status. * fix(search): discard the search term on clear instead of masking it `useSearchFilterValue` returned the debounced term whenever the input was non-empty, so clearing only hid the settled needle. The mask lifted on the next keystroke while the debounce still held the pre-clear term — opening a folder and typing within the window searched the whole workspace for the query the user had just abandoned. A clear now resets the settled term rather than hiding it, adjusted during render so the reset is visible to the render that follows the clear. The initial state is seeded from the first value so a deep-linked `?search=` still filters on the first render.	2026-08-19 13:55:53 -07:00
Vikhyath MondretiandClaude Opus 5	521348b529	feat(secrets): record which secrets each run resolves, and surface it per secret (#6823) ... * feat(secrets): record which secrets each run resolves and surface it per secret Redaction stops a value at a boundary but cannot stop code that never emits it — a Function block can print a key one character at a time and nothing ever matches the secret. That is undecidable in general, so this adds the other half of the posture: attribution. Every run now records which configured secrets it actually resolved, under whose identity, through which surface (workflow, Sim agent, MCP). The data already existed in ResolvedSecretTraceRegistry.addActiveEntry and was persisted only for paused runs; this persists it for every terminal path. Execution logs cannot answer this. They store the whole available encrypted environment rather than what a run referenced, they evidence a secret only where value-matching redaction happened to fire, and they expire under logRetentionHours — while "who has touched this key" outlives any single run. - secret_usage: per-UTC-day rollup keyed by workspace, secret, scope, owner, source, workflow, actor. A one-minute schedule touching three secrets would otherwise write thousands of rows a day, which is also why this is not audit_log. workflow_id/actor_user_id use '' sentinels rather than null so the unique key works on Postgres 14 without NULLS NOT DISTINCT, and are not FKs: they are historical facts, and an onDelete would rewrite a key column. - secret_owner_user_id is part of the key. Two people can hold a personal secret under one name and a shared personal secret resolves for a caller who does not own it, so name and scope alone do not identify a secret. It is NOT the actor: a scheduled run resolves the workflow owner's personal slice under the workspace's execution actor. - Direct environment reads are now detected in JS (TypeScript AST), Python (tokenizer-checked) and shell (quote/heredoc-scanned), so a secret read as environmentVariables['K'] or $K enters the run's provenance instead of going unredacted. Each detector prescans for names that are actually configured secrets before paying for a lex or quote-frame pass. - Copilot integration tool calls are covered: resolveCopilotEnvReferences substitutes {{SECRET}} into user-only params, which is a real use. - See usage lives behind a credential-admin gate, using the same predicate that reveals the value; members get a disabled chip explaining why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(audit): register the secret-usage route in the validation baseline Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): keep rollup metadata with its run, and stop shadowed bindings faking usage Review round 1. - record.ts: last_execution_id/last_trigger were assigned unconditionally while last_used_at was chosen by greatest(), so two runs completing out of order split one row between them — the newer run's timestamp beside the older run's execution id, making "View log" open a run the row does not describe. Both are now guarded on the timestamp actually advancing, so the row's metadata always belongs to the run that owns its timestamp. - javascript.ts: a local binding named environmentVariables (declaration, parameter, destructured binding, or bare reassignment) made reads off the user's own object look like mounted-secret reads. Any such binding now disables detection for the file; the AST already had parent pointers, so this is a kind check during the existing walk. - python.ts: same class of bug with no parser available, so the rule is an allowlist — every mention of the binding must be a literal subscript or .get(), otherwise detection is off for the file. This also subsumes the cross-line attribute case (other.\n environmentVariables['K']), which the previous space-and-tab look-behind missed. Under-reporting is the safe direction here: a trail that claims a use that never happened is worse than one that misses a use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(db): format the generated migration snapshot CI runs lint:check across every workspace; the drizzle-kit output in packages/db had never been through biome, so the branch was green locally (where lint had only been run inside apps/sim) and red on CI. Whitespace only — both files are byte-for-byte identical once parsed, and drizzle-kit still reports no pending schema diff against the reformatted snapshot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): detect every rebinding of the environment identifier, not just declarations Review round 2. A bare `for (environmentVariables of rows)` has no declaration to key off, so the previous check missed it and reads of the loop value were still recorded as secret usage. Rather than extend the hand-rolled node-kind list, this reuses the pair the same file already applies to reject a placeholder in a write position: isDeclarationIdentifier covers declarations, parameters, destructured bindings and imports, and isWriteIdentifier covers every assignment operator, ++/--, destructuring targets, and for-in / for-of initializers. That also closes four forms neither the review nor the original check named: logical (||=) and nullish (??=) assignment, and object and array destructuring assignment. Six of the eight added cases fail against the previous check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): apply the rebinding rule to shell, and say when a run's log is gone Review round 3, plus the docs that were left claiming the old behavior. - shell.ts: a script that writes a configured name (API_KEY=local, export/local/ readonly, read, for, unset) expands its own value from that point on, not the mounted secret, so recording it claimed a use that never happened. Every mention of the name must now be a `$NAME` / `${NAME}` expansion, matching the allowlist shape the Python detector already uses. Applied per name rather than per file: JavaScript and Python shadow one object holding every secret, whereas rebinding one shell variable says nothing about the rest. - The usage trail deliberately outlives execution logs, so a row routinely names a run whose log has been pruned. The read now left-joins workflow_execution_logs on its unique execution_id and reports availability, and the panel renders the chip disabled with the platform tooltip instead of linking into an empty Logs view. Three states: no run to link, a run whose log is gone, and a live link. - Docs said a direct environmentVariables/$KEY read does not activate masking, which this branch changes. Corrected in credentials.mdx, function.mdx and the logging FAQ, and the recognition limits are now written down: runtime-built names, reassigned bindings, and reads that cannot be told apart from text. Added a "See usage" section covering who can see it and why an empty trail means "nothing recognized" rather than "never used". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): writing a name is not reading it, and a bare mention is not a rebinding Review round 4. - javascript.ts / python.ts: `environmentVariables.API_KEY = 'x'` and `delete environmentVariables.API_KEY` touch the name without ever reading the mounted value, but the detectors matched the member access and recorded a use that never happened. JavaScript now asks the same isWriteIdentifier the placeholder rewriter uses (its parameter is widened to ts.Node — the body already walked generic nodes, so this is a type change, not a behaviour one) plus a delete check; Python excludes a subscript followed by `=` and a `del` target. - shell.ts: requiring every mention of a name to be an expansion also fired on text that binds nothing — a comment naming the key, or `echo "API_KEY=$API_KEY"` where the literal is an argument rather than an assignment — and dropping those cost masking on a genuine read. It now looks for actual writes: an assignment at command-word position, a binding builtin, `printf -v`, or a `for` target. The two directions are not symmetric, which is why this errs toward detecting the read: missing a write records a use of a secret the script only had in its environment, a misleading audit row and nothing more, since masking still searches for the real value and will not find it. Over-detecting a write suppresses masking on a value that does reach the log. This also makes the code match what the docs already described — skipping after a rebinding, not after any mention. 13 tests added; 11 fail against the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): an update reads before it stores, and a del target may be parenthesized Review round 5. The first of these is a regression from round 4. - javascript.ts: reusing isWriteIdentifier to answer "is this a read" was wrong. That predicate answers the rewriter's question — is this a target the substitution must refuse — so it treats every assignment operator alike, which is correct there and wrong here: `+=`, `||=`, `??=`, `++` and `--` all load the current value before storing, so they are genuine reads and were silently losing their masking. Only a plain `=` stores without reading. Replaced with a purpose-named predicate, and isWriteIdentifier's parameter is narrowed back to ts.Identifier now that nothing else needs it widened. A test committed last round asserted the wrong behaviour for `+=`; it has been corrected rather than left to pin the bug. - python.ts: `del (environmentVariables['K'])` slipped past a check that looked only at the characters immediately before the match. It now isolates the enclosing logical line and tests whether that is a del statement, which also covers `del((x))`, `del(x)`, `del a, x`, and a del after a semicolon. 12 tests added or corrected; 10 fail against the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): stop excluding Python writes, which kept leaking in the unsafe direction Review round 6. Greptile found that `del environmentVariables[environmentVariables['K']]` had its inner access — which computes a key, so it is a genuine read — skipped along with the delete, leaving that value unmasked. The narrow fix was another textual rule. Instead this removes the write and delete exclusions from the Python detector entirely, because they were optimizing the wrong direction. `resolvedSecretNames` feeds `outputSecretMatcher`, an exact-value matcher over the output. Naming a secret the code never read costs nothing there: the matcher scans for a value that does not appear. Failing to name one that was read leaves it unmasked. The two error directions are therefore not comparable, and the exclusions bought only audit-trail tidiness while every heuristic they needed has so far leaked into the dangerous side — first a parenthesized target, now a nested read. A `del` or an assignment is reported like any other access. JavaScript keeps its exclusion: a real AST answers the question per node, with no text to misread, and it has produced no such hole. Net 30 lines removed from python.ts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): report recognized reads instead of proving they are not reads Review round 7. Greptile flagged both directions at once — false usage from reporting a write target, and unmasked secrets from the file-wide shadow flag — so I traced what the signal actually drives before choosing. The chain: the compiler's names feed outputSecretPlaintextsByName and the exact-value matcher, NOT context.resolvedSecretNames, which starts empty. After execution activateOutputSecretProvenance scans the output and adds only names whose plaintext actually appeared; those become __resolvedSecretNames, which tools/index.ts turns into recordResolved calls, which is what the usage trail reads. So a compile-time false positive produces no usage row on the ordinary path — it only hands the matcher a value the code never emits. It does produce one on the !projection.safe fallback, where the system already over-approximates by design. A false negative, by contrast, keeps the value out of the matcher entirely, so a genuinely read secret is never masked on any path. That asymmetry decides it, so every "prove this is not a read" mechanism is gone: - javascript.ts: the file-wide shadow flag. A helper declaring its own environmentVariables discarded genuine reads of the mounted binding everywhere else in the file — Greptile's security finding, and real. - python.ts: the allowlist requiring every mention to be a subscript or .get(). Same hole: passing the dict to a function suppressed unrelated reads. - shell.ts: the rebinding check. It had the same hole in a form nobody flagged — `echo "$API_KEY"; API_KEY=local` dropped the first read, which is of the real secret. What stays is the question of whether the text is code at all — strings, comments, single quotes, quoted heredocs — plus the receiver check that `other.environment Variables['K']` is a different object, and JavaScript's node-precise write/delete exclusion, which cannot suppress a read elsewhere. Net 215 lines removed across the three detectors and their tests. Docs updated: the rule is now stated as reporting rather than proving, and that See usage may occasionally list a secret the code had available but did not read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(secrets): drop the last write-vs-read special case `environmentVariables` is a plain object deserialized from the run payload (route.ts:206), not a handle on the stored secret. Assigning to it changes nothing outside the sandbox and is discarded when the run ends, so separating a write from a read bought almost nothing while leaving JavaScript as the one language still trying to prove a read is not a read. Every language now follows the same rule: report a recognized read of a configured secret name. The only exclusions left are facts rather than inferences — the text is not executable (string, comment, single quote, quoted heredoc), the receiver is a different object, or the name is not statically knowable. Docs note that assigning to the binding does not edit the secret. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(secrets): ship only the fields the trail actually shows Five fields crossed the API and reached no reader: usageDate, firstUsedAt, actorEmail, workflowId and actorUserId. The panel renders the timestamp, the trigger, what used the secret, the actor's name, the run count and the run link; everything else was projected, serialized and discarded. first_used_at is dropped from the table as well. Nothing read it, and inside a per-day bucket "first used that day" says nothing next to "last used that day" — so it was a column written on every run for no question anyone asks. The upsert loses its least() with it. Migration regenerated; the identifier columns behind the joins stay, they simply are not returned. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): report referenced code secrets, not only ones that surface in output The Function route activated a secret's provenance — and therefore its usage row and downstream masking — only when the exact value appeared in the result, stdout, or error. That gate made the trail miss silent use entirely: a key that authenticates an API call and is never echoed reported nothing, and so did the founding scenario of this feature, a key exfiltrated character by character. The innocent run that echoed a key got a row; the run worth catching did not. Activation now follows the referenced set the compiler already computes: resolved {{KEY}} bindings plus recognized direct reads, filtered to configured values — the same set the unsafe-projection fallback already activated. An extra name only hands the output matcher a value that never appears; configured-but-unreferenced values are still never included. The output-scan activation path and its surface helper are deleted rather than kept alongside. One old test pinned the gate ("does not activate a referenced secret that does not cross the Function result"); it now asserts the reverse, with the reasoning attached. Two new tests pin the char-split exfiltration and the silent API-call case. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): shell escaping is backslash parity, not adjacency Review round 8. `\\$API_KEY` is an escaped backslash followed by a LIVE expansion — bash prints `\` plus the value — while `\$API_KEY` is an escaped dollar and stays literal. Checking only the character adjacent to `$` read every even run as escaped, dropping a real read from usage and masking alike; verified against bash before fixing. The scanner now counts the run of backslashes before the `$` and skips only odd runs, the same parity rule logicalLineEndAfterContinuations in this file already applies to line continuations. Six-case parity table added; the three even-run cases fail against the previous check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): recognize destructured environment reads Review round 9. `const { API_KEY } = environmentVariables` delivers the value by name with no property- or element-access node in the AST, so the member-access walk missed it entirely — and a missed read leaves an emitted value unmasked, the dangerous direction. The AST walk now also recognizes the declaration form (shorthand, renames, defaults, string-literal keys), the assignment form ({ KEY } = env), and a ...rest element — which names no key but takes every value, so it reports every configured name; the alternative left `const { ...all } = env; return all` entirely unmasked. A computed key stays unrecognized, the same runtime-name boundary as a computed subscript, and a receiver that is not the bare identifier is not attributed. Nine cases added; the six positive ones fail against the previous walk. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): one receiver rule for destructured reads, parentheses included Review round 10. Two accurate findings, folded into a generalization instead of two more special cases: - A parameter default (function f({ API_KEY } = environmentVariables)) and a binding-element default are the same by-name delivery as a variable declaration. The detector now keys on the ObjectBindingPattern itself and checks its parent's initializer, so every declaration position follows one rule instead of per-kind arms. - Parentheses group without changing the receiver, so (environmentVariables) is unwrapped before the identifier check — in the destructuring arm AND the member-access arm, which had the same hole unreported. Declined the for-of-over-array-literal finding: the receiver there is a container, not the environment object, and following data flow through containers has no fixed point — the same documented boundary as aliasing and computed keys. A test pins the boundary so it reads as chosen, not missed. Eight cases added; the seven receiver-rule cases fail against the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(secrets): a dot in prose is not a qualifier, and a literal computed key is a subscript Review round 11. Both findings were implementation-narrower-than-rule, fixed by consulting authorities the detectors already had rather than adding new ones: - python.ts: the receiver walk crosses whitespace so a parenthesized `other.` on a previous line is seen — but it landed on a comment's final period (`# Load the value.`) and discarded the genuine read on the next line. The landing position is now checked against the same lexer ranges that filter the candidates, which is also why the receiver check moves after lexing. - javascript.ts: `const { ['API_KEY']: key } = environmentVariables` is the element-access rule in pattern position, so a computed key holding a string literal resolves like a literal subscript; any other computed key keeps the runtime-name boundary a computed subscript already has. Eight cases added; the comment-period case and all three literal-computed-key cases fail against the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-18 18:32:20 -07:00
Vikhyath MondretiandClaude Opus 5	3a03774e42	fix(forks): stop copying connector-managed knowledge base documents (#6818) ... * fix(forks): stop copying connector-managed knowledge base documents A fork copies a KB's documents but never its connectors, so a connector-sourced document arrives with `connector_id` nulled and its `external_id` intact. The sync engine keys every existing/tombstone/ exclusion lookup off `connector_id`, so that copy is invisible to it - never updated, reconciled, or purged - and `doc_connector_external_id_idx` does not constrain it either, since its `connector_id` is NULL. Attaching a connector in the child then re-ingests every page as a NEW row on top of the snapshot. Each fork hop re-copies the previous hop's orphans and adds one more generation, so a prod -> UAT -> staging chain leaves three rows per page and a knowledge search returns the same page three times, one of them serving content frozen at the fork date. Exclude connector-managed documents from all four doors a document can enter a fork through: the whole-KB content copy, the in-transaction placeholder pre-creation, the sync-only copy into an already-mapped KB, and the content fill (guarded for payloads planned by a pre-change worker mid-rollout). The placeholder path matters as much as the copy loop - filtering only the content phase would leave a permanently archived row behind a persisted `knowledge_document` mapping. Skipped on both sides, the reference clears like any other uncopied document's. A document whose connector was deleted already has a null `connector_id` (the FK is ON DELETE SET NULL) and is static in the source too, so it still copies. One count(*) per copied KB logs what was left behind, since a fully connector-synced KB now forks to zero documents. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): keep the skipped-document count from failing a copied KB The connector-managed count feeds a log line, but it sat inside the KB's try block, so a transient failure on a COUNT(*) would roll back a copy that had otherwise succeeded and clear every reference to it. Move it into a helper that swallows its own error. Counting is not copying: only the copy itself may fail a resource. Test proven red by removing the catch - the mutation reports a knowledge-base failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): clean up full-KB placeholders planned before the exclusion The mapped-KB fill guarded a pre-change plan, but the full-KB path did not: a placeholder planned by an old worker for a connector-managed document is simply no longer returned by the page query, so nothing fills it and it stays archived behind a live mapping that a remapped document-selector still resolves to. Report those child ids as failed documents so the shared cleanup clears their references and drops the rows, and delete their persisted identity so a later sync does not resolve to a row cleanup removes. Keyed on the SOURCE being connector-managed, which can never become copyable, so it cannot race a concurrent attempt mid-fill the way a "source is gone" check could. The mapping drop is now one helper shared with the mapped-KB catch. Test proven red by removing the reconciliation block. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): make the stale-plan probe best-effort The probe ran inside the KB try, so a transient SELECT would reach the catch, roll back a complete copy, delete the child base, and clear every reference to it. Weighing it as "load-bearing, so fail closed" was wrong: the probe runs on EVERY copied KB that has referenced documents, while the state it repairs exists only inside a rollout window. Failing closed traded a common-path outage against a rare-squared one. It now swallows its own failure with a loud error log, leaving that pre-existing state in place rather than destroying a good copy. Test proven red by removing the catch - the mutation reports the KB failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-18 14:23:19 -07:00
Waleed	1c69372cba	feat(cli): follow a run, wait for one, and tail the log (#6813) ... * feat(cli): follow a run, wait for one, and tail the log Three commands the surface was missing, each polling or streaming something the generated command layer cannot express. `workflows run --follow` renders the SSE the execute route already emits, so a multi-minute agent run stops printing nothing until it ends. It rides on the generated `run` leaf rather than a sibling command — same operation, one different response encoding — and delegates to the handler it replaced, so every non-follow invocation still runs the generated path. Answer text, thinking and tool calls go to stderr; only the final envelope reaches stdout, so redirecting still yields the result. Reasoning and tool frames need the `X-Sim-Stream-Protocol` header, which is sent only when asked for, because negotiating also switches answer text to live chunks the server may retract. `workflows runs wait` closes the loop `--async` opens. Terminal is completed, failed or cancelled; `redacting` is not, since a run whose output is still being scrubbed is not yet a run you can read. A time pause keeps polling because the server resumes it, and a human pause stops with the resume command rather than burning the bound and calling it a timeout. Distinct exit codes keep cancelled and paused from reading as failure. The bound is `--wait-timeout` and not `--timeout`, because SIM_TIMEOUT_SECONDS already bounds one request and two knobs of the same name hide each other. `logs follow` tails runs as they arrive. Dedup keys on run id, not on the timestamp: a schedule fan-out starts many runs in the same millisecond, so a timestamp watermark either drops the siblings or reprints them. JSON output is one object per line, because a follow never closes an array, and the table header is printed once so columns stay aligned across polls. * fix(cli): disclose a truncated burst, and clear a stale retry notice Two review findings in `logs follow`, both verified against the code first. The page budget bounds one poll so an enormous burst cannot stall the follow, but on reaching it the live cursor was discarded: the remainder is older than everything collected and the next poll restarts at the newest page, so those runs were never printed and nothing said so. The budget stays — draining without one trades a bounded poll for unbounded buffering in a process meant to run for hours — but hitting it now warns on stderr, naming the count and pointing at `sim logs list`. That notice is written even off a terminal, because a piped log is where an unexplained hole is hardest to spot. The retry notice was cleared after the empty-rows check, so a poll that recovered but found nothing left "retrying in Ns…" on screen while the follow was already healthy. Clearing now happens as soon as a poll succeeds. The second test needed two failures to be worth anything: the teardown clears the line either way, so what separates fixed from broken is whether a bare erase lands before the second notice or only at the end. The first version passed against the bug. * test(cli): pin that a mixed page is the watermark, not a truncation A page holding a run already printed proves the follow caught up, so the truncation warning must not fire there — that is how every healthy poll terminates, and warning would report a hole on the ordinary path. The straggler sharing that page is still collected, because the filter takes every unprinted row on it rather than only those above the known one. * fix(cli): say when the requested backlog was larger than a page holds The logs API clamps `limit` into 1–1000 rather than rejecting it, so `logs follow -n 5000` came back with 1000 rows, anchored the floor to that partial page, and said nothing. The seed already knew — it computes whether a live cursor remained — but the caller discarded the answer. Guarded on both halves. Fewer rows than asked for is only a shortfall when more were waiting: a workspace holding ten runs answers `-n 50` with ten and nothing is missing, so warning on the row count alone would fire on every small workspace. The cursor is what separates the two.	2026-08-18 11:43:16 -07:00