mirror of
https://github.com/simstudioai/sim.git
synced 2026-10-30 19:32:46 +08:00
1465e94574073d5e1e9b649bb897056aa58baf00
287
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Waleed
|
1465e94574 |
feat(tinyfish): add TinyFish web agent, search, and fetch integration (#7177)
* feat(tinyfish): add TinyFish web agent, search, and fetch integration Adds the TinyFish integration: eight tools across the Agent, Search, and Fetch APIs, a block wiring them, the brand icon, and hosted-key support. Agent runs are metered on the step count TinyFish reports. Search and Fetch are free, so their hosted key costs nothing to run. The async run and its run read/cancel/list companions carry no hosting config — their charge accrues after the request returns and cannot be metered — so they always require the caller's own key. * fix(tinyfish): address review findings - Add tinyfish to PROVIDER_SECTIONS. The sectioned BYOK renderer drops any provider missing from a section, so the key field never rendered. Export PROVIDERS/PROVIDER_SECTIONS and assert they agree, so the next provider to miss a section fails CI instead of vanishing. - Replace the `any` payload params with raw snake_case wire types. This caught two real gaps: `status` could reach the output undefined, and `error` could be null where ToolResponse.error is `string | undefined`. - Reject an already-parsed array output schema, which a `json` block input can produce, and name malformed schema JSON instead of leaking a SyntaxError. - Count Fetch rate-limit usage from the submitted URLs rather than the returned arrays, so a URL in neither array cannot undercount. - Make "List runs" a literal canvas clause; a blank goal filter lists everything. - Regenerate the docs manifest for the new integration page. * chore(byok): audit that every hosted provider is fully wired Adds check:byok-providers. A hosted tool names its provider once in hosting.byokProviderId, but the id must also reach the zod enum, the settings PROVIDERS row, and a PROVIDER_SECTIONS section. Only the union is compiler- enforced; the rest fail silently, which is how TinyFish shipped with no settings row in the first place. Also flags drift between the two BYOKProviderId declarations. Correct the Search and Fetch rate-limit rationale: TinyFish documents both ceilings per API key, not per account, so the numbered key pool does raise the total. Records why a free product is still worth hosting — the endpoints require a key, so hosting is what removes the signup. |
||
|
Vikhyath MondretiandClaude Fable 5
|
5112b0711d |
improvement(db): stop declaring retired usage columns ahead of their drop (#7134)
* improvement(db): stop declaring retired usage columns ahead of their drop Contract-prep for the legacy usage-column drop. Removes the drizzle declarations for 19 retired user_stats counters, organization.departed_member_usage, and workflow_execution_logs.cost, so generated SELECT lists stop referencing them, plus the last incidental references: the cycle-close departed_member_usage resets, the v1 admin exposure of departedMemberUsage/currentPeriodCopilotCost (both serving dead values since the ledger cutover), the testing schema mirror, and a lock-order fixture. The DROP COLUMN migration ships separately once this is the deployed production app; contract-pending markers in schema.ts track it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(db): reserve the drop migration as a journaled no-op for schema parity CI requires drizzle-kit generate to be a no-op on every PR, so the declaration removal cannot ship without its migration. Committing the generated 0309 with every DROP commented out (a SELECT 1 no-op) keeps the parity gate and future generates clean while deferring the destructive half until this change is part of an already-deployed production release; the file documents the follow-up procedure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * improvement(db): keep declarations, route reads through live-column maps Restructures the contract prep per review: the no-op 0309 migration is gone and schema.ts keeps the deprecated declarations (with the contract-pending markers on the columns), so the follow-up PR ships the declaration removal and the real generated DROP migration together. What makes that safe is that no deployed query names the doomed columns anymore: every argless select()/returning()/relational read of user_stats, organization, and workflow_execution_logs (29 sites) now goes through omit(getTableColumns(...)) live-column maps, following the workspaceFileColumns precedent. scripts/check-pending-drop-tables.ts enforces this — it derives the guarded tables from the contract-pending markers themselves (picking up workspace_files' pending size drop too) and fails CI on any argless read, so the invariant holds between the two PRs and retires itself when the markers are deleted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): backfill residual cost_total projections before the cost drop The 0220 procedure projected every then-existing legacy cost json into cost_total, but a transition-window writer added 23 rows (all 2026-05-30, verified on the prod replica) carrying a numeric json total with no projection. Script migration 0009 re-runs 0220's exact candidate filter and projection in bounded batches at deploy time, so the pending cost DROP abandons nothing cost_total should hold. The contract PR that drops the column must deregister the script in the same change — it reads the column; the contract-pending marker says so. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): validate live-column builders and inline aliases in the pending-drop audit Hardens check-pending-drop-tables per review: the audit now derives each guarded table's doomed columns from schema.ts (@deprecated tags and column-level contract-pending markers), so a sanctioned omit(getTableColumns(t), [...]) or destructure-omit must name every doomed column away — an unrelated or incomplete key list is reported with the columns it misses, and schema.ts itself is no longer skipped, so the <table>Columns helpers fail the audit if a later deprecation is not added to their omit lists. Inline alias(t, ...) expressions passed directly to from()/getTableColumns()/insert/update/delete now resolve to their canonical table instead of escaping the checks. The testing mirror's *Columns aliases now omit the same doomed keys as the real helpers (and gain the missing proPeriodCostSnapshotAt entry), so tests cannot keep passing against doomed-column references the app no longer serves. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): derive pending-drop columns from the AST, and align the mock schema The doomed-column derivation matched `@deprecated` with a line regex, so a column documented with a multiline TSDoc block was never marked doomed and the live-column helpers were free to omit nothing for it. Derivation now runs off the parsed schema: guarded tables are pgTable calls whose enclosed comments carry a drop-flavored contract-pending marker, and doomed columns come from each property's own leading TSDoc block, so single-line and multiline forms read alike. Verified by mutation: making a deprecated column's TSDoc multiline and dropping it from the helper's omit list now fails with the column named, where the previous version reported clean. Also fills the seven live columns the testing mirror was missing (user_stats.limit_notifications, four organization settings columns, workflow_execution_logs.cost_total/models_used) so mock-derived selections have the same shape as production ones. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): resolve namespace and renamed imports in the pending-drop audit Table references were matched by bare identifier only, so an argless read reached through `import * as schema` — a live pattern here, e.g. `db.insert(schema.userStats)` in lib/auth/anonymous.ts — or through a renamed import was invisible to the audit. Resolution now runs over the file's schema-module import bindings (named, renamed, and namespace) as well as alias() bindings, at every position the audit inspects. Verified by probe: argless select() via `schema.userStats`, via a renamed import, and an argless .returning() on `db.insert(schema.userStats)` are all reported now, where the previous version reported none of the three. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): close the pending-drop resolver under variable assignment A table parked in an ordinary binding (`const t = userStats`) escaped the audit, since only alias() results were re-bound. Binding collection now resolves any declarator whose right side names a table — the table itself, a namespace member, an earlier binding, or an alias() call — and repeats to a fixpoint so declaration order does not matter. Verified by probe: argless reads through a direct assignment, a namespace-member assignment, and a chained binding are all reported now, where the previous version reported none; the repo stays clean, including a file where a local array shadows a pending table's name. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): match relational reads by table name, not by a literal query chain An argless findFirst/findMany reached through a binding that holds db.query escaped the audit, because the check required the receiver to be a literal `<x>.query` member. Drizzle's relational API is keyed by table name however the root is reached, so the check now keys on the table-named property itself, closing the whole class rather than the one aliasing form. Verified by probe: an aliased root (`const q = db.query`) is reported now where it previously was not, `tx.query` still is, a read that passes `columns` still is not, and the repo stays clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Waleed
|
6bbabbbac6 |
fix(cli): close follow-up gaps (#7137)
* chore: run the orphaned migration-safety test, and stop publishing real-looking ids One of the eight script tests was reachable from no entrypoint, so it had never run in CI — it passes, it was simply never invoked. This is the second time that hand-maintained list has drifted from the files beside it; the audit runner's own header records the first. The guard against a third is a `check:*` script rather than a test, because the runner derives its list from that namespace and so picks the guard up by name — a test would have had to be hand-added to the very list it guards. A published spec sat outside the generator's manifest and so outside its drift check, and carried six example ids with the texture of real generated ones rather than the pandigital placeholders the rest of the repo uses. The new check globs the directory instead of reading the manifest, since the manifest is what omitted the file. The one pre-existing borderline id is allowlisted with a reason: loosening the threshold to admit it would have hidden one of the six. * fix(cli): close the gaps black-box testing the shipped CLI found `sim profiles <anything>` still exited 0, so a probe reading the exit code to ask whether a command exists was told yes — the one group the earlier guard missed. The exemption was written for commands that are both a group and a leaf, but only `files restore` takes an operand; `profiles` takes none. Registering its listing as a default subcommand puts it back among the pure dispatchers the existing guard already covers, so the guard itself did not need widening. Three commands refuse a workspace API key and said nothing, while their menu siblings said so — reading as though they accept one. They are hand-written, so they never reached the code that appends the note. That note now comes from a helper taking the operation, so a command names the operation it invokes and the two cannot disagree, and a test fails if a hand-written command ever calls a restricted operation without it. A blank numeric value in a request body still became a real zero, the same coercion already fixed for query strings: the guard keyed off the slot when the distinction is the field's declared type. Twenty-one fields across fifteen operations were affected. An empty body string still clears a description. Blank values for the root endpoint, workspace and profile flags fell back to what was configured instead of being refused, and a whitespace workspace was accepted verbatim. A hand-written profile name carrying padding listed as reachable but resolved to defaults rather than erroring. Two schema descriptions named request fields that no flag spells, and a rejected value was echoed unredacted by four messages while their siblings redacted it. A write now re-emits a section header it was not asked to touch byte for byte. The blank-line normalisation around it is left alone: making the writer position-faithful is a change to its model, not a fix. * fix(scripts): match example uuids case-insensitively in the spec audit The pattern only recognised lowercase hex, so an uppercase id in a published spec was never examined and the audit reported success without having looked at it. Matching case-insensitively is not enough on its own: hex is case-insensitive, so a mixed-case id counts `A` and `a` as two digits and reports twenty distinct ones rather than sixteen. That inflated count clears the threshold the texture test uses to recognise a hand-authored placeholder, so a real id could have passed for one. The allowlist is an exact-string lookup and would likewise have missed an uppercase spelling of an entry. Both checks and the lookup now take a normalised id, while the finding still reports the spelling as it appears in the file. * fix(cli): stop a refusal being swallowed, and gate example ids by name A blank root flag was refused everywhere except `profiles`, where the catch that lets a broken profile still list absorbed it and the command exited 0 after printing the table. The refusal now carries its own error class, which is what the listing rethrows on — the two are distinguished by type rather than by matching message text, and a genuinely broken profile still lists. The unknown-profile message redacted the name the caller typed but not the suggestion or the list of configured names beside it, which come from the same file and are equally attacker-influenced once it has been hand-edited. Those are redacted now, as is every other message in these two files that quotes a name read out of the config, and the profile listing flattens the names it renders the way it already flattened the error column. The example-id audit judged a uuid by its digit texture, on the premise that a real one essentially never looks hand-authored. Measured against ten million generated ids, 0.81% of them do — one in 124, where this change alone replaced six. Requiring each digit exactly twice takes that to zero but rejects all fourteen placeholders now in the specs, so it is no cheaper than the alternative. The audit now holds the eighteen ids the specs actually use, which is one file rather than the twenty-seven a reserved format would touch, and a new id fails until someone lists it — which is the review the check exists to force. * fix(scripts): match the uuid sentinels exactly rather than by shape Accepting any id built from at most two distinct hex digits let something through that was never on the approved list. A generated id essentially never has that shape, so the practical risk was small — but this check had just stopped being a shape test and become a list, and a structural exception is the one thing that undoes that. The two ids it exists for are the nil and max sentinels, and both are matched by value now. |
||
|
Theodore Li
|
af7a12c23e | improvement(cli): automate npm package version bumps (#7127) | ||
|
Waleed
|
c930830310 |
fix(v2,cli): second audit pass over the v2 API and CLI (#7126)
* fix(chat): resolve a caller-supplied conversation id through its owner The v2 chat route used the caller-supplied conversationId verbatim, with no existence, owner, or workspace check, against a store keyed by bare text with no owner column. A caller who knew another user's conversation id reached that conversation. Ids now resolve through the same owner-scoped loader the web chat path uses, and anything unresolvable answers one uniform 404 before any lifecycle work runs. Omitting the id mints a server-issued conversation. The contract also accepted any 1-128 character string for a column typed uuid, so a malformed id raised a driver error and rendered 500 while an unknown but well-formed id rendered 404 - a shape oracle, and a 500 on ordinary input. The ownership predicate had no coverage anywhere: the route test mocked the module and the lifecycle test drove a chain mock that ignores its where clause, so deleting the owner condition left both suites green. It is now asserted by composition and by condition count, which is what catches a dropped condition. Also renames the reply's model identifier away from a term the project's own copy rules forbid on a user-facing surface. * fix(v2): conceal workspace absence, and stop archived tables faulting their page Two reads answered a caller more than they were entitled to know. A workspace a caller cannot reach at all returned FORBIDDEN while one that does not exist returned NOT_FOUND, so a workspace-key holder could enumerate which workspace ids exist by diffing the two. Both now answer the same absence, using the concealment policy the billing routes already use. A refusal from inside the workspace - a member whose role is too low - still answers FORBIDDEN, because that caller already knows the workspace exists. Separately, archiving a folder cascades onto its tables but leaves each table pointing at the archived folder row. The archived listing resolved those paths strictly, so one such row faulted the whole page and no cursor could step past it - which also made the ids undiscoverable and left restore unreachable for exactly the tables that need it. The archived scope now resolves leniently to the root, where a restore would place them, matching the shipped workflows behavior. Active listings still fault loudly on a dangling folder. * fix(knowledge): validate upload processing options without stranding live sessions recipe and lang were accepted as free strings up to their length caps, silently discarded, and echoed back nowhere, so a typo was unobservable: uploading with a misspelled recipe returned 200 and quietly used the default. Both are now validated at the boundary and a bad value answers 400 naming what is accepted. The accepted recipe set deliberately includes the sentinel every first-party caller sends today alongside the three real chunker recipes, and the three are derived from the chunker's own union so removing one there is a compile error here rather than a silent 400 in production. The same schema also parses metadata read back off a persisted upload session, so tightening it would have thrown out of resume and complete for any session created before this - a 500 on work that could then never finish. The read-back path now drops a value it no longer recognises instead of rejecting it; the request boundary stays strict. Neither field reaches chunking, so nothing here moves chunk boundaries, embeddings, or search results. * fix(v2): honour a requested stats window, and answer a claimed graph id with a conflict Log statistics accepted a start and an end, filtered the totals by them, and then built the series against wall-clock now. Bucket width was computed over a span the caller never asked for, and every bucket past the requested end was structurally empty - so a bounded historical query returned a wrong-width series with fabricated trailing buckets, under a window label that disagreed with the request. Each edge now honours the bound it was given and keeps its previous derivation when omitted, so an unbounded request is unchanged. Separately, block, edge and subflow ids are global primary keys while the delete that precedes a state replace is scoped to one workflow. An id owned by another workflow survived that delete, the insert violated the key, and because callers pass their own transaction the driver error escaped unclassified as a server fault. The write now refuses such an id up front with a conflict naming it, and re-classifies the same violation if one races past the check, since the lock covers only the workflow being written. The dry run checks the ids a commit would insert and reports the warnings a commit would report, which is what its own contract already promised. * fix(secrets): let a workspace secret change its metadata without resending the value Restoring redaction cost more than removing it. The only way to flip a secret back to redacted was to re-send the plaintext, because the write required a value and omitting it fell into an interactive prompt that cannot run in CI. A workspace secret can now change its description or visibility on its own; the stored value is never re-encrypted or rewritten, a write that names no existing secret answers not-found rather than creating one, and a personal secret still requires a value because it has no other writable field. The path parameter was also one shared schema across the write and the delete, so a single description had to cover both and the delete documented an argument that could create and replace. Split, mirroring the credentials pair. The metadata write is a new update against the credentials table, so its scope is asserted by composition and by condition count: an unscoped update would let one workspace flip another workspace's identically-named secret out of redaction, and the cache invalidation would then carry that flag into the other workspace's runtime catalog. * fix(v2): say what an error means in terms the caller can act on A size-limit refusal collapsed every value under a kilobyte to "0 Bytes", so a 28-byte file over a 27-byte ceiling read "is 0 Bytes, above the 0 Bytes limit" - self-contradictory, and useless for choosing a value that would work. Errors and field descriptions also told callers to invoke raw HTTP endpoints. These strings serve the REST reference and the CLI's own help equally, so they now name the operation and its object rather than a method and a path. A sweep test walks every v2 schema description and holds the line, with the remaining offenders in files this change does not own recorded explicitly rather than left to be rediscovered. Listing the editors of a built-in skill claimed the skill did not exist, while reading the same id succeeded - a well-formed request for a real resource is not malformed, so the list answers an empty roster and only the mutations refuse. Bulk folder deletion recorded only the leaf name in its audit trail while the single delete recorded the full path, leaving two same-named folders under different parents indistinguishable after the fact. Bulk chunk enable, disable and delete each treated an unmatched id differently behind one sentence of documentation. They now follow one rule. A workspace-scoped list refused with the name of a resource the caller never addressed, which reads as an empty workspace rather than an unreachable one. * fix(cli): stop a config value forging a section it was never meant to write The config file is written by joining names and values into INI lines, and nothing checked what was in them. A profile name carrying a newline and a section header wrote a section that merged into a different profile and took over its endpoint - and the next command sent that profile's stored API key there. A workspace value could do the same from the other side, since only the endpoint flag validated its input. The refusal now lives at the writer, the single place untrusted text enters the document, with the flag-level checks kept for the better message. Either alone blocks the forgery; the pair is deliberate. Rejecting rather than escaping, because the format has no escape syntax and these files are hand-edited and read by other tools that would not decode one we invented. The forbidden set covers control characters and the two Unicode line separators, which the previous guard missed - those parse as an unreadable line, so the key silently vanished on read and the next write appended a duplicate while the command reported success. Login also wrote the key before the settings, so a malformed response from the deployment could leave a key on disk with no endpoint beside it, and the next command would send it to the default host. Settings are written first, and the response is checked before anything touches disk. Name validation applies only when creating a profile, so a hand-written one that predates the rule keeps working. * fix(docs): tell the reader which key a command needs, and stop the ids contradicting the CLI Around sixty v2 operations refuse a workspace API key, and the CLI's help said nothing about it - the caller found out from a 403 after the request went out. The restriction is already stated in the API spec, so the generator now reads it from there and the command description carries it. The sentinel sentences are imported from the spec's own constants rather than copied, so a reword cannot silently unmark every command, and the test pins the count as well as named operations because a reword confined to one family would otherwise slip past. The generated reference also rendered an empty default as a sentence pointing at nothing - "Defaults to ." - for every repeatable filter. Omitted now, while false and zero still render, which is the trap that shape of check usually walks into. The hand-written guides used a workflow-shaped id for workflows that the CLI's own help says never names one, and five other families were equally wrong. All of them now match the scheme the CLI declares, consistently per entity across pages, with the shared ones taken from that help text so the two read as one voice. The page documenting every flag was linked from nowhere; both landing links pointed at the overview instead. And the generator's test file was absent from the hand-maintained list CI runs, so its guards never executed. * fix(cli): stop a page-size default capping a destructive filter Every request field named limit inherited the pager's default of 100, but only a cursor-paginated command interprets that flag. The two filter-based row mutations declare no cursor, so the default went onto the wire as a row cap: a filter matching 250 rows deleted 100, exited 0, and said nothing - while the confirmation the user had just answered promised every matching row. The flag's own help offered 0 for everything, which those endpoints reject; the unbounded form is the field being absent. The pager's default now applies only where the pager runs, and the tests pin the omission on the request body rather than in help text. A cap typed alongside an explicit row list was silently ignored; it is now refused on the client, where refusing costs nothing to already-installed versions. Lists also truncated at a hundred with no signal in any format, and the two inventory endpoints that do report truncation had that field dropped on the way out - so a caller reconciling against a clipped list could not tell. One note now goes to stderr while stdout stays a bare array, and a flag raised on a later page survives the fold. Also: a folder whose name contains the separator no longer prints a path that resolves to a different folder; validation errors name the flag the user typed instead of the wire field; an unknown subcommand with --help exits non-zero instead of printing the parent's help; a fractional or negative page size is refused rather than floored; an empty query filter is refused rather than silently returning everything; and the two spellings of the missing-workspace message became one. * fix(cli): gate destructive table imports and fix follow-mode rendering A `tables import --mode replace` empties the table before its first batch, so the only warning was in the describe. It now confirms, and the wording tells the truth per mode: cancelling a replace leaves a prefix of the new file with the originals already gone, while an append re-adds its rows if the file is imported twice. `--yes` skips the gate, and the gate runs before the file is opened. Import and export cancellation carried no describe at all; the import one now confirms, the export one records why it deliberately does not. Follow-mode output truncated cells to whatever the first row happened to measure, so a longer status or workflow name arrived clipped with no signal. Cells now clamp at a shared ceiling and pad to the lock, and the log columns carry width floors so a short first page cannot pin a column narrower than its own values. Interrupting a staged download left the staging directory behind; it is now removed on SIGINT and SIGTERM before the signal is re-raised. `--select-output` without `--follow` selected from a response that does not carry outputs, and said nothing. It is refused client-side, with a separate message for `--async`. Its describe now names what the path addresses. `secrets set` always read a value, even when only metadata flags were passed. Off a TTY that was an immediate refusal, so a metadata-only edit exited 1 in CI for a value it was never asked for; on a TTY it stopped to prompt, and the prompt rejects an empty entry, so there was no way to say "leave the stored value alone" short of re-typing the secret. The read is now skipped and the field omitted, which is what lets a metadata-only edit run unattended. On a TTY, setting only a description no longer prompts. Passing both spellings of the reveal flag is refused rather than silently resolved. Four mandatory hand-authored flags now say so, `billing logs` names its key-type scope, and the dispatch list declares its columns. * fix: close the gaps an adversarial review of this branch found A conflict handler added earlier in this branch was dead code. It read the Postgres error code off the thrown object, but the driver error arrives wrapped with the real one on `cause`, so the check returned false on its first line and the 409 never fired. Its test passed only because it threw a flat shape production never produces. It now reads through the cause chain with the shared helpers, compares the constraint name exactly instead of matching a substring of the SQL, and its test throws the real wrapped error. Resuming a conversation checked its workflow and its workspace but not its type, so a conversation created by the web surface could be continued as a CLI turn. It now refuses through the same uniform 404 as every other mismatch, which closes the same omission on the web posting path. Minting one no longer leaves a blank untitled row at the top of the Chat list. The pre-write check on a minted API key refused fewer characters than the writer does, so a key the check accepted could still fail at the write — after the endpoint beside it was already stored, pairing a new endpoint with the previous key. The two had drifted because the set was spelled three times; there is now one. A description claimed a processed count reported only the chunks that changed. The update returns every row it matched, so re-enabling chunks that were already enabled counts them all. Two OpenAPI sentences promised no conflict detection and no persistence warnings in a dry run, both of which the same branch had just made false. A described window was wrong whenever a start was supplied without an end. Listing the editors of a built-in skill answered a read with a modification refusal on the internal surface. Archived table listings could reach the strict folder projector again through a third scope value the input type still allowed. A metadata-only secret write skipped the guard its personal-scope twin has. The internal document boundary still took the two processing fields as unbounded strings. Truncation was reported only from the response envelope, so a clipped file body, row search and workflow-stats list said nothing. A staged download stopped watching for signals before it finished removing its directory, and cleared every listener for the signal rather than its own. Three tests asserted a contract constant against itself; they now drive rendered help, real argv, or real render output. * chore: regenerate the API reference, CLI surface, and CLI docs The published reference still marked a secret value required and described the delete parameter as one that also creates, the CLI surface still lacked the marker that says which operations refuse a workspace key, and the reference rendered an empty sentence for every repeatable filter whose default is an empty list. * test(cli): use the package's own delay helper in the staging poll The audit bans a hand-rolled setTimeout promise. `sim-cli` does not depend on the shared utils package, and its own idiom is `node:timers/promises`. * fix: act on a second review round, and correct two earlier claims The conflict pre-check read block ids from the wrong side. The writer inserts each block's own `id` field while the check read the record key, and the two can diverge because preparation copies a value under its key without reconciling them. Edges already read the value and subflows are genuinely keyed by the record key, so only blocks were wrong — collecting every family from the values, as first suggested, would have broken subflows instead. A minted API key carrying leading or trailing whitespace passed the pre-write check but failed the writer, leaving the new endpoint on disk beside the previous key. It is refused up front now rather than trimmed: a key is opaque, so trimming would store a value the server never issued and turn a loud failure into an unexplained 401 later. The endpoint normalizer does trim, which is what made a padded `--endpoint` fail only after the browser flow had already minted a key. A metadata-only secret write raced with deletion returned 500, because the follow-up read that only assembles the response body threw an unclassified error; it now reports the same not-found the non-racing miss already gave. An unusable output format in the environment silently printed a table instead of refusing. Two validation messages printed control characters verbatim. A dry run now reports the preparation warnings its own commit path returns. The chat route created a titled conversation and never wrote a message, so it appeared in the Chat list promising content it did not have. Both sides of a successful turn are now persisted; a failed turn still writes nothing, so a question is never stored without its answer. Two claims of mine were wrong. The earlier commit message said `secrets set` sent an empty value that overwrote the stored secret — it did not; the prompt refuses off a TTY and rejects empty on one, so the old behaviour was a clean refusal. And the delay helper commit said this package's idiom is `node:timers/promises`; the package carries its own `sleep`, which is the audit's sanctioned home and has five callers. It uses that now. Also: a test asserting a deadlock stays unclassified could not fail, since every candidate rejects it; it now pins a unique violation carrying no constraint name. Workflow ids spelled with the file prefix are corrected in the remaining fixtures, leaving the genuine file ids alone. * fix: close a credential-misdirection path this branch had opened Making the endpoint normalizer trim handled whitespace around a value but not a control character inside one, and the URL parser removes those from anywhere in its input — so a value that reads as one host could resolve to another, and the profile's key went with it. The flag and environment paths never touch the config writer, so its guard did not cover this. The normalizer now refuses the same character set the writer does, which also keeps the invariant that nothing it blesses can be refused by the write that stores it. Comparing the parsed URL back against its input was the alternative and is wrong: the parser rewrites percent-encoding, case, internationalized hosts and default ports, so legitimate endpoints would be refused. The blank-query guard tested for exactly empty, so a whitespace-only value still reached the wire — as a real zero on a numeric filter, an explicit false on a boolean one, and as an encoded space the server then rejected. It now refuses any value that is blank once trimmed, while a body string keeps its meaning, an explicit zero still sends, and a value with content around its whitespace is passed through untouched rather than trimmed. A graph-id conflict reported 409 on the v2 route and fell through the older persistence wrapper as an unclassified 500. That wrapper now classifies orchestration failures through the cause chain, which also fixes a pre-existing case where a workflow archived between authorization and the locked read reported 500 rather than 404. Persisting a chat turn claimed its row by id alone, so a conversation soft-deleted mid-turn still received the messages and was bumped back up the list. It now requires a live row. A turn whose caller hung up after the model had already answered persisted nothing, though the work was done and billed; it now persists and still reports the connection as closed. An empty workspace id from the login response was read as no workspace at all. A published description still promised a language-tag standard the schema does not enforce. The test asserting that a turn is stored before the final event drained the whole response first, so it held whichever order the code used. It now reads the stream incrementally and fails if the write moves after the event. |
||
|
Theodore Li
|
d86fdc152f |
feat(api): publish agent tool input schema (#7109)
* feat(api): publish agent tool input schema * fix(api): bound agent tool inputs |
||
|
Theodore Li
|
b391b28d44 | fix(docs): validate generated API navigation (#7099) | ||
|
Vikhyath Mondreti
|
77b5ae3b29 |
fix(security): require explicit trust for internal tool routes (#7089)
* fix: require explicit trust for internal tool routes * fix(tools): harden internal route validation * fix(tools): close request audit gaps * fix(tools): validate request trust policies |
||
|
Waleed
|
4508ec75d2 |
fix(cli): resolve blockers and majors from a full command-surface audit (#7083)
* fix(cli): resolve blockers and majors from a full command-surface audit Audit of all 222 CLI leaves against a live deployment, plus fixes for every defect it confirmed. Blockers: - An unrecognized --profile resolved to built-in defaults, so a typo silently targeted production and transmitted the API key there. - sim logs follow sent an undeclared query key and failed on every invocation. - sim workflows run exited 0 on a failed run, so CI reported success. - knowledge connectors documents update matched rows already in the target state, making exclude and restore permanent no-ops. - PDF text layers below the OCR threshold are transcribed by a model and stored verbatim with no record that it happened. Majors include: rollback --version was swallowed by the program-level flag and silently did nothing; nullable string flags could not send null despite their help promising it; six protocol commands discarded excess arguments, dropping files on upload; sim chat crashed with EPIPE when piped to head; tables import dropped malformed CSV rows without reporting them; audit-logs required an organization id no API surface exposed; secrets could not opt out of redaction or read a value from a file; bulk deletes and moves exited 0 having done nothing; and MCP registrations were destroyed by undeploy rather than restored. Adds extraction_method to documents so OCR output is distinguishable from parsed text, and reports the applied scope on billing logs so the two ledger questions are no longer indistinguishable. * fix(mcp): bound MCP restore by server and re-check uniqueness under the lock Two gaps in the archive/restore lifecycle this branch introduced. The candidate query bounded archived rows and deduplicated to one per server afterwards, so several archived generations stacked on one server consumed the whole budget and every other server the workflow had been published on fell out of the result with no warning. Deduplication moves into SQL so the bound applies to servers, preserving most-recently-updated-per-server. The live-registration check ran before the server lock was acquired, so a concurrent tool create could land in between and the restore would un-archive a second live row for the same server and workflow, violating the partial unique index and rolling back the whole deployment. That check now runs under the lock alongside the tool-name, capacity, and metadata-budget checks it belongs with. * fix(review): address round-three review findings across CLI and server Restore now picks candidate servers by recency: DISTINCT ON forces its own key to lead the sort, so bounding on that statement kept the lexicographically lowest server ids and left a workflow's most recently used servers archived. Deduplication and bounding are now separate stages. CLI: a total miss on tables move reported only in notFound exited 0; unsetting a key or removing a profile mutated the first duplicate INI block while reads merged later ones, so the removal appeared to succeed and did nothing; an import that rejected cells but no rows showed a clean progress line; and the --run-id help implied idempotency it does not provide. Server: a run with no recorded output projection let block-name selectors past the new validation; the billing window comparison still fired on a bound that parsed but failed the shared schema; CSV rejection accounting reached only the streaming path, so buffered and synchronous imports still dropped records silently, through to the Copilot tool that reports them; case-insensitive tag name uniqueness now serializes on the knowledge-base row the delete paths already lock; and a failed sync claim reports the lifecycle reason rather than always claiming a sync is in progress. Reverts an over-scrub from the previous commit: workspace-file-imports is consumed only by Copilot, so naming save_upload and glob there is the correct remediation rather than a leak, and the sweep that guards against leaks now exempts it explicitly. Corrects two contract descriptions that promised a bulk tag save would rename or relocate an occupied slot, which it deliberately no longer does. * fix(cli): remove flags a caller cannot use, and correct three that misled Removes surface that should not have shipped: - `files uploads get` is hidden. Its `--upload-token` was required, and the token is minted and consumed inside a single `files upload`, which completes or aborts its session before returning. Nothing in the CLI could produce the value, so the command answered every invocation by asking for something unobtainable. The same flag is dropped from the two table-import commands, where a CLI-created import is already queryable without it. - The `--no-<flag>` companion that sent JSON null is gone. `--no-X` means "send boolean false" on thirty-seven other flags, and one spelling should not carry two meanings. `--description ''` already clears the displayed value, and the help now warns that the literal word null is stored as text rather than suggesting a substitute, because on the OAuth client fields null revokes a stored grant and an empty string does not. - The document extraction method column and its contract field are reverted. Nothing read them, they were null for every existing document, and the name collided with the parser metadata field that already exists. Corrects flags that misled: the workflow move destination is `--to`, matching its two siblings rather than meaning the opposite of `--folder` one command over; `files list --recursive` is a bare flag like the four folder deletes rather than a twelve-alias string; the dispatch row cap takes a count instead of its wire object; `--yes` no longer claims to be required on commands that accept `--dry-run`; cancelling every run on a table is confirm-gated; and the retry-processing negation, which the route rejects, is suppressed. Extends the guard that missed all of this: it swept only `--x-` prefixes over generated commands, so a header spelled without one was invisible to it. It now derives every header name from the operation table and sweeps the assembled program. * fix(mcp): budget MCP restore against the workflow's live server fanout Restore bounded its candidates at the per-workflow server limit counting archived rows only, never subtracting the memberships the workflow already holds live. The fanout validation a few lines later in the same deploy transaction counts live servers against that same limit, so a workflow with both live and archived registrations could restore past it and roll the whole deployment back. The candidate query now bounds on the remaining headroom, and the budget is re-checked under the server locks and spent once per accepted candidate, so a create that lands between the count and the unarchive cannot push it over. Candidates that do not fit are dropped by recency, matching how the set is already selected, and stay archived with a warning naming the workflow, the server, and the reason — restore still never throws inside the deploy transaction. One residual is left open deliberately: a create on a server outside the candidate set is serialized by neither the locks nor the recount. Closing it would need a workflow-level lock, which would change the ordering every other writer here depends on, and the create path runs its own limit check. * fix(mcp): stop restore spending its budget on servers it cannot restore A server can hold both a live registration and archived ones for the same workflow: the partial unique index constrains only the live row. Such a server was counted twice — once shrinking the restore budget, once consuming one of its slots — before the liveness check under the lock skipped it. While the bound was the full server limit that waste was invisible; once the bound became the remaining headroom, every slot spent that way cost a registration that could have been restored. The candidate query now excludes servers the workflow is already live on, so the budget and the candidate set agree. The exclusion sits on the inner stage, before deduplication and the bound, and is a pre-lock optimisation only: the check under the server lock stays authoritative, because the query can go stale between reading and unarchiving. Candidates rejected for a tool-name collision, the per-server cap, or the metadata budget are still not replaced. That case is only knowable under the lock, so replacing it would mean fetching past the bound and locking servers outside the candidate set, widening an ordering every writer here relies on. |
||
|
Vikhyath MondretiandClaude Fable 5
|
a25d993266 |
improvement(billing): ledger-only usage + period-advance cycle close (#7078)
* improvement(billing): make usage ledger-only and close cycles off period advance Usage is now the attributed usage_log ledger everywhere: the userStats baselines (currentPeriodCost / currentPeriodCopilotCost), the includeLegacyBaseline compatibility flag, the pro-snapshot join/leave machinery, and departedMemberUsage accrual are removed from all read and write paths. Cycle rollover (final sub-threshold overage collection, billedOverageThisPeriod reset, last-period bookkeeping) moves off the invoice.finalized payload parsing — dead for org subscriptions since May — onto a period-advance sweep with a durable per-subscription close marker (subscription.last_closed_period_start), transaction-enlisted Stripe outbox invoicing, and stamp-matched ledger sums. Enterprise closes are bookkeeping-only; reporting-anchor orgs advance the marker alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): include departed actors in close refresh and gate threshold billing on close currency Cycle close now unions current members with every actor holding org-stamped ledger rows in the closed period, so a departed member's daily-refresh consumption offsets the final overage exactly like their billed usage. Threshold billing defers with a pending-cycle-close no-op while a subscription's close marker lags its current period, so the shared billedOverageThisPeriod tracker can never mix an elapsed period's settlements with the new period's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): seal cycle-close races and align refresh actors with billed usage Threshold settlement revalidates the close marker and period under the tracker lock via the same isSubscriptionCycleCloseCurrent predicate the preflight uses, so a rollover between check and transaction aborts as a concurrent-state retry instead of settling against the wrong cycle. Terminal bookkeeping claims the close marker with its tracker reset, so a deletion racing an in-flight sweep close serializes through the one marker and the loser rolls back rather than re-billing settled overage; anchored enterprise deletions only claim the marker since their windows derive from the anchor, not Stripe bounds. A close with overage due but missing Stripe identifiers now defers loudly instead of claiming the marker and silently forgiving the money, the closed window's start derives from the ledger's own period stamps so anchor-day drift cannot misalign the refresh window, calculateSubscriptionOverage unions departed ledger actors into the org refresh deduction like the close does, and blocked accounts report their real ledger usage while staying blocked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): defer ownerless org closes and make the drift test load-bearing A close with overage due but no owner-role member now defers loudly like the missing-Stripe-identifier case instead of claiming the marker and silently forgiving the money. The stamp-drift test pins the marker before the stamped boundary so only the ledger-stamp lookup can produce the asserted window. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(helm): bump chart to 1.6.0 for the billing-cycle-close cron job Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): revalidate the org roster under close locks and align invoice labels The close re-reads the member roster inside its transaction and defers on any change, mirroring threshold billing — an owner transfer moves the billed-overage tracker between rows, so a pre-lock roster could settle against the wrong tracker. Invoice labels now use the closed period's end month like every other overage path, and the sweep test's rows are shaped like rows the candidate query can actually return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): hold cycle close for a settlement grace after rollover Billing attribution is frozen at run start, so a run straddling a rollover can insert elapsed-period-stamped rows after the period ends. Closing only once the rollover is older than any possible in-flight run guarantees the close's ledger sums are final; the sweep picks the period up on a later run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): resolve reporting windows through the canonical period resolver The close paths now ask resolveSubscriptionUsagePeriod whether a subscription derives its windows from a reporting anchor instead of re-checking metadata shape locally, so a malformed hand-edited anchor that the resolver rejects (falling back to Stripe bounds) books its Stripe-stamped ledger rows normally instead of skipping bookkeeping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): union departed ledger actors in org threshold settlement Organization threshold billing now reads the period ledger per user and unions the actors holding org-attributed rows with the current roster before computing refresh deductions — the same actor set calculateSubscriptionOverage and the cycle close use — so a departed member's usage cannot be settled without their daily-refresh offset. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): claim the terminal period before deletion settlement and delete vestigial refresh bounds Subscription deletion now claims the close marker from the fresh subscription row before computing or charging final overage, serializing with the cycle-close sweep so both paths can never bill the same period — an in-flight close fails its guarded claim and rolls back, and the deletion settles against the row's real period instead of a possibly stale webhook payload. The per-user refresh bounds machinery is deleted outright: its only source was proPeriodCostSnapshotAt, which this PR stopped writing, and ledger entity stamps already scope refresh to org-attributed rows — a joiner's pre-join usage is user-stamped and can never enter the org refresh scan, while a departed member's org-stamped rows participate exactly like a current member's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): scope daily refresh by entity stamps and close lagging periods before deletion settlement The refresh actor-list threading (userIds + departed-actor unions) violated the entity-stamp principle the ledger sums already follow: org-attributed rows from departed members counted in pooled usage but never consumed refresh on the monitor/resolved-usage paths. Daily refresh now scopes rows by the same write-time entity and period stamps as the ledger — no actor list anywhere — which deletes the unions, the rollup memberIds, and the org-specific refresh variant. Deletion settlement now closes any elapsed-but-unclosed period (grace bypassed — no later sweep revisits a canceled sub) before claiming the terminal period, so a deletion racing the sweep can no longer jump the marker past an unsettled period and silently forgive its final overage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(billing): bucket refresh by clamped day so stamped stragglers stay in the deduction Refresh membership is now the entity/period stamps alone — identical to the ledger sums it offsets. A row written after the rollover but stamped to the elapsed period (attribution frozen at run start) is billed by the stamp-based close, so it must consume refresh too; created-at now only assigns the day bucket, clamped into the period, instead of excluding the row entirely. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(billing): pair the overage tracker with the marker's period and pin test clocks billedOverageThisPeriod only ever holds collections for the period that began at the close marker — the threshold gate blocks settlement whenever the marker lags. Both consumers now honor that pairing: a close that skipped forgiven periods counts nothing from the tracker against the period it bills, and the deletion settlement ignores the tracker when the marker was still lagging at claim time. Ignoring is provably safe in both cases because a lagging marker means no current-period collections exist. The cycle-close and daily-refresh suites pin the system clock: their grace and window checks compare fixed period fixtures against Date.now(), which made them dependent on the host date. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(billing): reject lagging terminal claims and thread billingInterval into deletion bookkeeping claimTerminalPeriod no longer advances the marker over an unclosed elapsed period: a lagging marker is reported without a write so the deletion handler can run the elapsed close once more (healing a rollover that committed between close and claim) and only then seal the marker explicitly, with an error log, when the period is genuinely unclosable. Sealing preserves the in-flight-sweep abort guarantee. Deletion bookkeeping now passes the subscription's billingInterval through, so an enterprise reporting subscription whose interval lives on the row column (not metadata) still resolves as reporting-anchored and keeps its bookkeeping no-op, matching every other resolver call site. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Theodore Li
|
4c0bd944dc |
feat(slack): launch v2 triggers and backfill custom bots (#6873)
* feat(slack): launch v2 triggers and backfill custom bots * fix(slack): propagate legacy webhook dispatch failures * fix(slack): continue shared legacy webhook fanout * fix(slack): acknowledge filtered webhook deliveries * fix(slack): finalize custom bot migration rollout * fix(slack): dedupe migrated bots per workflow * fix(slack): harden custom bot rollout * fix(slack): acknowledge permanently ignored deliveries * fix(slack): retry failed webhook deliveries |
||
|
WaleedandTheodore Li
|
656840a1b1 |
feat(api): make the platform operable headless over v2 (#6912)
* feat(v2): download run output files by API key
Adds GET /api/v2/workflows/{id}/runs/{runId}/files/{fileId}, closing the
async-run loop for headless callers. A run's output carries UserFile URLs
pointing at /api/files/serve/..., which rejects x-api-key outright, so an
async run that produces a file previously had no byte path out for an API
key at all.
The file is addressed by the id the run reported and resolved against the
run's own recorded execution data, from which the storage key is read. The
request never supplies a storage key, so the endpoint cannot be aimed at
bytes the run did not produce. Resolution deliberately reads the
materialized-but-undisplayed recording, because the display projection
strips exactly the `key`/`context` fields a byte read needs.
Also hardens normalizeStartFile to derive a file's storage key only from a
validated internal serve URL, discarding any caller-supplied `key`/`context`.
A workspace API key has no human subject, so the executor resolves its actor
to the workspace billing owner (preprocessing.ts -> resolveSystemBillingAttribution);
verifyFileAccess then authorizes a workspace-context key as that owner, whose
reach is not bounded by the key's workspace. Accepting an attacker-authorable
key made that substitution exploitable as a confused deputy. Normalization is
all-or-nothing, so a forged file now drops the whole files input.
* feat(workflows): one graph-write door, principal-derived audit source, and v2 authoring endpoints
Extract replaceWorkflowNormalizedState as the single persistence primitive for a
workflow graph replace and route both the internal editor save and the Copilot
edit tool through it, so neither can skip state preparation, the row lock, the
lastSynced stamp, or custom-tool extraction by choosing a different entry point.
Derive the audit source from the acting principal instead of hardcoding
'copilot', then widen workflows.variables.apply_operations and
workflows.bulk.move to every principal kind.
Add GET/PUT /api/v2/workflows/{id}/state, POST /operations, /duplicate,
/restore, PATCH /variables, and POST /api/v2/workflows/move over surface-neutral
application use cases; move the edit engine to lib/workflows/editing.
* test(workflows): cover the graph-write primitive, audit source, and the v2 authoring surface
Pin the two-doors fix (preparation runs, the row is locked, custom-tool
extraction is post-commit and best-effort) and the false-audit fix (a session
principal writes source: 'session', a delegated one writes its service). Both
were verified to fail with the fix reverted.
Add the application matrix for replaceWorkflowState, applyWorkflowOperations,
readWorkflowGraph, and restoreWorkflow — role floor, principal-kind rejection
before canonical load, asserted-scope concealment, lock, validation, atomic
conflict, plan gate, and audit-then-notify ordering — plus route tests for
every new endpoint.
* test(workflows): pin the internal graph-write door and the v2 list scope
Characterize saveWorkflowNormalizedState's statuses, messages, and notification
after the persistence extraction, and cover the new scope filter on
GET /api/v2/workflows including a cursor replayed under a different scope.
* feat(api): add v2 block, tool, connector-type, and enrichment catalogs
Adds six read endpoints under /api/v2 that publish Sim's code-defined
catalogs: GET /blocks, GET /blocks/{blockId}, GET /tools,
GET /tools/{toolId}, GET /connector-types, and GET /enrichments.
These read like static reference data and are not. What a caller may
place is decided per workspace by its permission-group integration
allowlist, per organization by which unreleased blocks have been
revealed, per deployment by ALLOWED_INTEGRATIONS, and per workspace
again by the workflows it has deployed as blocks. So all six are plain
defineWorkspaceOperation reads at minimumRole 'read' with
workspaceApiKey 'allow' — the exact policy of credentials.providers.list
— and every response keeps Cache-Control: private, no-store, because an
unrevealed preview block's existence must not leak across organizations
through a shared cache.
Trigger blocks ride as ?capability=trigger rather than a second
endpoint, and workspace custom blocks ride inside /blocks discriminated
by `source`, so "what may I place?" stays a one-call question.
The block projection is extracted out of the Copilot get_blocks_metadata
tool and rewritten onto @/tools/metadata and @/tools/metadata-outputs.
That cuts the tool's own @/tools/registry edge as a side effect: its
module graph drops from 6,756 to 1,318, and the new routes land at
1,673-1,734, next to the shipped /v2/credentials/providers baseline of
1,668.
Supporting changes:
- scripts/sync-tool-metadata.ts derives hostedApiKey ('always' |
'conditional' | 'none') from each tool's `hosting`. The config itself
stays excluded because it holds closures, but "does Sim host the key"
is a first-order authoring question, so the answer is emitted.
- getCopilotToolDescription takes hostedApiKey as an option instead of
reading `hosting` off the tool, so both an executable ToolConfig and
the generated metadata can answer it through one shared derivation.
- principalUserId / allowedIntegrationTypes move out of
lib/credentials/application/provider-catalog.ts into
lib/integrations/principal-scope.server.ts. Two copies of the
workspace integration gate would diverge first on the workspace-key
path, which has no user for permission groups to key on.
- scripts/check-tool-registry-boundary.ts walked page.tsx/layout.tsx
under app/workspace only, so a route importing the executable registry
passed green. It now walks a list of entry sources, seeded with the
four catalog route subtrees and the shared projection barrel. Routes
are covered per subtree rather than wholesale because 122 of ~1,130
route files legitimately execute tools.
Registry sweeps parse every block, tool, connector type, and enrichment
through its published response schema and compare against the wire
round-trip. They caught a real drift while being written: an operation's
inputs were typed as a union of the tool-param and block-input shapes,
and the union resolved to whichever member matched first, silently
dropping a block input's `schema`.
* docs(api): stop publishing a 413 GET /workflows/{id}/state cannot emit
* feat(v2): read upload-session state
Adds GET /api/v2/files/uploads/{uploadId}. Only DELETE was exported, so a
caller that lost track of a transfer could abort it but could not ask
whether the session was still alive, already finalized, or failed — the
resume story was missing.
Runs on a new files.upload.read operation at minimumRole 'read' rather than
reusing uploadCancel, which is a 'write': asking about a session must not
require permission to destroy it. The GET is a control leg like every other,
so it carries the signed upload token and re-authorizes the caller's present
workspace permission through reauthorizeWorkspaceUploadPurpose instead of
resolving the session on its id alone.
* fix(api): reconcile v2 catalog and workflow-authoring integration
Merging the catalog and workflow-authoring branches surfaced four issues
that neither produced in isolation.
- Route and OpenAPI counters were bumped to the same value on both
branches, so git merged them as one change while the merged tree holds
the sum. Corrects the route ratchet to 1142 and the workflows document
to 29 operations (152 total), then regenerates the OpenAPI documents
and the CLI surface from the reconciled contracts.
- The seven new workflow operations were published in the spec but absent
from the workflow API reference groups, which `check:openapi` rejects.
- `route-policies.ts` reached `WorkflowOperationsNotAppliedError` through
`apply-workflow-operations`, dragging the edit engine — and its diff and
comparison dependencies, which reach a client OAuth hook — into every
route that uses the shared workflow error policies. The class moves to
its own leaf module, mirroring `WorkflowImportError`, and each importer
now takes it from there.
- The operations route test shadowed that class inside its module mock, so
`instanceof` matched a fake and the assertion pinned a message the
production class never emits. It now uses the real class and asserts the
real message.
* feat(v2): extract ZIP archives over the public API
Adds POST /api/v2/files/{fileId}/extract and widens files.extract_archive
from principalKinds ['session'] / workspaceApiKey 'deny' to admit personal
and workspace API keys at the unchanged 'write' role.
The widening is an authorization change, so the justification lives in the
operation's TSDoc: extraction grants no capability an API key lacks, since
every file it writes could be created one at a time through files.create and
files.upload.create, both already 'allow' at the same role. It only collapses
many calls into one. The previous ['session'] restriction read as an artifact
of the UI having been the only caller. Delegated services stay out — no
copilot or executor caller exists and admitting one is a separate decision.
The response is counts plus the destination folderPath, never the extracted
files: a large archive would otherwise materialize thousands of objects into
one body. Callers page GET /api/v2/files?folderPath=... instead. The use case
returns the internal display path and the adapter projects it to a v2 path,
keeping the use case surface-neutral.
* feat(v2): extract file text over the public API
Adds GET /api/v2/files/{fileId}/text. Text extraction previously sat behind
checkInternalAuth on /api/files/parse, a route that also mixes in external-URL
fetching, execution-file upload, and multi-file aggregation, so it could not be
reused. The parse call is lifted into a thin application use case instead.
Runs on the existing files.read_content operation unchanged — it is already
workspaceApiKey 'allow' at the read role, and turning bytes it already
authorizes into text grants no further reach.
`degraded` is a required, non-optional boolean on the response. The legacy doc
and ppt parsers deliberately return best-effort or placeholder content rather
than throwing, so an omittable flag would let a client that never checks it
treat guessed text as extracted text. It is reported honestly rather than
converted into an error, because the parsers' behaviour is deliberate and
characterization-tested.
The read is bounded on its input at 25 MiB before extraction rather than on its
output after, given the parsers' documented DoS history; a caller may lower the
ceiling but never raise it.
* feat(v2): restore archived folders and list the archived set
DELETE /api/v2/files/folders archives recursively, so a recursive delete was
unrecoverable over the API: the archived files stayed visible through
GET /api/v2/files?scope=archived, but nothing could rebuild the folder
structure.
Adds POST /api/v2/files/folders/restore, path-addressed like the rest of the
v2 folder family, and a `scope` selector on the folder list so a caller can
find the archived path to hand it.
`scope` extends the files folder-list query rather than the shared
v2ListFoldersQuerySchema: only workspace files have an archived folder set, so
adding it to the shared schema would give tables, workflows, and knowledge a
parameter they ignore. GET /api/v2/files/folders is a FULL_SET_LIST, not paged,
so no cursor binding changes — list-pagination.test.ts passes unchanged.
Restore resolves the archived folder from its path by scanning the archived
set rather than walking the live tree, which by definition does not contain
the folder being restored. The folder-restored analytics hook now reports the
folder actually restored rather than the requested selector, which carries no
id on a path-addressed surface.
* feat(v2): bulk-download a file selection as a zip
Adds GET /api/v2/files/bulk-download, an adapter over the existing
downloadWorkspaceFileItems use case and its internal binary route.
Path collision: a static segment beside [fileId] permanently shadows a file
whose id equals it, and workspaceFileIdSchema does accept [A-Za-z0-9_-]+.
Rather than invent a new shape, this follows the existing bulk-delete sibling:
the hyphenated form cannot be produced by either minted id shape (UUID v4 or
wf_<shortId>), so the shadowed id is unreachable in practice. Documented on
the contract so the reasoning is not lost.
Folders are addressed by path, matching the rest of the v2 file surface. The
paths resolve against the folder set the selection already loads, so it costs
no extra query, and a path matching no folder is rejected rather than silently
dropped — a misspelled folder must not yield a zip of whatever else was
selected. The empty-selection and folder-count guards now account for
folderPaths, which a path-only selection would otherwise have tripped.
Selections are comma-separated only: v2 rejects a query parameter sent more
than once, so a repeated-parameter form would never reach the schema. Pinned by
a test so the contract cannot advertise a form the boundary rejects.
* feat(v2): expose run output files and optional inline bytes on the runs read
GET /api/v2/workflows/{id}/runs/{runId} now reports the files a run produced,
each with the downloadPath that fetches its bytes, and can inline them as
base64 on request.
Gated by includeOutput, matching `output`'s nullability: a caller that did not
ask for output does not receive a file list it did not request. The async
execute request's rejection of includeFileBase64 is deliberately left alone —
at submit time the run has not happened, so there is nothing to inline; reading
a finished run is the first moment the question means anything.
Inlining is capped per file at the executor's 16 MiB inline ceiling, which a
caller may lower but never raise. A file above it answers 413 naming that
file's downloadPath, so the caller is told exactly how to get the bytes rather
than being left stuck.
The descriptor deliberately omits the storage key — files are addressed by id
and the key is re-derived from the run's recording — and omits an expiry, which
the recording does not carry and which would be fabricated if published.
The route becomes headSafe: false, since inlining reads object storage. The
builder enforces that this requires the use case to expose authorize(), so HEAD
still answers from a real authorization rather than from authentication alone.
* feat(v2): permanently delete an archived file
DELETE /api/v2/files/{fileId} only archives — the OpenAPI says its stored bytes
are never removed — so there was no way to actually destroy a file over the API.
Adds the repository primitive, application use case, operation, and
DELETE /api/v2/files/{fileId}/permanent.
A distinct path rather than a flag on the ordinary delete: a query parameter
that turns a recoverable archive into an irreversible destruction is set by
accident, and the two acts carry different minimum roles, which one route
declaration cannot express. The file must already be archived; a live file
answers 409 naming the archive step, so no single request can turn a live file
into lost bytes.
minimumRole 'admin', which forces workspaceApiKey 'deny' since the workspace-key
ceiling is 'write' — the desired policy anyway: unattended credentials should
not destroy bytes.
Row first, then object. The two legs commit independently, so one can survive a
crash between them: deleting the row first leaves at most an orphaned object for
the storage sweep, while the reverse would leave a live row pointing at bytes
that no longer exist — a file that lists and opens but can never be read. A
failed object delete is therefore reported as objectDeleted: false rather than
thrown, because the request has genuinely succeeded once the row is gone. Both
directions are pinned by failure-injection tests, verified to fail when the
order is reversed.
Audited as a distinct FILE_PERMANENTLY_DELETED action, not a reuse of
FILE_DELETED, which records the recoverable archive step.
* feat(api): v2 log analytics, itemized cost, filters, and sortable query
Adds the aggregate and rich-read halves of the public logs surface, and
fixes three defects the existing reads carry.
Aggregate analytics. `GET /api/v2/logs/stats` returns time-bucketed run
counts, success rate, error count, mean latency, and the window bounds,
per workflow and for the workspace. The first-party route was a raw
handler with inline SQL and inline aggregation, so it is split into a
repository (`lib/logs/stats-queries.ts`), a pure aggregator
(`lib/logs/stats.ts`), and an application use case. That route keeps its
legacy authorization — it answers a caller without workspace access with
a zeroed 200, where v2 conceals the workspace as a 404 — and consumes
only the two surface-neutral halves.
`segmentCount` had no `.int()`, `.min()`, or `.max()`, so `0` divided by
zero and `1e9` allocated two billion-element arrays: both caller-reachable
500s. Bounded on both contracts. `workflows` is capped, with the workspace
totals still computed from every workflow and the cut reported as
`workflowsTruncated`.
Detail reads gain the itemized `cost.items` ledger (`null` and `[]` are
distinct answers and both reachable) and `workflowInput`, restoring a
v1→v2 regression.
The list gains `workflowName` and `status` filters, and `includeJobRuns`,
which unions Chat and Sim-agent job runs into the sequence behind a new
`kind` discriminator — without it a job run is indistinguishable from a
run whose workflow was deleted. A filter no job row can answer drops the
branch outright rather than meaning two things across the union.
`POST /api/v2/logs/query` carries the additional sort columns. `GET /logs`
is untouched: its single `order` param rests on there being exactly one
sortable column, and both escapes from that are ruled out, so the rich
read gets its own endpoint — the split the table surface already ships.
It uses the shared keyset scheme with the two nullable sort columns read
through a sentinel, since a keyset cannot compare against null.
`folderPaths` now covers a folder's whole subtree on the public path, as
it already did everywhere else; it previously omitted every nested run
with no error. The path strings did not change, so a folder-scope version
is stamped into the cursor and in-flight tokens restart rather than
silently skipping rows.
Also fixes `folderName`, which ILIKEd `workflow.name` — a copy of the
clause above it — and so searched workflow names instead of folders.
`buildLogSortCursorCondition`'s `IS NULL` disjunct is documented and
pinned: under `NULLS LAST` the null block is only reachable through it,
so removing it as a duplicate-row fix makes those runs unpageable.
Ratchets: route count 1142 -> 1144; logs OpenAPI operations 2 -> 4; total
operations 152 -> 154.
* feat(api): v2 tables run state, dispatch polling, batch update, bulk, archive
Closes the headless gaps on the v2 tables surface.
- Per-cell run state is now readable through an opt-in `includeRunState` on
`GET /rows`, `POST /query`, and `GET /rows/{rowId}`. The default projection
is byte-identical; a page whose sidecar outgrows its byte budget is a 413
rather than a silent truncation.
- Run dispatches are addressable: `GET /tables/dispatches/{dispatchId}`
publishes the column's full four-state domain so polling a finished run is
not a 500, and `GET /tables/{tableId}/dispatches` lists what is in flight.
- `POST /rows/batch-update` takes one distinct patch per row. Its transaction
moved out of the Copilot-only module into a surface-neutral use case both
surfaces now call.
- `GET .../enrichment/{groupId}` publishes the provider cascade, cost, and
timing behind one enrichment cell.
- `POST /tables/bulk-move` and `/bulk-delete` reach the existing bulk use
cases, which now accept folders by canonical path and resolve them inside
the application layer.
- `DELETE` is recoverable: `scope=archived` on the table list plus
`POST /tables/{tableId}/restore`.
* feat(api): expose knowledge chunks, tag writes, archive/restore on v2
Closes the knowledge cluster's remaining public-surface gaps.
Chunks: list/read/create/update/delete/bulk under
`/api/v2/knowledge/{id}/documents/{documentId}/chunks`. `queryChunks` gains
an `id` tiebreaker on every sort so the list pages on a keyset rather than an
offset — `tokenCount` and `enabled` are both non-unique, so a page boundary
inside a run of equal values used to repeat or drop the tied rows. The
internal offset caller is unchanged; the two positioning schemes share one
read.
Tag definitions: create, update, delete, next-slot, usage, and the
document-scoped save and cleanup. Without them a caller could write a tag
value into a slot with no definition and then had no way to name it, so
tag-filtered retrieval was unbuildable end-to-end. `v2KnowledgeTagSchema`
gains `id`, without which PATCH and DELETE are unaddressable. The
document-scoped DELETE is pinned to `action: 'cleanup'`: the domain's `'all'`
deletes the whole knowledge base's tag vocabulary from a document path.
Archive/restore: `GET /api/v2/knowledge/archived` as a sibling route rather
than a `scope` param — the two reads bind different operations and a v2 route
declares one — plus `POST /api/v2/knowledge/{id}/restore`. `knowledge.restore`
is a new workspace operation carrying `delete`'s policy, since an operation's
inverse must not be harder to reach; the internal session route now delegates
its workspace branch to the shared use case and keeps only the legacy personal
one.
Also: `POST .../documents/from-workspace-files` surfaces `addWorkspaceFiles`,
so a file already in workspace storage no longer has to be re-uploaded
byte-for-byte to be indexed; the `chunkingConfig` write widens to the
first-party five-key schema with its refines and separator bounds, while the
response stays `.catchall` so a legacy JSONB row cannot 500; and
`CONNECTOR_MANAGED_RESOURCE_READ_ONLY` joins `FORBIDDEN_DETAIL_CODES` now that
the bare 403 on connector-managed chunk writes is wire-reachable.
Document upsert is deliberately not included.
* feat(api): add v2 credential rotation and a gate-exempt capabilities endpoint
PATCH /api/v2/credentials/{credentialId} rotates service-account secret
material or renames a credential in place, preserving the credential id so
existing workflow, deployment, paused-run, connector, and webhook references
keep working. Re-posting to POST /api/v2/credentials answers 409, and
delete-and-recreate mints a new id, so rotation previously had no door.
The route is adapter-only: updateWorkspaceCredentialUseCase already owned the
rotation, its audit projection, and credentials.update. It gains one additive
assertedWorkspaceId field for the v2 workspace assertion, and the per-principal
credential-type table that deleteCredentialUseCase already applied is lifted
into requireManageableCredentialType so both operations share it. Without it a
personal API key could rename an env_workspace row and toV2Credential's throw
would surface as a caller-reachable 500.
CredentialProviderOperationError now maps to 503 with Retry-After when the
provider is unreachable, instead of the 400 its OrchestrationError('validation')
base projected. A transient outage rendered as a permanent input error invites a
caller to revoke a working credential.
GET /api/v2/meta reports the calling key's rollout cohort, type, and expiry.
It is the one route declaring the new typed gate: 'exempt' option, because the
rollout gate and the unknown-path catch-all answer byte-identical 404s and a
gated /api/v2/meta could never resolve that ambiguity. Authentication still runs
first, so the only fact disclosed is one about the caller's own credential.
* feat(api): publish deployment lifecycle and workflow-MCP v2 surfaces
Adds the four deployment-lifecycle operations v2 was missing, and the
workflow-as-MCP publishing surface, both as adapters over application use
cases that already existed.
Deployment lifecycle:
- PATCH /api/v2/workflows/{id}/versions/{version} relabels a version.
Deliberately not the internal route's body-shape dispatch between
"rename" and "promote to live".
- POST .../versions/{version}/activate promotes a version. Same use case
as rollback under a different transition, on its own path because the
two mean opposite things to a caller.
- POST .../versions/{version}/revert overwrites the draft. Accepts the
literal `active` alongside a version number.
- PATCH /api/v2/workflows/{id}/deployment toggles unauthenticated public
execution.
`workflows.public_api.update` widens from session-only to session plus
personal API key: it is an admin-role change the same accountable human
may make from a script. Workspace keys stay denied. Its EE refusal now
carries PUBLIC_SHARING_NOT_ALLOWED instead of a bare forbidden.
Workflow MCP servers:
- /api/v2/workflow-mcp-servers list, create, update, delete, plus
publish and unpublish of a workflow as a tool. Named apart from
/api/v2/mcp-servers, which registers the external servers Sim calls.
- The six mcp_servers.workflow_deployments operations widen from
['delegated'] to admit sessions and personal API keys; roles and the
workspace-key denial are unchanged.
- The server list gains keyset pagination, matching its external
sibling, since nothing caps how many a workspace publishes.
- Server, tool, and workflow reads move out of the use case into
lib/mcp/queries.
Route ratchet 1150 -> 1160; OpenAPI operations 161 -> 171.
* feat(api): extract chat deployments and publish the v2 surface
Chat deployment was a shipped module with no public API and two
authorization systems: `lib/workflows/application/chat-deployments.ts`
had deploy/undeploy extracted, but only Copilot used them — the REST
routes reimplemented workflow authorization inline, and `PATCH
/api/chat/manage/[id]` additionally owned password encryption, the
auth-type field-clearing matrix, identifier uniqueness, the
redeploy-gating protocol with two 409s, a raw db.update, and a manual
recordAudit.
New `lib/chat-deployments` domain:
- `chat_deployments.list/read/update/delete`, keyed on the deployment
whose workspace is derived by joining its workflow. Creation stays
`workflows.chat.deploy`, which is keyed on the workflow.
- The PATCH extraction, including the field-clearing matrix and the
asynchronous-cutover invariant the route had hand-mirrored from
`performChatDeploy`.
- One `buildChatDeploymentUrl`, replacing three constructions that had
already drifted onto two different host helpers. There is no chat
subdomain, so nothing publishes a host.
- Repository reads moved out of the use cases into
`lib/chat-deployments/queries`.
Internal routes are now adapters over those use cases. `GET /api/chat`
is deliberately not migrated: it scopes by `chat.userId` while every
other chat operation authorizes by workspace admin, and reconciling the
two is a product decision. `PATCH` keeps its 400 for an identifier
collision through a typed `ChatIdentifierInUseError`; v2 reports the
409 the condition actually is.
v2 surface at `/api/v2/chat-deployments`: list, create, read, update,
delete. Workspace-scoped, keyset-paged, and a stored password is never
readable — reads carry `hasPassword` only, and the session-only reveal
endpoint deliberately has no v2 counterpart.
Also: an email- or SSO-gated chat with an empty allow-list is now
refused in the use case rather than only at the internal boundary, since
it is unenterable; and the doc comment on `processHostedKeyCost`
claiming a `usageLog` write is corrected — no such write exists.
Route ratchet 1160 -> 1165; OpenAPI operations 171 -> 176.
* fix(api): close three review findings, two of them caller-reachable
- Run output files are filtered to keys under the run's own execution
prefix. The recording they came from is not a trustworthy key source:
the start block copies every caller-supplied input field verbatim into
its output and `collectUserFilesById` accepts anything carrying the
`UserFile` shape, so a caller could name any storage key and have the
download and base64 paths — neither of which authorizes per file — serve
it back.
- `getBlock` reads own keys only. `BLOCK_REGISTRY` is an object literal,
so `constructor`, `toString` and friends returned inherited functions
that every consumer then treated as a block, turning a path segment into
a 500. `getToolMetadata` already guarded this way.
- A folder-scoped log page no longer unions in every job run in the
workspace. The guard read `filters.folderIds`, which the public surface
never sets — it carries the folder filter in `folderScope` — so the page
contradicted the contract's promise that job runs are dropped whenever a
filter they cannot answer is set.
Also: the log cursor stamps `includeJobRuns` only when it is on, so its
`.default(false)` no longer puts a constant in every fingerprint and
rejects cursors minted before it existed; and the `folderName` subquery is
scoped to the workspace and to workflow folders instead of scanning the
whole `folder` table.
* fix(api): close two more review findings, one an authorization bypass
- `workflows.operations.apply` no longer admits a workspace API key. The
use case authorizes against three per-user policies — the EE permission
config, block visibility, and credential reachability — and all three
take a human subject. An actorless key has none, and both substitutes
fail open: attributing to the workspace billing owner evaluates the
batch as the least-restricted account in the workspace, and passing no
user makes `getUserPermissionConfig` return `null`, which every caller
reads as unrestricted. Either way a workspace constrained by an
allowlist was edited as though it were not. Personal keys keep the
capability, so headless editing is unaffected for a credential that
names a human.
- `GET /workflows/{id}/state` reads its variables through
`parseWorkflowVariables`, and the stored variable response schema drops
the two assertions the column cannot honour. The column has carried a
JSON string and a legacy array as well as the current record, the
realtime `variable.add` op types `type` as `z.any()`, and the parser
writes `name` through verbatim — so the input bounds on the read turned
a stored workflow into a 500 on the endpoint that opens it. The write
schema keeps them, which is where they can still be honoured.
- `GET /workflows?scope=archived` projects folder paths tolerantly.
Archiving a folder cascades onto the workflows inside it but leaves
their `folderId` dangling — which is why restore has to null it — so the
strict projector threw a bare `Error` and took the whole page down with
no cursor able to step past the row.
* fix(files): bind Start-block file keys to the executing workspace
The Start block derived a file's storage key by parsing the caller's own
`url`, which `isInternalFileUrl` matches on any host and
`extractStorageKey` returns verbatim — so a request body could name any
tenant's bytes. The key is now accepted only when its own layout names
the workspace the execution runs in, and every file is dropped when the
execution carries no workspace.
Also bounds `includeFileBase64` with an aggregate response ceiling and a
worker pool instead of an unbounded `Promise.all`, scopes the bulk
download's authorization resource to the workspace when folder paths are
requested, makes the folder-restore selector mutually exclusive at the
type level, and names the bound in the `maxBytes` validation message.
* fix(api): close v2 log review findings
Cursor scope: `scope` on the workflow and table lists carries
`.default('active')`, so it entered every fingerprint as a constant and
refused every cursor minted before the param existed — with the
"cursor does not match the requested filters" 400, which is actively
misleading for a caller that changed nothing. Both now stamp the
default as absent, so only a caller who asked for `archived` gets a new
sequence.
Dashboard stats: `maxWorkflows` capped the response, not the
allocation. Segment series are now densified after the cut instead of
before, so returning 200 series no longer materializes one
`segmentCount`-length array per workflow in the window. The aggregate
still sums every workflow, now from the sparse per-workflow maps.
Cost keyset: `cost_total` is an unconstrained `numeric`, so its anchor
travelled through `Number()` and was compared back at full precision —
rows differing beyond float64 collapsed onto one anchor. Adds
`decimalKey`, which carries the digit string and binds it `::numeric`.
Run detail: `cost_total` is a backfilled projection, so a run predating
the backfill reported `cost: null` even with a real ledger, making
`items` unreachable for exactly the runs the ledger explains. Falls
back to the ledger total.
Also caps the log folder-path index reads at MAX_FOLDERS_PER_WORKSPACE
like every other reader, publishing the folder-tree 413 on the four log
operations; reverts a dead `status` widening in `v2CommaListSchema`;
drops an unread `executionData` select; corrects the segment-count and
searchLogs prose; and replaces the sort-cursor SQL-text assertions with
a two-page walk over a fixture with a null block.
* fix(api): close knowledge v2 review findings
- widen knowledge.list_archived to the delete/restore policy so a workspace
API key can discover what it may restore
- escape LIKE wildcards on the now-public chunk search
- derive tag slot capacity from TAG_SLOT_CONFIG per field type
- type updateKnowledgeBase's chunkingConfig as ChunkingConfig and project
every declared field explicitly
- attribute a restore to the calling surface instead of a literal 'api'
- gate 'knowledge chunks batch-update' behind --yes, since it can delete
- present the tag-cleanup action from the parsed request rather than
faulting on the domain result after the delete committed
- unbind asserted-scope workspaceId from the nested knowledge cursors,
matching the table-row lists
- add executed-SQL coverage for the chunk keyset
* fix(catalog): close the catalog and registry-boundary review findings
The module-graph ratchet treated an entry with no baseline row as
informational, so the six catalog routes and the projection barrel were
unratcheted while the summary still read "within their module-count
baseline". An unbaselined entry now fails --check, the summary counts only
what was actually compared, and the baseline is re-recorded.
The Copilot block-metadata tool — the reason the shared projection exists,
6,756 modules down to 1,321 — was in no guarded subtree. It is now an entry
source and a catalog boundary root.
Catalog behaviour:
- hostedApiKey is gated on the deployment, so a self-hosted install reports
none instead of promising 127 tools' keys it will never supply
- block detail resolves an unversioned base type to its newest version and
projects through the viewer's visibility, so it can no longer 404 a block
the list contains or name it differently
- offset-cursor ordering compares code units rather than the process locale
- projections copy every array they publish instead of handing out the
registries' own
- an options function returning a thenable throws rather than silently
widening the providers-store substitution across the event loop
- a throwing block projection costs the Copilot tool one block, not all of them
- the trigger-kind log returns to debug: chat/manual/api are entry-point
kinds, not authoring defects
Also sweeps the custom-block detail branch against its response schema,
guards each projection module rather than the dead barrel over them, and
drops a provably dead branch in processHostedKeyCost.
* fix(workflows): close v2 workflow-authoring review findings
- Read a blockless draft back as an empty graph. `PUT /state` of
`{ blocks: {}, edges: [] }` — the contract's own published example —
deletes every block row, and the loader answers `null` for a blockless
workflow, so the following `GET /state` answered 404 while the list
endpoint still showed the workflow. Existence is the workflow row's to
decide; the null is now projected as an empty graph.
- Rewrite the `readWorkflowGraph` authorization test so it can fail. It
called `authorize?.()` and asserted only a negative, so deleting
`authorize` or replacing it with a no-op both passed — the invariant the
head-safe `HEAD` path depends on.
- Route `setWorkflowBlockEnabled` through `replaceWorkflowNormalizedState`,
the same door the other two graph writes use, instead of writing the
normalized tables itself without state preparation or custom-tool
extraction.
- Count applied operations directly. Enablement refusals landed in the same
skipped-item array and were subtracted from the operation count, which
`Math.max(applied, 0)` then masked when it went negative.
- Give a `disabled_ancestor` refusal its own member of the published skip
enum instead of reporting it as `block_locked`.
- Refuse an `atomic` batch whose credential or hosted API key would be
stripped, and carry the dropped inputs in the 409 details.
- Publish the whole lint report — `sources`, `sinks`, `orphanBlocks`,
`emptyOutgoingPorts`, `invalidBranchPorts`, `invalidConnectionTargets`,
`fieldIssues`, and the `kind` discriminator on unresolved references —
rather than only free-text reference prose.
- Stop reporting unresolved lint references as `inputValidationErrors`.
`collectUnresolvedReferences` is read-only, so those values stay
persisted; they were double-reported, and falsely as dropped inputs.
- Replace two unfalsifiable negative-principal tests, which used a
principal kind `Exclude`d from `PrincipalKind`, with a reachable one.
- Nits: drop a stranded TSDoc block; assert the membership predicate in the
selector-validator admin test; make the HEAD test assert a representation;
assert the sanitized graph is what `replaceWorkflowState` writes; add a
route test rejecting `baseGraph` in a v2 body; carry
`principalAuditSource` on restore/duplicate/moveBulk audit; unify the two
`base64MaxBytes` ceilings on `MAX_INLINE_MATERIALIZATION_BYTES`.
* fix(api): close seven v2 review findings, one an authorization bypass
Raise mcp_servers.workflow_deployments.update_server to admin: its body
carries isPublic, and a public server executes with no Sim credential, so a
write member could remove authentication from every workflow it publishes.
create_server already grants the same visibility at admin.
Pin the widened operations in registry tests — the six workflow-MCP ones,
the four workflow widenings, and files.extract_archive.
Reject secret fields on a credential that has no rotatable secret instead of
dropping them behind a 200, classify a non-transient provider 4xx as caller
error rather than a retryable outage, and reconcile a provider outage to 503
with Retry-After on all three surfaces.
Give /v2/meta a declarative principal policy through a new defineOperation
factory, carry the key expiry on the auth context instead of reading the
api_key table from the application layer, and make the impossible principal
branch an invariant error rather than a codeless 403.
Enforce the rollout-gate exemption at definition time, against the one
contract path it is reserved for, and remove the gate parameter from the
exported admission helper so the builder is its only door.
* fix(tables): bound the run-state sidecar, and close six v2 review findings
Enforces the 2 MiB run-state ceiling INSIDE the sidecar drain rather than
over its materialized result, refuses the unbounded query form paired with
it, and normalizes the two stored blobs the v2 surface publishes from bare
`as` casts.
- The run-state budget now travels into `loadExecutionsByRow`, which drains
row ids in bounded chunks and refuses before fetching the next one. The
post-hoc `requireBoundedRunState` walk is gone: it measured a spike that
had already happened, and re-serialized every entry to do it.
- Both row reads that accept `includeRunState` cap the page at
`V2_MAX_RUN_STATE_ROW_LIMIT`, and `POST /tables/{id}/query` additionally
refuses the flag paired with `limit: 0`.
- `runState.status` and the enrichment cascade blob are projected onto the
published shape before presentation; both were caller-reachable 500s on a
well-formed read.
- `sim tables bulk-delete` now gates behind `--yes`, and the CLI sweep that
should have caught it covers destructive non-DELETE forms.
- `POST /tables/{id}/restore` is idempotent (200, no audit) like its
knowledge sibling, and bulk folder selection deduplicates after resolution.
- The batch-update backstop keeps the looser Copilot ceiling and says so in
TSDoc: it is a backstop no surface reaches, because the contracts stop a
v2 caller at 1000 and the Copilot tool stops itself at 5000. Each caller
sees the bound that actually applies to it; neither surface's cap moved.
* fix(chat-deployments): close v2 chat review findings
Fixes the chat-deployments slice of the v2 review, several of which are
regressions the application-operation extraction introduced.
- Stop a `500` on schemaless JSONB: the response now declares a stored
shape without bounds and `toV2ChatDeployment` projects
`customizations`, `outputConfigs`, and `allowedEmails` onto it. The
request schemas keep `.strict()` and their bounds.
- Restore the specific validation message on `POST /api/chat` and
`PATCH /api/chat/manage/[id]`, and the deleted test that pinned it.
- Restore `chat_deployments.read` to workspace `admin`; the detail read
serves the visitor gate.
- Narrow the list projection so `chat_deployments.list` can stay a
`read` operation reachable by a workspace API key: `allowedEmails`,
`hasPassword`, and `customizations` are gone from the list entry and
available only from the admin-gated detail read. Serialized field by
field so a field added to the detail shape cannot reach the list by
default.
- Classify create-path failures: `performChatDeploy` carries an
`errorCode`, so an in-flight deployment is a `409` and an invariant
failure a `500` instead of every refusal being a `400`.
- Delete the callerless `GET /api/chat`, which served the encrypted
password column with no response contract.
- Propagate undeploy infrastructure failures instead of concealing them
as `404`, and return `ChatDeploymentView` from both delete paths.
- Name `CHAT_AUTH_MODE_NOT_PERMITTED` on the create path.
- Guard `getBaseUrl` inside `buildChatDeploymentUrl`, which otherwise
throws on a self-host with no `NEXT_PUBLIC_APP_URL`.
- Correct the published allow-list claim: a replacement `allowedEmails`
is applied after the auth-type clear, so it does survive.
- Assert `workspaceId` on the v2 detail routes and reconcile the
concealment TSDoc with what the error policy actually renders.
- Move `resolveActiveWorkspaceApplicationContext` to the workspaces
domain so chat-deployments no longer imports workflow application code.
* test(credentials): pin the reconciled provider-outage status
The internal route alone answered 502 where the v2 surface, the shared
status helper and `PROVIDER_OUTAGE_CODES`' own TSDoc all say 503. The
test pinned the divergence; it now pins the reconciliation, including the
`Retry-After` a 503 carries. Corrects a stale comment that still named
502 as the value callers see.
* fix(executor): restore cloud-storage Start files, dropped by the key rule
The ownership check accepted a key only when it could be parsed out of an
internal `/api/files/serve/...` URL. But the server-side uploader for run
inputs returns a *presigned cloud* URL whenever object storage is
configured, whose path is the bucket key — so every chat-deployment
attachment, API `files[]` payload and generic-webhook file field resolved
no key, and because normalization is all-or-nothing the entire `files`
input was dropped with no error. It passed locally and under vitest only
because the uploader falls back to an internal URL when no object storage
is configured, which is exactly why no test caught it.
The test is ownership, not provenance: a key is accepted when its own
layout names the executing workspace, whether it arrives directly or is
parsed out of the URL. Neither field has to be trusted, since both are
caller-authored and both are held to the same check. A payload whose key
and URL disagree is refused rather than resolved in the caller's favour —
a genuine uploader writes the two consistently, so only a forged pairing
is turned away.
`context` is now derived from the accepted key rather than read from the
payload or the URL's `?context=`, so an owned key can no longer be
labelled with a bucket its bytes do not live in — the hardening the
previous comment claimed but did not perform.
* fix(api): close four defects the fix pass introduced
- `resolveLatest` built a `RegExp` from the caller's block id and read the
registry with a bare lookup, so the catalog detail route — moved onto it
by the version-alias fix — routed around the `ownBlock` guard added for
exactly this. `GET /api/v2/blocks/%5B` was a `SyntaxError` 500 and
`.../constructor` an inherited function. Matched by string comparison
now, the way `tools/tool-ids.ts` resolves the same convention, and read
through `ownBlock`.
- The run-state byte budget was applied inside `queryRows` rather than at
the callers that publish it, so the first-party table grid — which reads
run state at five times the row limit and publishes no ceiling — turned
a large page into a hard failure, with an error naming a parameter it
does not expose. The budget is now an explicit option the public reads
pass and internal callers omit.
- Three graph-write CLI commands shipped ungated because the sweep meant
to catch them matched only the names already enumerated, so it could
never fail. It now forces every non-`GET` operation into a destructive
or non-destructive list, and the three carry confirmations.
- Unbinding `workspaceId` from the knowledge-documents cursor was right on
the merits and wrong in effect: the value is constant per sequence, so
removing it changed the fingerprint and refused every cursor already in
flight. Restored there; the chunks list is new in the same change and
keeps the cleaner reading.
* fix(api): resolve a detail read to a version the viewer can see
`getLatestBlockForViewer` took the newest version and then hid it, which
inverted the contradiction it was written to close: `slack_v2` and
`table_v2` are preview-gated while their v1 deliberately stays in the
toolbar, so an unrevealed viewer got a `404` on a detail read for a type
`GET /api/v2/blocks` was listing in the same breath. It now walks versions
newest-first and answers with the first one visible to that viewer.
Also:
- The chat password guard ran after `performFullDeploy`, so a request that
could never succeed burned a real workflow deployment version and then
answered 400. Its two sibling gate guards already refuse ahead of the
deploy; this one now does too.
- The Copilot sub-block serializer published the registry's own `options`
and `dependsOn` arrays by reference. Pre-existing, but the catalog
projection this parallels copies every array it publishes precisely
because they are process-global and shared by every request.
* fix(api): restore the locked read-modify-write and the password validator
Two findings verified as real regressions against staging, out of ten
checked — the rest were pre-existing, latent, or false.
`setWorkflowBlockEnabled` read the graph outside the row lock and wrote it
back inside a later transaction. The editor's own save takes that same
lock, so an autosave committing in the window was silently discarded: this
operation writes a whole graph, not a delta. The persistence primitive now
accepts a reader that runs after the lock is taken, and the toggle
re-reads and re-decides there. Its lock predicate is also scoped to the
workspace and to a live row again, so a workflow archived mid-flight is
refused rather than written.
The v2 chat-deployment contracts inlined their own password rule twice
instead of using `chatDeploymentPasswordSchema`, losing the refusal of a
whitespace-only password — which the internal contract rejects precisely
because it strands the deployment behind a password the visitor form will
not submit. Both sites use the canonical validator now.
* fix(api): one folder projection, one dynamic-provider list, honest 413s
- `toV2Folder` existed twice, and the second copy had been written without
the name/path invariant — so a row the list read refuses loudly would
have been served with a mismatched pair by the restore read. One
definition, guard included.
- The catalog projection restated `DYNAMIC_MODEL_PROVIDERS` and had
drifted by one member. Derived from the canonical list instead.
- The tables reads documented a `413` for run state that they cannot emit
— the budget became opt-in, and the row limit is the bound now — so the
claim is removed rather than declared. The workflow run read has the
opposite problem: it genuinely emits one, on a single file *or* the
run's inlined total, and declared neither. Now declared, and the
sentence covers both.
- Reclassifies the operations staging added into the destructive sweep, so
the triage stays exhaustive.
* fix(v2): classify storage and uniqueness failures, drop permanent file delete
- remove the permanent file-delete endpoint; the platform offers no such
action in the UI, and its manager wrote outside a transaction with no
storage accounting
- extract a generated document's text from its compiled artifact rather than
its generation source, matching the download path; a `.pdf` source was a
500 and a `.docx` source returned generator JavaScript as clean content
- report a run file whose object retention has already swept as 404 rather
than 500, on both the inline base64 read and the download stream
- report a knowledge tag that loses at a unique index as 409, naming whether
the slot or the display name is taken
- gate `workflows versions revert` behind a CLI confirm; it overwrites the
draft graph and was classified non-destructive
* feat(v2): report lint from both graph writes and add dry-run previews
- `PUT /workflows/{id}/state` now returns the same `lint` report as
`POST /operations`; an agent authoring a graph from scratch needs the
findings at least as much as one editing incrementally
- extract the report into one shared builder so the two writes cannot drift,
and one shared presenter so the wire shape is identical
- skip the credential/tool reference pass when the caller has no human
subject, rather than resolving it against the workspace billing owner:
that would misreport what the workflow can reach and disclose another
person's grants. `lint.notes` says when it was skipped
- add `?dryRun=true` to both graph writes: validates and lints, persists
nothing, records no audit, notifies nobody. A query param, not a body
field, since the body of a PUT is the resource itself
- CLI: a dry run no longer demands `--yes`; requiring confirmation to preview
a change teaches callers to pass `--yes` reflexively
- CLI: name the graph commands for their verbs — `workflows state get`,
`workflows state replace`, `workflows operations apply` — instead of the
derived `state list` / `state update` / `operations create`
- document when to use `rollback` vs `versions/{version}/activate` on both
* chore(docs): sync generated docs manifest for the new CLI pages
* feat(v2): add the missing workflow-MCP reads and align bulk naming
- add `GET /workflow-mcp-servers/{serverId}` and
`GET /workflow-mcp-servers/{serverId}/tools`. The resource could be
PATCHed and DELETEd but never read, and its tools could be published and
unpublished but never listed — the server list reports tool names only, so
nothing published the `workflowId` that addresses a tool for deletion.
Both mirror `mcp-servers` beside them, and carry that family's
workspace-API-key denial rather than the wider `mcp_servers.read` policy
- rename `POST /tables/bulk-move` to `POST /tables/move`, so tables matches
the shipped `files` resource exactly (`move` + `bulk-delete`)
- name the CLI commands for their operations instead of the derived
`... create`: `tables move`, `workflows move`, `tables bulk-delete`, and
`tables rows update-each` for the per-row batch, which sits beside the
existing filter-based `tables rows batch-update`
`POST /tables/{id}/rows/batch-update` keeps its name: a distinct payload per
resource is precisely AIP-234 BatchUpdate, and `bulk-` would have collided
one word away from the filter form.
* fix(v2): correct documented statuses and a caller-reachable 500
- duplicating a workflow into a locked destination folder answered 500:
`FolderLockedError` is a plain Error carrying `status = 423`, which the v2
error policy does not classify. Converted to OrchestrationError('locked')
at the application boundary, matching the bulk-move path
- restore workflow promised a 413 for an oversized folder tree that its
response list never published; the cap is real, so the status now is too
- move workflows and apply variables documented 409/423 they cannot emit:
every per-item lock and conflict is reported in `failed`, not thrown
- bulk download and delete knowledge tag can both 409 and did not say so;
cleanup tag definitions cannot and did say so
- apply workflow operations denies workspace API keys but never documented it
- the dry-run responses are not byte-identical to a committed write:
`needsRedeployment` describes the pre-write state and persistence warnings
cannot appear. Reworded rather than overclaimed
- read file text and get file upload are head-safe, so the "HEAD skips the
effect" sentence did not apply to them
* fix(v2): guard tag field-type changes and publish the 415 every body route can return
- `PATCH /knowledge/{id}/tags/{tagId}` accepted a `fieldType` incompatible
with the slot the tag already occupies. Slots are enumerated per field
type, so a text tag could be relabelled `number` and every later read
would interpret its values as the wrong type. Create checked this; update
now runs the same two checks
- derive `415` from the contract the way `413` already is: the JSON builder
answers UNSUPPORTED_MEDIA_TYPE for any body under a content type it cannot
read, so all 100 body routes could return a status none of them published
- give version activation its own result component instead of publishing it
as `RollbackResult`; the shipped rollback keeps that name
- correct descriptions that promised behaviour the code does not have: a
`processingStatus` field never returned, a `gmail_send` resolution example
that short-circuits, bucket widths that overflow the window, a bulk tag
save that relocates rather than overwrites, and per-server tool names
actually gathered under a page-wide budget
- drop 409 from three knowledge and upload reads that cannot emit it
* docs(v2): correct the upload transfer contract and 16 other published claims
The upload transfer step was documented as Sim's own data plane on every
deployment: "success is 204" and "a failure is the v2 error envelope". That
holds only when Sim stores objects itself. With object storage configured the
URL is the provider's presigned URL, so S3 and GCS answer 200 and Azure 201,
and a failure is the provider's XML — a client written to the old text reads a
successful cloud upload as a failure. Also states that part ETags do not need
retaining: completion takes no body because Sim lists the parts from the
provider itself.
Other corrections, all to shipped descriptions rather than behaviour:
- DELETE table and bulk-delete files archive rather than erase, and neither
said so; bulk delete also cannot emit the 409 it declared
- complete knowledge upload published a 402 only the create leg can raise
- billing status conceals a foreign workspace id as 404, not the 403 its
TSDoc and description both claimed
- audit entries null a folder's resourceId and strip folder ids from metadata
at every level; neither redaction was documented
- details=full adds the workflow summary to workflow runs only, never to job
runs; GET /logs folderPaths covers a subtree like its two siblings; getLog
now carries the retention sentence
- list secrets returns description too, and the logs and resources documents
described only part of what they serve
* improvement(api): consolidate the v2 surface and close seven defects
Endpoint consolidation:
- Fold POST /logs/query into GET /logs; add sortBy/sortOrder, cap the
comma lists, and move the list onto the shared keyset codec
- Fold GET /knowledge/archived into GET /knowledge?scope=archived,
matching files, tables, and workflows
- Re-home chat deployments as a singleton under the workflow they
belong to; keep the workspace-scoped discovery list
- Move the tag-definition writes off the document path onto
/knowledge/{id}/tags, where they already acted
- Nest the table export and dispatch reads under their parent table
Defects:
- Publish isPublicApi on the deployment read; it was write-only, so a
workflow could be opened to unauthenticated execution unauditably
- Stop publishing raw storage keys and an unusable URL in log files
- Classify a chat-identifier unique violation as 409 rather than 500
- Fall back to the root path instead of throwing when a knowledge
base's folder is archived
- Cap bulk-download at the ceiling it actually enforces
- Normalize variables through one helper on both graph write paths
- Fix a folder-name log filter that matched workflow names
Naming and gaps:
- Rename /files/{id}/extract to /unarchive, /rows/find to /rows/search,
/rows/batch-update to /rows/bulk-update, /columns/run to /dispatches
- Type the last six generic [id] path segments
- Add table folder restore and id-addressed dispatch cancel
* chore(audits): record the v2 catalog routes in the boundary baseline
* fix(api): accept a null chat password and correct three published claims
- performChatDeploy validated `password: null` as a password, so the
replace-shaped chat PUT answered 400 for every mode that owns no
password — public (the default), email, and sso. The declared payload
type has always allowed null, and the stored value is cleared by
authType regardless, so null needs no validation of its own. The route
test could not catch it: it mocks the orchestration module and pinned
the exact null the real guard refused.
- Redirect the two docs slugs this branch retired that were genuinely
published: findTableRows and runTableColumns.
- The table folder restore described an idempotent no-op for an already
active folder; it answers 404. Say so, and say where the path comes
from, since the tables folder list cannot yet report archived folders.
- Name the customizations exception to the chat PUT's replace semantics.
- A cursor-binding case used status=error, which is a level and not a
status, so it failed contract validation and never reached the cursor
check. Use an accepted value and pin the reason, not just the status.
* chore(cli): classify the new v2 chat operation as non-destructive
* feat(cli): expose canonical resource URLs
* fix(api): close three caller-reachable failures found by the final probe
- GET /files/{fileId}/text called parseBuffer unguarded, and parseBuffer
signals every failure as a bare Error that no v2 policy classifies. A
zero-byte upload or a mislabelled archive was an unhandled 500. Empty
bytes now answer empty text — a zero-length file has no text — and
unparseable bytes answer 409, matching the rendered-artifact resolver.
- GET /workflows/{workflowId}/state asserted write-side bounds over
stored data. workflow_blocks.name and .type are bare text() and the
realtime rename op accepts z.string(), so a block renamed past 255
characters made the workflow unreadable, and unrepairable, over v2.
The read shape now takes the same input/stored split the variable
schema already had. Stored subflow conditions are coerced in the
loader beside the existing numeric guards.
- GET /knowledge stamped scope into the cursor fingerprint
unconditionally. scope defaults to active and is new on that list, so
every cursor the deployed build handed out would have been refused
with a message saying the caller changed a filter they never sent.
Its siblings already carry the guard and the comment.
Also: POST /workflow-mcp-servers answers 201 like every other v2 create;
GET /logs/stats reuses the log list's entry ceilings; the execute and
resume routes install the media-type-aware 415 they publish; and a
rationale citing an endpoint that never reached the wire is corrected.
* fix(tables): carry the dispatch terminal timestamps through the stale sweep
Staging's abandoned-dispatch recovery builds its own `DispatchRow`, and
this branch had added `completedAt`/`cancelledAt` to that shape for the
id-addressed dispatch read and cancel. The merge was textually clean and
left the new mapping short two fields.
* fix(workflows): deny workspace API keys on the graph replace
`PUT /workflows/{workflowId}/state` stores blocks and their tool wiring
wholesale, but the policies deciding which of those a member may add —
the EE permission config and block visibility — take a human subject.
A workspace API key has none, and both substitutes fail open: the
billing owner is a different, typically less-constrained person, and
passing no user makes the permission lookup return null, which every
caller reads as unrestricted.
That made the replace a second graph-write door storing what its sibling
`POST /workflows/{workflowId}/operations` refuses, which denies workspace
keys for exactly this reason. Both doors now agree. Personal keys keep
the capability, so headless authoring is unaffected for a credential
that names a human.
* chore(api): drop the enrichment catalog endpoint
GET /api/v2/enrichments listed the code-defined table enrichments. The
per-row enrichment run detail stays; only the catalog read goes.
Removes the route, contract, response and query schemas, the semantic
operation, the use case, the projection module, its registry-boundary
entry, and the CLI command. The "not found" and "blank search" cases it
covered are repointed at the connector-type sibling so the shared
behaviour stays tested rather than deleted with it.
* improvement(api): make the workflow operations endpoint self-describing
Two things stood between this endpoint and a caller who has only the
published spec.
The accepted `params` keys existed only in the edit engine's source. The
spec said "the accepted keys depend on the target block type", so a
caller reading it could create a nameless empty block and nothing more —
while the Copilot tool catalog, over the same engine, has always spelled
out the envelope. That guidance now lives in the contract, shared by the
add, edit, and insert_into_subflow parameter schemas so the two surfaces
cannot describe one engine differently: `inputs` keyed by sub-block id,
`retry`/`triggerMode`/`advancedMode` beside it rather than inside it,
`connections` keyed by source handle, and `removeEdges` for dropping one
edge without restating the rest.
A `block_id` that is not already a UUID is replaced with a minted one,
and the mapping was computed and then dropped. A caller could not
reference the block it had just created except by re-reading the graph
and matching on name. The engine now returns it and the response
publishes it as `mintedBlockIds`, with the in-batch versus cross-request
rule stated in the operation description.
* fix(api): close the pre-merge scan's blocking findings
Docs and public wire, all of it permanent surface once released.
- Two published tag groups, Catalog and Meta, had no sidebar entry in any
locale, so six operations shipped unbrowsable. Added to all six, and a
test now fails when a published tag has no entry.
- deleteWorkflowChatDeployment pointed callers at DELETE on /deployment;
the undeploy verb is on /deploy.
- PUT /state still promised workspace API keys a degraded lint pass after
the operation began rejecting them outright. It also lived in a
single-quoted string, so the shared clause would not have interpolated.
- POST /tables/move took targetFolderPath as nullable-but-required, which
rendered the CLI flag as a required `<json|@file>`: `--to /Archive`
failed to parse and omitting it failed outright, while the docs said
"omit for root". Now optional and a plain string, matching
POST /files/move; the route supplies the null the use case wants.
- Table dispatch status and row run state published `cancelled` beside
imports, exports, and job state publishing `canceled`, and the note
explaining the split was wrong about its own sibling. Both new schemas
now publish `canceled`; the stored column is unchanged and mapped at
the presenter. The shipped `cancelled` count field is left alone.
- applyWorkflowVariables can answer 423 and both workflow-MCP deletes can
answer 409; none declared it. Hand-assembled error lists replaced with
the shared sets.
- /files/{fileId}/unarchive became /unzip. `extract` reads as "extract
text" and `unarchive` reads as the inverse of restore on a resource
where archived means soft-deleted; unzip collides with neither and is
what the implementation calls itself.
* improvement(cli,docs): finish the naming and clear the enrichment leftovers
- Four single-record GETs derived to `list` while returning one thing:
meta, a workflow's chat deployment, log stats, and file text. Renamed to
`meta status`, `workflows chat status`, `logs stats`, `files read`,
matching the `workflows deployment status` correction already in this
branch. None of the old spellings shipped.
- Dropped five `renamedFrom` aliases pointing at spellings that never
existed, each of which built a hidden command and a permanent
deprecation warning for argv nobody could have typed. `tables rows find`
keeps its alias — that one really shipped.
- Removed the enrichment catalog from three prose sites left behind when
the endpoint went: the resources spec description, its Catalog tag, and
the contract and pagination-test comments.
- Documented the six new command groups in the CLI index table. That page
is a guide page, so the docs staleness check cannot flag it.
- Published the `customizations` exception to the chat replace semantics.
It was in the route TSDoc and invisible to every caller reading the
spec, which is where the claim "Replace, not merge" is made.
* improvement(cli): use batch- for the tables bulk delete, matching its siblings
The CLI renames a bulk form only when it would collide with its singular
sibling, and uses AWS's `batch-` prefix when it does — `files
batch-delete`, `tables rows batch-delete`, `knowledge chunks
batch-update`. `tables delete` exists, so `tables bulk-delete` was that
same rename reaching for the other word, and the only `bulk-` command on
the surface. No `bulk-` CLI command has ever shipped, so this costs
nothing now and would be permanent later.
`files bulk-download` keeps its name: there is no `files download` to
collide with, and it is one archive rather than N operations. Its config
block now says why it exists at all, since the command is never built —
the builder skips non-JSON response modes, but the contract sweeps still
read the entry and require the folder-path field to be marked.
* fix(workflows): allow operations on blockless drafts
* feat(workflows): add manual and run-from-block execution
* fix(cli): update workflow run description test
* fix(tests): align fixtures with current contracts
* fix(api): derive chat activity from workflow deployment
---------
Co-authored-by: Theodore Li <theo@sim.ai>
|
||
|
Waleed
|
bdda083f93 |
refactor: consolidate three drifted copies, close a gate blind spot, fix stale docs (#7062)
* refactor(workflows): one owner for new-workflow sort order The same ~35-line query — parent condition for workflows and folders, two parallel min(sortOrder) reads, fold to a min, subtract one, fall back to 0 — existed three times: lib/workflows/utils.ts inline in createWorkflowRecord lib/workflows/orchestration/... as a file-private nextWorkflowSortOrder lib/workflows/persistence/duplicate inline, inside the duplicate transaction The first two are character-identical modulo the table alias. The third had drifted: it omits isNull(workflow.archivedAt), which the other two apply, so a folder whose lowest-sortOrder workflow is soft-deleted positioned a *duplicate* differently from a *create*. The folder-side query agrees in all three, which marks it as a copy-paste slip rather than intent. Promotes the helper to lib/workflows/sort-order.ts, taking an optional DbOrTx so the duplicate path can keep reading inside its transaction. Its own module rather than utils.ts because duplicate.test.ts and workflow-lifecycle.test.ts both mock '@/lib/workflows/utils' wholesale — from a separate module the real query still runs under those suites, so their existing sort-order assertions keep their meaning and needed no edits. Note the archived-row behavior itself is not unit-testable here: the shared dbChainMock does not evaluate WHERE predicates. The guarantee is structural — one query builder instead of three means the predicate can no longer drift. * fix(ee): case-fold the stored integration allowlist ee/access-control re-implemented the allowlist intersection instead of calling intersectIntegrationAllowlists, and lost the case-folding: normalization only happened on the envAllowlist !== null branch, so with ALLOWED_INTEGRATIONS unset a stored config went through untouched. Callers compare against blockType.toLowerCase(), so a stored 'Slack' failed to match 'slack' and the block was denied. The access-control UI writes block.type directly and block types are lowercase, so this is not reachable from the UI — but allowedIntegrations is a bare z.array(z.string()) on the wire, so any API client can store mixed case. Replaces the fork with the shared helper. Adds two tests; the first fails against the old code. * fix(queries): forward the abort signal to getFullOrganization useOrganization destructured `signal` from the queryFn and passed it to fetchOrganization, which named the parameter `_signal` and never used it — so the org detail fetch could not be cancelled. Switching orgs rapidly left every prior request in flight, free to resolve out of order. Better Auth takes cancellation two ways and both are already used here: fetchOptions on the params object (session.ts:21) and a second argument (admin-users.ts:129). Uses the former. This was the only `_signal` under apps/sim/hooks. The existing transition test asserted the exact call shape, so it now asserts intent via objectContaining. Adds a test for the signal itself; it fails against the old code. * fix(canvas): select from useWorkflowRegistry instead of subscribing whole check-zustand-v5-selectors matched /use[A-Z]\w*Store\(/, and exactly one Zustand store in the repo is not named with a Store suffix — useWorkflowRegistry. So the store behind the canvas went unchecked, and two bare whole-store subscriptions had accumulated in the action bar, re-rendering it on every registry mutation (clipboard, hydration, pendingSelection, activeWorkflowId). Every other call site in the repo already uses a selector. Widens the pattern to (?:Store|Registry) and fixes both call sites. The widened gate reports these two and nothing else, so there is no cleanup tail. * docs: correct comments that name symbols which no longer exist Each of these points a reader at an identifier that is not in the repo: - table/import-data.ts, table/service.ts — `acquireTablePositionLock` and `nextAutoPosition` were removed with the service.ts split; the surviving lock is `acquireRowOrderLock`, which import-data.ts already imports and calls. service.ts's mention is load-bearing: it exists to tell the reader which other lock this one mirrors, for lock-ordering. - resources/orchestration/restore-resource.ts — named `performRestoreFolder` (the callee is `restoreFolder`) and described a `'workflow'` default it falls back to. There is no such default: resourceType is required and the config lookup is a bare index. Describing a fiction is how a future reader talks themselves into relaxing the total Record to a Partial. - knowledge/search/queries.ts — cited apps/docs/app/api/chat/route.ts, deleted with Ask AI. The k=60 it pins against now lives in the docs search route. - rate-limiter/hosted-key/queue.ts — documented a `waitForHead` method the class does not have; the queue exposes `checkHead` and the polling loop is private to the consumer. - logs/log-views.ts — a "Level 1.5 / 2 / 3" scheme that appears nowhere else; the real contract is the five named views. Dropped, and the three banner rules with it (CLAUDE.md bans banner separators). Comment-only apart from the log-views banners. * refactor(ui): derive three values instead of storing or memoizing them - import-modal: browserId and profileId were state corrected by two effects when a reload dropped the selection. That commits and paints one frame in which the profile still belongs to the previously selected browser — and Import is enabled during it, submitting via a `profiles.find` that searches every browser's profiles. Both now fall back during render, and `selected` searches only the current browser's profiles. Covered by the existing 'never leaves a profile selected that belongs to another browser' test. - workflow.tsx: isWorkflowEmpty was a second useMemo over the same [blocks] dep computing exactly !hasBlocks, allocating its own Object.keys array. Both feed primitives, so neither memo bought identity stability. - thinking-loader: an effect seeding cycleVariant whenever variant is defined, which `shown = variant ?? cycleVariant` can never read. On the one transition where cycleVariant becomes visible (variant going undefined) the cycling effect assigns it in every branch — settle, reduced-motion, and tick — in the same flush, so the seed was never observable. * fix(review): exclude deleted folders from sort order; make the allowlist tests real Three findings from cubic on #7062, all valid. 1. nextWorkflowSortOrder consulted the folder minimum without excluding soft-deleted folders. Because the helper returns min - 1, a deleted folder holding the lowest slot ratchets the floor down permanently — the same class of bug as the archived-workflow one this PR set out to fix, on the other half of the query. lib/folders/orchestration.ts already documents this exact rationale for the folder-creation side of the same algorithm, and the uploads folder manager filters it too; this was the outlier. 2. The two new allowlist tests did not call setEnterpriseOrgWorkspace(), so resolution never reached the group queries and validateBlockType returned early. They passed against the unfixed code when run in isolation and only appeared to fail in a full-file run, where mock state leaked from earlier tests. Verified with 'vitest -t': both now fail without the case-folding fix and pass with it. 3. The restore-resource comment this PR rewrote was itself wrong. A Partial map does not defer the failure into the cascade — the lookup widens to FolderResourceType | undefined and the error lands on the restoreFolder call site. Reworded to say that, and why keeping the check at the mapping matters. |
||
|
Waleed
|
3251bf13a6 |
fix(ci): make the utils gate see the wrapped forms of what it bans (#7047)
`check-utils-enforcement.ts` scanned line by line, and every idiom it bans is a
multi-token expression the formatter wraps at 100 columns. So it printed
`✓ No banned patterns found` while eleven files carried the wrapped form of
e instanceof Error ? e.message : fallback
which CLAUDE.md mandates `getErrorMessage` for. The same class as the two blind
spots already fixed in check-react-query-patterns.
Patterns now run against the whole file, with match offsets mapped back to line
numbers by binary search over the line-start table — verified against every
offset of a multi-line fixture.
Eight of the eleven are now `getErrorMessage(error, fallback)`.
`auto-layout-utils` collapses a redundant `instanceof ApiClientError` arm on the
way, since that class extends `Error`; `upgrade.ts` keeps its `rawBody ?? message`
arm, which the helper cannot express, and only its tail collapses.
The other three stay, because the helper genuinely does not fit, and they carry a
`// utils-lint-allow: <reason>` annotation — the same escape hatch
check-react-query-patterns already has, which this gate lacked:
- the two auth routes return the message to an unauthenticated caller, so a
non-Error throw must surface the fixed copy rather than its own text.
`getErrorMessage` passes a thrown string straight through, which is the
disclosure shape #7015 closed.
- `e2b.ts` probes E2B's own error shape — a record-like carrying `message` or
`value` — which has no equivalent.
An annotation with no reason does not suppress, so the hatch cannot be used to
silence a finding without saying why.
Also corrects the header, which claimed Biome's `noRestrictedImports` covers
"crypto named imports". It lists only `nanoid` and `uuid`. Named crypto imports
pass both gates deliberately — server code building cipher IVs wants node's
crypto, not the cross-context wrapper — and the comment asserting otherwise would
mislead the next person auditing this.
Verified the gate can fail in both directions: reintroducing a wrapped ternary
reports it, and emptying an annotation's reason reports it too.
|
||
|
Waleed
|
67fc2aeb30 |
fix(ci): require a default export before treating a file as a route entry (#7028)
* fix(ci): require a default export before treating a file as a route entry Follow-up to #7026, which added `error.tsx` to the entry filenames and with it picked up `[workspaceId]/components/error/error.tsx` — named like a boundary, and not one. It exports `ErrorShell` and `ErrorState` for the thirteen real boundaries to use; Next would reject it as a boundary for having no default export. Counting it inflated the coverage number and would have recorded a shared component in the graph-weight baseline as though it were a route. The filename was never the right test. Every convention-composed entry must default-export the thing Next renders, so that is the discriminator now. Entry count goes 60 → 59, and all thirteen real `error.tsx` boundaries still walk. Also adds `template.tsx` and `default.tsx`. Neither exists under `app/workspace` today, so this changes nothing now — but the enumeration claims to cover what Next composes, and leaving two out makes that claim false the day someone adds one. Both raised in review on #7026 (Cursor and Greptile respectively); I merged before reading them, so this lands separately. * fix(ci): count every form that declares a default export `export { default } from './page'` is a valid Next entry and the regex required `as default`, so such an entry would have dropped out of the walk and skipped both the registry gate and the graph-weight ratchet — silently, which is the dangerous direction for a discriminator to fail in. Latent rather than live: the form appears once under `app/workspace`, in a barrel, not in an entry filename. Four forms now count — `export default …`, `export { default } from`, `export { default, … } from`, and `export { X as default }`. `export { default as X }` still does not: it re-exports another module's default under a name and leaves this one without one. Verified all ten variants, including that last distinction. Raised by both Cursor and Greptile on #7028. |
||
|
Waleed
|
1aa714cf91 |
fix(ci): stop failing the API audit for adding a compliant route (#7027)
`totalRoutes` sits at 1162 and the repo has exactly 1162 routes, so the next
route fails CI whether or not it is contract-backed:
API validation audit failed:
- route count increased from 1161 to 1162
The invariant worth holding is that every route has a contract, and
`nonZodRoutes` states exactly that. It is 0, and it rises the moment a route
ships without one — `zodRoutes === totalRoutes` today, so the total adds no
information the other two counters do not already carry.
What it adds instead is a habit. The only way past it is editing the number, and
this file holds seven other baselines that work only while nobody bumps a
baseline casually.
The total is still printed; it is no longer a failure. Verified both directions:
a compliant new route passes where it previously failed, and a route without a
contract still fails through `nonZodRoutes`.
|
||
|
Waleed
|
f37c24e541 |
fix(ci): walk every route entry the workspace app composes, not just pages and layouts (#7026)
The tool-registry guard collected `page.tsx` and `layout.tsx`, and Next composes three more entries by convention: `error.tsx`, `loading.tsx`, `not-found.tsx`. Twenty-six exist under `app/workspace` and none was walked. `error.tsx` is always a Client Component — Next requires it — so a registry edge there reaches the browser bundle exactly as one from a page does. Coverage goes from 34 entry graphs to 60. Nothing new is reported: the hole was unexploited, and closing it costs nothing. The root deliberately stays at `app/workspace`. Widening it to `app` reports `(interfaces)/resume/[workflowId]/[executionId]/page.tsx`, a Server Component (`runtime = 'nodejs'`, `force-dynamic`) whose `PauseResumeManager` import resolves server-side and never reaches a client bundle. The guard cannot distinguish server from client entries, so it stays where its premise holds. |
||
|
Waleed
|
cc16d23d90 |
fix(react-query): close the lint's blind spots, and the drift they hid (#7020)
`check-react-query-patterns.ts` reported a clean strict zone while never
looking at part of it. Two gaps in one regex:
`\buseQuery\s*\(` does not match `useQuery<Row[]>({ ... })` — a type argument
sits between the name and the paren. Twenty query calls carry one, ten of them
inside the zero-tolerance zone, so that zone's "0 violations" was partly a
statement about what the scan could see.
`useQueries` was absent from both the call pattern and the file pre-filter,
where `\buse(Query|...)\b` rejects it on the trailing `s`. All sixteen call
sites were unscanned, and its options nest one level deeper — inside a
`queries` array — so it needs its own pass per entry rather than one that reads
the wrapper and takes a single `staleTime` anywhere inside as covering them all.
With both closed, three real violations surfaced:
- `knowledge-base-selector` served `knowledgeKeys.detail(id)` with an inline
`60 * 1000` while `useKnowledgeBaseQuery` serves the same cache key from
`KNOWLEDGE_BASE_DETAIL_STALE_TIME`. The two agree only by coincidence, and
TanStack resolves staleTime per observer, so tuning the constant would have
left this component on the old window for the same entry.
- The same call dropped the `AbortSignal`, which `fetchKnowledgeBase` accepts.
- `use-permission-config` gave `staleTime` as a literal with no named constant.
The new `stale-time-literal` category makes the second half of the CLAUDE.md
rule enforceable — it required a named constant, and only the presence of
`staleTime` was ever checked. `0` is exempt: it is the sentinel for "always
refetch", not a window anyone keeps in step with a prefetch.
Verified the new rules can fail by reverting each fix and watching the audit
report it, then restoring.
|
||
|
Waleed
|
95d08d2a49 |
fix(ci): stop the migration safety audit from passing on a branch it never read (#7022)
* fix(ci): stop the migration safety audit from passing on a branch it never read
The zero-downtime audit reports the same empty file list for 'this branch adds
no migrations' and 'I could not diff against the base', and the second prints
as `✓ No new migrations to check` with exit 0. Reproduced on this checkout:
$ bun run scripts/check-migrations-safety.ts origin/does-not-exist-branch
✓ No new migrations to check. exit=0
`changedMigrationFiles` returned `[]` whenever `git diff` failed, with a comment
deferring the decision to the caller — but the caller only recognised a missing
git binary (`git rev-parse HEAD === null`), never an unusable ref.
CI supplied exactly that input. `git fetch --depth=1 … 2>/dev/null || true` hid a
failed fetch, leaving `origin/<base>` absent, so a PR adding a destructive
`DROP COLUMN` would clear the only guard on production DDL with a green check.
Two halves:
- The audit now distinguishes the cases. Absent git is still the one legitimate
skip and is checked before the diff; a diff that fails with git present raises
`BaseRefUnusableError` and exits 1.
- The fetch is its own step with no `|| true`, so a failure fails the job. Depth
stays 1: without a merge-base the audit diffs the two tips, which under
`--diff-filter=AM` is exactly the migrations new on the branch.
Covered by a test that runs the script end to end, since the defect was in the
exit code rather than in any function's return value. Verified it fails when the
throw is reverted to `return []`.
* fix(ci): fetch the base ref once, and stop swallowing the failure
The same `git fetch --depth=1 … 2>/dev/null || true` appeared in both base-ref
audits. Fixing only the migration one would have left the identical defect a few
steps above it.
Neither audit can tell an absent base ref apart from a branch that changed
nothing. The block-registry check at least degrades to a visible
`⚠ Could not diff against base ref — skipping`; the migration audit printed
`✓ No new migrations to check` and exited 0.
Both now share one fetch step that fails the job when it fails.
|
||
|
Waleed
|
66a9a18d1f |
improvement(perf): eight verified cuts to workspace cold-load JavaScript (#6996)
* improvement(perf): eight verified cuts to workspace cold-load JavaScript Second round of load-time work, adversarially verified for strict behaviour preservation before implementation. Each item is an import-graph fix — none changes what renders, when it renders, or any data path: - knowledge/[id] imported one modal through the [documentId] components barrel, which also exports the chunk editor and therefore js-tiktoken (~2.5 MB gzip of BPE tables) on a route that never edits chunks. Deep import. - prepareBlockState moved out of stores/workflows/utils.ts into its own module. It is the only function there needing the block registry and the generated tool-outputs artifact (~476 KB gzip), and utils.ts is reached by the persistent shell — so every workspace route paid for a canvas-only helper, including a module-scope JSON.parse of a 5.4 MB string. - ExecutionSnapshot (the frozen-canvas modal) is now React.lazy behind its interaction gates, per the code-splitting procedure in sim-imports.md: deep import, dead barrel re-export deleted, sibling imports in log-details deepened to break the parent->child barrel cycle, local Suspense at both render sites. Takes ~7.6 MB of source off logs hydration. - The api contracts barrel no longer re-exports ./tools, ./selectors, ./v1, or ./demo-requests (~58 KB gzip of Zod schema construction on every route). Zero importers used the barrel path for any of them. - createCsvParser (streaming csv-parse, a Node Transform) moved to a server-only module so its stream polyfill leaves client bundles. Deliberately not re-exported from the lib/table barrel. - jszip is dynamically imported at both remaining static call sites (skill zip extraction, pptx parsing) — both already-async, user-triggered paths, mirroring the existing pattern in workflow import-export. - The desktop local-filesystem tool executor is dynamically imported in use-chat; a chunk-load failure now reports an error completion so the server-side tool call settles instead of hanging. Production build, JS downloaded before the load event, vs the previous release: /home 4.44 -> 3.87 MB /logs 4.44 -> 3.64 MB /knowledge 4.22 -> 3.68 MB /tables 4.17 -> 3.61 MB /files 4.68 -> 4.10 MB /w/[id] 4.80 -> 4.67 MB /home total after idle prefetch: 8.15 -> 5.52 MB The lazy snapshot was exercised end-to-end: its chunk loads when a log detail opens (off the route's cold path, warm before the View Snapshot click) and the modal renders without errors. Boundary baseline retightened. * improvement(logs): contain snapshot chunk-load failures and settle the local-fs tool on recovery failure Review round: wrap both lazy ExecutionSnapshot render sites in a small error boundary (Suspense handles the lazy import's pending state, not its rejection — a failed chunk load would have unwound to the route boundary and replaced the logs page over an optional modal; mirrors PreviewErrorBoundary), and contain rejections inside the local-filesystem executor's load-failure recovery so a failed completion report degrades to a log instead of an unhandled rejection. * fix(logs): recover cleanly from snapshot chunk failures * fix(logs): preserve snapshot modal while loading |
||
|
Vikhyath Mondreti
|
cbf3aad569 |
fix(workflows): scope the canonical sub-block index to the active surface (#6990)
* fix(workflows): scope the canonical sub-block index to the active surface A block that is both an action and a trigger holds one `subBlocks` array — its own fields plus its trigger's, spread in after them. The two sets routinely share a `canonicalParamId` under different ids, so indexing them together collapses a trigger field into an action pair whose `basicId` it can never be. Every group-relative question about that field then answers for the dormant surface. The serializer was never affected: `shouldSerializeSubBlock` drops the inactive surface before the canonical collapse reads it, so it resolves against a value map the dormant surface cannot appear in. Every other caller resolves against the block's full value map, so the scoping has to live in the index. - add `getCanonicalSubBlocksForSurface` / `buildCanonicalIndexForSurface`, and move the three sites that already had the filter inline onto them - `getCardSubBlocks` derives its own index instead of accepting one; it already took `triggerMode`, and accepting an index is what let all three callers pass one built for the other surface - scope the remaining consumers that resolve against a full value map: the canvas card, autolayout, both preview surfaces, the dependsOn gate, the canonical value hook, reactive conditions, and the copilot selector lint - keep a canonical group with no advanced member out of the legacy `advancedMode` path, which deleted its basic member and republished nothing - merge legacy type-scoped tool modes as a baseline under index-scoped ones, so the first re-toggle stops reverting the ids the user has not touched * fix(workspace-forking): scope the canonical gates to the block's active surface `createCanonicalModeGates` indexed a block's whole `subBlocks` array, so on a mixed action/trigger block a trigger field sharing a `canonicalParamId` with an action pair was read as a member of THAT pair. Being neither its `basicId` nor in its `advancedIds`, `isDormantMember` answered true the moment the shared mode resolved to advanced — and a fork acts on that by clearing the value, so a configured trigger field was silently wiped on fork/sync. Reachable without any explicit toggle: a block configured as an action with a manual id and then switched to trigger mode leaves the pair's value heuristic resolving to advanced on its own. - `createCanonicalModeGates` takes the surface and scopes its index - thread `triggerMode` through `RemapForkContext`, `SubBlockTransform`, `clearDependentsOnRemap`, `collectClearedDependents`, the reference scanners, and the promote cleared-ref collectors - nested tool params and the dependent scan are unchanged: a tool is always the action surface, and the dependent scan already narrows its configs * fix(workspace-forking): keep the dormant surface classified as it was before scoping Surface scoping decides canonical membership for the ACTIVE surface. Applying it to every key also re-classified the dormant surface's own values: they stopped being dormant members, which meant the remap no longer cleared them AND started detecting them as references — turning a stale action selector on a trigger-mode block into a mapping requirement that can block promote/sync. The gates now pick the index per key: the scoped one for anything the active surface defines (the fix — a trigger field gets its own group instead of being read as a stranded member of an action pair), the whole array for everything else, which is byte-for-byte the pre-scoping behavior. Also adds `check:canonical-index`, an audit that fails any call building a canonical index off a config's whole `subBlocks`, or calling the fork gates without a surface, unless annotated with why. This defect shipped three times in three subsystems; the 14 sites that legitimately mean one fixed surface now say so at the call. * fix(audits): stop the canonical-index guard flagging its own source The audit holds `buildCanonicalIndex(` and `createCanonicalModeGates(` as string literals to search for, and its own regex matched them — the arg-count rule then fired on the literal. It passed locally only because the file was still untracked when it ran, so `git ls-files` did not list it; committing it made the audit scan itself and fail CI on the first run. Exempts the audit's own source alongside the module that defines the primitives. Verified the guard still fails on both regression shapes after the exemption. |
||
|
|
c26529a82e |
feat(bitbucket): add repository webhook triggers (#6934)
* feat(bitbucket): add repository webhook triggers * fix(bitbucket): harden webhook trigger delivery * fix(bitbucket): address final trigger review * chore(bitbucket): address review conventions * fix(bitbucket): harden triggers and connector sync --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Waleed
|
d1b01849a8 |
improvement(perf): cut server-only and unused code out of the workspace client bundles (#6975)
* improvement(perf): cut server-only and unused code out of the workspace client bundles Every workspace route shipped JavaScript it never executes. Four independent import edges, each fixed by moving a symbol rather than changing behaviour: - js-tiktoken's BPE rank tables (5.4 MB source / 2.5 MB wire) reached the workflow editor because the tokenization barrel re-exported the exact counters alongside the character heuristics. Split into lib/tokenization/accurate.ts, which the barrel no longer re-exports. - crypto-browserify (~105 KB gzip, all 26 workspace routes) came from the Salesforce and Gong triggers importing webhook provider modules that reach node:crypto through @sim/security. The two symbols they actually needed are now in crypto-free modules. - tables, files and knowledge each imported one dependency-free hook from the sidebar-hooks barrel, whose other exports reach the 5 MB generated tool-metadata artifact. Deep-imported per the code-splitting rule in sim-imports.md. - lib/workflows/subblocks/display.ts imported a string constant from a React module under app/, inverting the app/lib layering. Moved to lib/. Also adds a loading boundary to chat/[chatId]. Without one, a dynamic route is prefetched as nothing, so clicking a chat held the previous chat on screen for the whole server round trip. Measured on a production build, JS downloaded before the load event: /w/[workflowId] 7.38 MB -> 4.80 MB (-35%) /logs 4.57 MB -> 4.44 MB /knowledge 4.35 MB -> 4.22 MB /home 4.57 MB -> 4.44 MB crypto-browserify no longer appears in any shipped chunk. The tool-registry boundary baseline is retightened so the reclaimed graph weight cannot silently regress. * improvement(sidebar): move useContextMenu to shared hooks Review flagged the four workspace routes deep-importing `useContextMenu` from the sidebar's hooks barrel as a barrel-convention violation. Fair — the code-splitting exception in sim-imports.md is written for `lazy()` splits, and these are static imports. The hook was in the wrong place to begin with. It is entirely generic — no sidebar-specific references, just right-click state and positioning — and nine consumers across tables, files, knowledge, home, the terminal and the preview editor already reached across features to get it. Moved to `@/hooks`, the repo's shared-hooks location, and every consumer including the sidebar's own now imports it from there. This satisfies the barrel convention rather than making an exception to it, and keeps the graph win: tables, knowledge and files stay off the sidebar barrel's path to `stores/workflow-diff -> serializer -> tools/metadata`, unchanged at 20.19 / 20.89 / 21.37 MB of reachable source. |
||
|
Vikhyath MondretiandClaude Opus 5
|
fe0b92bb6d |
feat(secrets): show where a secret is referenced, beside its usage log (#6947)
* feat(secrets): show where a secret is referenced, beside its usage log
"See usage" answered who has run something with a key. It could not answer the
question a rotation actually starts from — where is this wired in — because a
secret four blocks depend on but nothing has executed yet has no usage rows at
all, so the panel read "This secret has not been used yet" for a live key.
The usage view now carries two tabs. Logs is the existing trail, unchanged and
still the default, since that is what the header action has always opened.
References is new: the blocks that name the secret as {{KEY}}, grouped under
their workflow, then the custom tools and MCP servers whose own bodies carry it.
Detection is the workspace-fork remapper's. remapSubBlocks already walks nested
tool-input params, resolves canonical basic/advanced pairs, and skips dormant
and condition-hidden members, so calling it per block inherits every rule a
fork already obeys. Only the aggregation is new: scanWorkflowReferences
collapses its output to unique (kind, sourceId) pairs and discards the workflow
— right for building a mapping table, wrong for locating a key. Nothing under
ee/workspace-forking changed.
- Candidates come from strpos(sub_blocks::text, name) > 0, deliberately not
LIKE: `_` is a LIKE single-character wildcard and nearly every env key
contains one, so SB_ACTION_ROUTER_SECRET would match text it does not occur
in. The prefilter can over-match but never under-match; the scanner decides.
The plan is an index scan on workflow by workspace, nested-looped into
workflow_blocks, so cost tracks the workspace rather than the table.
- Scope gates the read but does not narrow it. A {{KEY}} names a key, not a
scope, so the same sites answer for a workspace secret and the personal one it
shadows; narrowing here would report a personal secret as unreferenced the
moment a workspace variable of the same name existed.
- References reports one field per block, not a list. The remapper dedupes a
block's references by (kind, sourceId), so a block naming the secret twice
yields one entry — the type says so and a test pins it, because the row
renders that field as its whole description.
- Reads live state, not deployed: a draft workflow referencing the key must
show. Blocks are capped and the cap is reported as `truncated` rather than
silently trimming the list.
- Authorization is the existing usage gate, renamed requireSecretTrailReadAccess
and shared verbatim, so the two tabs can never disagree about who may look.
UI is existing primitives only — ChipModalTabs for the strip, DetailSection per
workflow over RESOURCE_LIST_STACK rows, IntegrationTile for the block glyph so a
block reads here as it does on an integrations row, SettingsEmptyState for the
gates. No new component, no new class.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): close the reference-scan scope bypass and bound its output
Review round 1.
- use-cases.ts: `scope` was a caller-controlled assertion the reference scan never
narrowed by, so `scope=personal` returned from the shared gate before any check
and handed any workspace member the admin-gated reference map for any workspace
secret. The usage trail can trust that scope because it filters the read by
`secretOwnerUserId`; a name-based workspace-wide scan cannot. References now
authorize on what the NAME resolves to — a workspace secret under that name is
admin-gated outright, and absent one the caller must actually hold a personal
secret of that name, which also stops a member enumerating arbitrary names.
`scope` is dropped from the input, the contract, the hook and the query key
rather than merely ignored: a parameter that does not exist cannot be asserted.
The trail gate keeps its old name and a note saying why only a scope-narrowed
read may reuse it.
- scan.ts: the prefilter matched the bare name, so `API_KEY` also read every block
holding `{{API_KEY_TEST}}` or the words "the API_KEY value" — and those false
positives counted against the row cap, so on a workspace with enough of them
genuine references sorted later were never read at all. It now matches the
reference syntax (`{{name}}`, with the whitespace ENV_REF_PATTERN allows), so a
candidate is a real occurrence and the cap means what it says. A name outside the
env-key charset short-circuits, which is also what makes it safe to inline into
the regex unescaped. Verified against a real workspace: the exact key still
returns its 16 blocks, its prefix now returns 0 where it previously matched all
16, and a metacharacter name touches no query.
- scan.ts: capping tool and server ROWS did not bound the output — one MCP server
emits an entry per matching header plus one for its url, so 200 rows could expand
past the contract's 400-entry bound and make the route reject its own response,
turning a successful scan into a 500 and the tab into "Could not load
references." Emission now stops at the bound and reports `truncated`.
- secret-references-panel.tsx: the empty-state early return preceded the truncation
banner, so a capped scan that filtered everything out claimed the secret was
unreferenced. Both paths now share one note, and silence from a capped scan reads
as absence of evidence rather than evidence of absence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): cover unicode whitespace, legacy keys, and the shadowed-personal tab
Review round 2 — all three follow from round 1's own fixes.
- scan.ts: the syntax prefilter anchored on `[[:space:]]`, but the two engines
disagree about what whitespace is. `ENV_REF_PATTERN`'s `\s` accepts U+00A0,
U+202F and U+3000; Postgres `[[:space:]]` matches only the ASCII set. So a value
pasted with a non-breaking space inside the braces is a reference the executor
resolves and the prefilter silently dropped — the one failure direction this
feature must never take, since the answer it gives is "unused, safe to delete".
Anchoring on `[^[:alnum:]_]` instead accepts every whitespace encoding while
still rejecting a longer key on either side, and needs no code-point list that
could drift. It can admit a non-reference like `{{-NAME-}}`; that costs one
candidate row, and the scanner re-checks every candidate regardless. Erring loose
here is deliberate. (Greptile's `{{\tAPI_KEY\t}}` example was already handled —
tab is ASCII — but the unicode half of the finding was real.)
- use-cases.ts: the gate read `keyAccess.knownKeys` as "a workspace secret exists
under this name", but that set only covers names with an `env_workspace`
credential row. A legacy value written before the ACL existed has no row and
still wins at run time, so it fell through to the personal branch and handed a
non-admin the reference map for exactly the oldest keys. It now reads the
authoritative `workspace_environment.variables` map through a new
`hasWorkspaceEnvValue`, which is documented against `knownKeys` so the two are
not confused again. `getWorkspaceEnvKeyAdminAccess` keeps its existing contract —
its `knownKeys` still answers the ACL question its other callers ask.
- secret-references-panel.tsx: a personal secret shadowed by a same-named workspace
variable could open the view (its owner may read their own Logs) but References
always hit the workspace refusal and rendered a generic load error — a tab
offered in a state where it cannot succeed. The refusal is correct; the tab now
states the shadowing instead of asking for a map it will be denied, reusing the
wording the detail page already shows. No request is made in that state.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): re-check the reference gate's volatile input after the scan
Review round 3.
The name-resolution gate reads whether a workspace value exists, then scans. A
workspace secret created between the two makes the map now in hand admin-gated,
so a personal owner could receive it without workspace-secret administration.
The window is small and the data is derivable — a workspace member can already
open every workflow and read its `{{KEY}}` references — but the gate's stated
contract is that references follow the same predicate as revealing the value, and
a point-in-time check that can be overtaken does not honour that. An advisory lock
or a snapshot transaction would serialize a read-only view against secret writes
for it, which is the wrong trade.
Instead the one volatile input is re-read after the scan and the request fails
closed if it flipped. `requireSecretReferencesReadAccess` now reports which branch
authorized: an `admin` grant holds however the name resolves and pays nothing,
while a `personal` grant — the only one resting on absence — is re-checked. A
non-admin loses nothing they were entitled to keep; the request is refused the way
it would have been a moment later.
Adds a `listSecretReferencesUseCase` suite covering both denial paths, the legacy
value, the personal owner, the admin short-circuit, and the race itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): accept JSON-escaped whitespace in the reference prefilter
Review round 4.
The prefilter reads a `::text` rendering of a JSON column, and `jsonb::text`
renders a real tab inside a string value as the literal pair `\` `t`. `t` is
alphanumeric, so `[^[:alnum:]_]` could not consume it and the row was discarded
before `ENV_REF_PATTERN` ever ran — the References tab omitting a live reference
and reporting `truncated: false` while doing it.
Round 3's fix was verified against a raw text value rather than the JSON
rendering, which is exactly why it looked correct: `E'{{\tAPI_KEY\t}}'` matches,
`jsonb_build_object('v', E'{{\tAPI_KEY\t}}')::text` does not.
The gap between `{{` and the name now accepts three encodings at once — raw
characters (covering every Unicode space, which Postgres `[[:space:]]` misses),
JSON two-character escapes, and `\uXXXX` (how a vertical tab survives the same
rendering). Verified against the real jsonb rendering: tab, newline, carriage
return, vertical tab and form feed all recover, U+00A0 / U+3000 / space / plain
keep matching, and `{{API_KEY_TEST}}`, `{{MY_API_KEY}}` and prose are still
rejected — so the row cap keeps meaning what it says.
Plain-text columns (`custom_tools.code`, `mcp_servers.url`) carry no JSON
escaping, but tool code is JavaScript source and can contain the same escape
sequences literally, so the one predicate is right for every column.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(secrets): land the References link on the block, and name its field
Feedback round.
- Logs leads the tab strip. It was already the default tab; the order now says so.
- The usage view drops its resource heading for a plain "Usage" title. The back
chip already names the secret, so the tile and the subtitle underneath were
saying it a second time. `CredentialDetailLayout` gains an optional `title`
that renders the same element, class and column position the settings shell
gives `SettingsPanel` — which is how the sibling Forks "Activity" view titles
itself. Existing callers pass nothing and are unchanged.
- A block row now lands on the block instead of the workflow's default framing.
The editor had no URL params at all, so `?block=` is its first: read once on
arrival, acted on, and stripped. It is a navigation signal rather than canvas
state — the carve-out in sim-url-state.md is about pan, zoom, selection and
drag, which are socket-synced or high-frequency; this is neither, and it rides
in the link so a middle-click or reload keeps it where an in-memory handoff
could not.
The consuming effect mirrors the note-search reveal in the same file, including
the three details that make that one work: read from `displayNodes` so a target
arriving before its node mounts is retried on the mounting commit, route
selection through `resolveSelectionConflicts`, and latch in a ref. It also
claims `userFocusedWorkflowIdRef` the way a node click does, because `onInit`
re-reads that inside its own rAF and would otherwise `fitView` over the camera —
and that ref is reset by exactly the `workflowIdParam` change a deep link causes.
The panel opens for free: `syncPanelWithSelection` already follows selection.
`useSearchParams` needs a Suspense boundary and the editor's ancestry has none,
so the read lives in a leaf under its own `fallback={null}` rather than wrapping
the editor and adding a `loading.tsx` to its mount path. `next build` passes.
- A tool-input reference showed `tools-tool-0-code`. Those
`{subBlockId}-tool-{index}-{paramId}` keys are documented as an ephemeral,
client-only projection of the canonical `tool-input` value and are not meant to
be persisted, but older rows carry them — so the scanner reported whichever the
record yielded last. They are dropped before scanning, which is right even where
the two disagree: `tool.params` is what executes, so a mirror the canonical no
longer matches describes a reference that no longer runs.
- The row now shows the field's label from the block config rather than its
storage id — "API Key", "Tools", "Code", "Bot Token" — falling back to the id
when the block or field is unregistered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): make the reference prefilter exactly as tight as the authority
Review round 6.
The gap between `{{` and the name accepted any non-word character, so
`{{-API_KEY-}}` and `{{"API_KEY"}}` matched in SQL while `ENV_REF_PATTERN`
rejects them. The previous commit called that free — "costs a candidate row and
nothing else" — which was wrong: a candidate row is a slot under
BLOCK_SCAN_LIMIT, so enough near-misses sorted earlier exhaust the cap before a
genuine reference is read, and the tab reports a live key as unused. That is the
same failure the tightening in round 1 was meant to remove, reintroduced by the
round 4 loosening that fixed JSON-escaped whitespace.
The gap now enumerates exactly the whitespace `\s` accepts, in each encoding it
can arrive in: `[[:space:]]` for raw ASCII, `\\[tnrf]` and `\\u000[bB]` for the
JSON escapes, and an explicit class for the Unicode spaces Postgres emits
verbatim but `[[:space:]]` does not match.
That class is generated from a code-point table rather than written literally.
Writing it by hand put a run of invisible characters in the source — a reviewer
cannot check them, and a formatter or editor can silently mangle them. The table
is the readable form and `toPgEscape` renders it.
Verified against the real jsonb rendering, 17 cases: raw space, tab, newline,
carriage return, vertical tab, form feed, U+00A0, U+202F, U+3000 and an embedded
reference all match; `{{-API_KEY-}}`, `{{"API_KEY"}}`, `{{API_KEY_TEST}}`,
`{{MY_API_KEY}}`, prose and an across-braces span all do not. Every candidate the
SQL admits is now a real occurrence, so the cap counts references and nothing
else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): cap the reference scan on results, not candidates
Review round 7.
The prefilter now matches reference syntax exactly, but `remapSubBlocks` filters
further on semantics SQL cannot see: it drops dormant canonical members and
condition-hidden fields. So a block whose only `{{KEY}}` sits in a hidden field is
a genuine candidate that yields nothing, and with the cap counting candidates,
enough of those sorted earlier displaced active references out of the answer.
Unlike the previous two rounds this is not fixable by tightening the prefilter —
no SQL predicate can evaluate canonical modes or field conditions. So the cap
moves to what it should have counted all along: blocks REPORTED. Candidates are
read a page at a time up to a ceiling far above the result limit, so filtered rows
are absorbed as extra reads instead of taking result slots.
Paging rather than one large read because the alternative is holding every
candidate block's `sub_blocks` in memory at once; peak memory is now one page.
`blockId` joins the ordering as a final tiebreak, since OFFSET paging over a
non-unique sort can repeat or skip rows across pages — which here would
double-report a block or silently lose one.
This does not make the scan unconditionally complete, and the ceiling says so:
bounded work and guaranteed completeness cannot both hold, so the only real
choice is where the bound sits and whether it counts something the reader can
see. It now counts results.
Query plan re-checked with the OFFSET in place: still an index scan on workflow by
workspace, nested-looped into workflow_blocks. Test added that pins the fix — 2,500
prose candidates sorted ahead of one real reference, which the previous cap
dropped entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): drop the paging that caused drift, and stop false truncation
Review round 8.
- Paging removed. It bought headroom and paid with drift: `OFFSET` is positional,
so a block renamed, inserted or deleted between page queries shifts the result
set, and the scan skips a live reference or reports one twice. That is a worse
failure than the one paging was added to fix, and it was self-inflicted last
round. Candidates are read in one statement again — one statement is one
snapshot, so neither skew nor duplication is possible — with the ceiling
lowered to 4,000 so a single read stays a sane amount of memory. Result-capping
survives, which was the actual point: filtered rows are still absorbed as extra
reads rather than taking result slots.
- `truncated` no longer fires on an exact landing. The block path now uses the
limit-plus-one read and strict `>` the resource paths already used, so a scan
that ends precisely on a bound reports complete instead of warning about
references that were all returned.
- The deep-link target is released when its block does not exist. It was cleared
only on a match, so a link to a since-deleted block left the id set with the
param already stripped: the effect re-checked on every canvas update forever and
shadowed a later link to the same block. Once any node has mounted the canvas is
populated, so an id still absent is gone and the target is dropped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): gate the deep-link release on the workflow being ready
Review round 9.
Round 8 released a deep-link target once `displayNodes` was non-empty, reading
that as "the canvas is populated, so a missing id is deleted". It is not: arriving
from another workflow the store still holds that graph, so nodes are present while
the linked workflow is still hydrating — and a valid `?block=` target was dropped
before its own blocks ever mounted.
The file already had the right predicate. `isWorkflowReady` pins
`hydration.phase === 'ready'`, `hydration.workflowId === workflowIdParam` and
`activeWorkflowId === workflowIdParam`, which is exactly "the graph now loaded is
this workflow's". Absence is only conclusive under that, and a node count never
was — it says something mounted, not whose.
This is the second fix to this release condition in two rounds, both from guessing
at a readiness signal instead of using the one the component already computes for
the same question.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Siddharth Ganesan
|
58aa6379e0 |
feat(cli): add chat command (#6937)
* feat(cli): add chat command * fix(cli): harden chat command execution |
||
|
Waleed
|
00d8a3fbfe |
fix(knowledge): refund a processing attempt whose dispatch never happened (#6938)
* fix(knowledge): refund a processing attempt whose dispatch never happened `markDocumentsQueued` spends one attempt from the processing budget on every dispatch, and `clearDocumentsQueued` already withdraws the queue stamp on the one path that proves nothing was dispatched — but it left the attempt spent. The budget exists to stop re-billing a document that keeps failing the same way in processing. An attempt that never reached a worker teaches it nothing, so an infrastructure outage — a Trigger.dev region error, an exhausted quota — burned the allowance without a single run. MAX_PROCESSING_ATTEMPTS such outages dead-letter the document, and the connector sweep filters on `processingAttempts < MAX_PROCESSING_ATTEMPTS`, so automatic recovery stops for a document that was never processed once. Refunded in the same guarded statement as the stamp, so it can only ever give back the charge this call made, and floored at zero. Also corrects the stale-lock TTL doc, which still described the reclaim as measuring `updatedAt` after it moved to `COALESCE(syncLockLeaseAt, updatedAt)`, and the attempt-budget rationale, which predates the refund. * chore(audits): re-record the route module-graph baseline Every workspace route had drifted past what the baseline records, uniformly: the shared `[workspaceId]/layout.tsx` grew +28 and each route inheriting it grew +27 to +29. The knowledge page carries +18 of its own on top, which put it +46 over a +44 allowance and was the only entry actually failing. Bisecting the growth across the last five commits on staging shows 2249 → 2251 → 2252 → 2253 → 2255 — one or two modules per unrelated feature PR, not a single regression dragging in a fat dependency. That is the organic creep the `max(25, 2%)` ratchet is meant to absorb and the re-record is meant to settle. Counts only: all 34 entries are preserved and every entry's gateway set is unchanged, so the registry-reachability gate and the per-entry ratchet keep exactly the strength they had. |
||
|
Vikhyath Mondreti
|
ab66ce9149 |
fix(admin): harden dashboard billing operations (#6914)
* feat(admin): make dashboard billing operations durable * fix(admin): close dashboard recovery gaps * fix(admin): preserve member operation lock order * fix(admin): harden durable operation boundaries * fix(admin): report member operation failures accurately * test(invitations): cover locked seat admission * feat(enterprise): gate owner activation on acceptance * fix(enterprise): keep owner activation recoverable |
||
|
|
42f6287911 |
feat(byok): add organization-wide key inheritance (#6834)
* feat(byok): add organization key management * feat(byok): inherit organization keys at runtime * feat(byok): add organization scope to BYOK settings * fix(byok): refresh org key state after mutations * fix(byok): hide stale inherited status badges * chore(db): drop colliding byok migration ahead of staging merge Staging independently claimed 0293. Remove ours so the merge is clean; it is regenerated at the next free index right after. * chore(db): regenerate byok migration at 0296 Staging claimed 0293-0295 during the merge; the regenerated SQL is byte-identical to the dropped 0293. * docs(byok): document organization scope, precedence, and the full provider list The BYOK section described workspace-scoped keys only. Add the organization scope, its Enterprise requirement, the per-provider precedence rule, what an entitlement lapse does, and the Pi sandbox exposure. Refresh the provider table from the settings page, which had drifted from 14 to 34 entries. * feat(byok): open organization keys to every organization plan Organization BYOK was gated on Enterprise, but an organization is the only thing that can hold the keys, so every plan that can own an organization should qualify — Pro for Teams, Max for Teams, and Enterprise. Add checkOrgPlan/resolveOrganizationPlan beside the Enterprise pair rather than widening checkEnterprisePlan, so the Enterprise-only gates (Access Control, whitelabeling) are untouched, and restore resolveOrganizationEnterprisePlan to module-private now that BYOK no longer needs it. * perf(byok): cache the organization entitlement, not the key material getBYOKKey runs once per agent block and once per hosted-capable tool call, so a loop over N items resolved N times — and each organization-inheriting resolution paid three sequential billing queries on top of the two key reads. Split the two reads by staleness tolerance. Key rows stay fresh, because revocation must be immediate. The entitlement is a billing gate that tolerates bounded staleness in the harmless direction (a lapsed organization keeps using its own key for <=60s), so cache it per organization with an in-flight share so concurrent blocks issue one query set. The management surfaces keep reading it fresh, so an organization that just upgraded is never told otherwise. Also run the block check and subscription read in parallel inside resolveOrganizationPlan, and carry the resolved scope on BYOKKeyResult so a log line can say whether a run used the workspace's key or an inherited one. * feat(byok): let workspaces store the Z.ai and Cohere keys the runtime reads Both ids were already in the BYOK contract enum and both are resolved at execution time — getApiKeyWithBYOK reaches 'zai' (GLM models are in the hosted catalog, so the BYOK branch runs), and 'cohere' backs both the Embeddings block and Knowledge Base reranking — but neither appeared in the settings list, so there was no way to store the key either path looks for. Cohere had no icon; add one from the official multi-color mark so it stays legible on a light and a dark page. Cohere's embed-v4.0 is kbEligible:false, so the description says 'Embeddings and Knowledge Base reranking' rather than claiming KB embeddings. * improvement(byok): shorten the workspace scope chip to 'Workspace' It sits beside 'Organization', so the scope reads from the pair; 'This' only added width. * fix(byok): do not cache a billing outage as an unentitled organization resolveOrganizationPlan maps a failed billing read to false, which is indistinguishable from a real plan lapse. The entitlement cache stored that, so one transient outage held the gate shut for the full TTL and every inheriting run silently fell back to a metered hosted key — and the cache's rejection path, which exists to prevent exactly this, was unreachable. Give the resolver the onError option its neighbours already have and let the cached read ask for 'throw', so a failure stays out of the cache and the next resolution retries. Behavior for the call that saw the error is unchanged: getBYOKKey still fails closed. Reported by Cursor Bugbot. * fix(byok): propagate the subscription read's failure too The previous commit threaded onError through resolveOrganizationPlan's own catch, but getOrganizationSubscriptionUsable soft-fails to null on its own, so a failed subscription read still arrived as an ordinary 'no usable subscription' and returned a successful false — which the entitlement cache then stored for the full TTL. Thread the option into that call as well. Test it at the billing layer rather than the cache layer: the entitlement test mocks resolveOrganizationPlan wholesale, so it could never have caught this. Verified the new test fails against the previous commit. Reported by Cursor Bugbot. * refactor(byok): cache the entitlement with LRUCache, like copilot entitlements The hand-rolled version reinvented three things the codebase already has a canonical answer for. lru-cache is a declared dependency of apps/sim and lib/copilot/entitlements.ts already caches an entitlement with it — by storing the in-flight Promise, which is what makes concurrent callers collapse onto one resolution with no in-flight bookkeeping at all. TTL and the size bound come from the library. That removes the second Map, the manual eviction (and its interaction with an in-flight entry), and the dead value-while-refreshing state: 23 executable lines. The one thing the library does not cover is dropping a rejected promise so a billing outage is not cached for the TTL, which is kept and pinned by a test that fails without it. TTL expiry is no longer re-tested — that is the library's behavior, not ours, and lru-cache reads its clock at module load so faking timers never moved it. * refactor(byok): coalesce the entitlement read with the shared singleflight lib/concurrency/singleflight.ts is the codebase's coalescing primitive and oauth/credential-service.ts already pairs it with a read-through cache. Adopting that shape fixes a case caching the promise directly did not: a *hung* billing read wedged every caller for the full 60s TTL, where coalesceLocally evicts and rejects at its settle deadline. It also removes the hand-rolled rejection eviction — the cache is written only on the success path, so an outage leaves no entry by construction. The cache now holds booleans, which introduces the one trap worth a test: a truthiness check would read a cached false as a miss and re-query billing on every resolution for lapsed organizations. Pinned. * fix(byok): keep an abandoned entitlement producer from writing the cache coalesceLocally does not cancel a producer it timed out — its docstring says so explicitly — so writing the cache from inside the producer let a late billing result overwrite a fresher answer a retry had already cached, and hold it for a full TTL. Move the write onto the value the caller actually received. A caller that timed out throws before reaching it, so an abandoned producer now resolves into nothing. The test reproduces the overwrite and fails against the previous shape. Reported by Cursor Bugbot. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> |
||
|
|
ea70f8dcf1 |
feat(harmonic): add contact workflow integration (#6902)
* feat(harmonic): add contact workflow integration * fix(harmonic): sync docs manifest * fix(harmonic): address integration review findings * feat(harmonic): add the missing people endpoints and fix two error paths Extends the integration from 4 to 13 tools, covering every non-deprecated people-scoped Harmonic endpoint, and repairs two defects found by validating the existing tools against Harmonic's OpenAPI and API reference. New tools: - Enrich Person (POST /persons) — the only path from a LinkedIn URL or email a workflow already holds to a Harmonic contact. - Get Person, Get Company Employees — account-based sourcing; employees returns URNs that chain into Batch Get People. - Saved-search net-new results and their acknowledgement, so a monitor stops reprocessing the entire result set on every poll. - Bulk email enrichment: submit, poll, and quota, plus Get Enrichment Status. Fixes: - The error extractor dropped Harmonic's string and object `detail` envelopes. A tool that names an extractor gets no fallback chain, so every FastAPI abort surfaced as "Request failed with status 403". The enrichment 404 also carries the scheduled `enrichment_urn`, which was being discarded — that URN is the only handle on the job, so it is now kept in the message. - The saved-search selector failed the whole dropdown instead of degrading: the response cap was half the sibling value on an endpoint that is unpaginated and returns every saved search with its full query object, and the option ceiling threw rather than truncating. Raised to 1MB and switched to truncate-and-warn, matching the other data-driven selectors. Clearing net-new results now requires an explicit scope. Harmonic treats an absent `entity_urns` as "clear everything", so an empty field would have silently discarded the backlog. Scope deliberately excludes company-side, deal, typeahead, network, and Scout streaming endpoints, and every endpoint retiring on 2026-11-05. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Theodore Li
|
a99f61bee9 |
feat(api): add v2 resource management endpoints (#6900)
* feat(api): add v2 resource management endpoints * fix(cli): gate destructive v2 commands * fix(test): update v2 request-slice count * feat(cli): add shared workspace profiles |
||
|
|
d9cfd7c68e |
improvement(mothership): v0.9 (#6815)
* checkpoint
* Checkpoint
* dot fixes
* Make async tool resume delivery recoverable
* Support split table tools and option recovery
* Harden VFS mutation handling
* feat(platform): platform subagent support — docs corpus VFS, search_docs, account context
Squash of the feat/platform-agent branch (sim side): mounts the Sim docs
corpus in the copilot VFS, wires search_docs and retires the legacy docs
search tools, and syncs the generated tool catalog and trace contracts for
the platform subagent.
* Align Copilot tools and resource handling
* Expand workflow log query support
* checkpoint
* Port desktop-improvements-0 desktop and browser-agent work
* fix(desktop): keep browser-agent input alive through live-SPA re-renders
* Harden workflow sanitization and Slack setup
* feat(desktop): coordinate clicks, caret insertion, and drag for the browser agent
* Revert subagent group eager auto-collapse
* fix(chat): keep sends FIFO across the streaming-to-idle drain gap
* Add the steering backend surface for mid-turn sends
* Sync generated contracts for async subagent orchestration
Pulls the mothership tool catalog (wait_agents / tail_agent / steer_agent /
interrupt_agent), trace spans (chat.async_subagent.*, chat.orchestrate.*), and
trace attributes (copilot.async_subagent.*) into the generated TS contracts.
* Add display titles for the async subagent orchestration tools
wait_agents / tail_agent / steer_agent / interrupt_agent get natural-language
running titles (naming the agent id being waited on, tailed, steered, or
stopped) and a Steering→Steered completed-verb rewrite.
* Show orchestrator-chosen subagent names on agent groups
A subagent_start whose payload data carries a name (the orchestrator's new
name trigger parameter) now labels the agent group with that mission name —
the agent-type icon stays. The name flows through the live stream path, the
turn model (AgentNode.displayName) and its serialize/rebuild round-trip, and
persisted transcripts (PersistedContentBlock.name), so reloads keep the label.
* Improve Copilot error handling and logging
* Backfill the subagent display name from the second start event
The dispatch-time subagent_start fires before the trigger args (and therefore
the name parameter) have streamed; the phase-3 start re-announces the lane with
the name. The block builder was dropping that duplicate wholesale, losing the
name on streaming providers — now it backfills subagentName onto the existing
block instead. (The home turn-model path already reconciled this case.)
* Support Slack bot connection flow
* Harden Copilot error and VFS handling
* Harden VFS resource operations
* Show 'Waiting for the first of N agents' for mode-any waits
The wait_agents title ignored the mode argument, so an any-mode wait over
three agents read 'Waiting for 3 agents' while the model narrated waiting for
the first — contradicting the transcript.
* Collapsed-by-default agent cards with live intent status lines
Subagents now narrate their work through <intent>3-5 words</intent> tags (a
fleet-wide prompt protocol on the mothership side). The turn model streams
each subagent's text through a split-safe tag parser: complete tags update the
agent's currentIntent and disappear from the prose, tags split across deltas
are carried until their close arrives, and a tag that never closes flushes
back as plain text.
The agent card renders as one line — display name (or agent label) plus the
latest intent, replaced inline as the agent shifts gears — and never
auto-expands; expanding to the full tool log is a deliberate click. Only an
outstanding permission prompt or a browser hand-back forces a group open.
Intents persist on the subagent block (and through the legacy persisted-
message paths) so reloads keep the last status, and a renamed reinvocation
now takes the latest name instead of pinning the first.
* Add the internal in-band tool execution route for live mothership turns
POST /api/copilot/tools/execute (INTERNAL_API_SECRET, Go→Sim) runs one
sim-server tool through the same server tool router the resume driver uses
and returns the result synchronously — no checkpoint. This is what lets
background (async) subagents write files/tables/knowledge, and lets the main
lane keep streaming (instead of checkpoint-pausing and killing every
background run) while async agents are live.
* Persist resource side effects for in-band tool execution
Files/tables created through the internal execute route now register on the
chat's resources exactly like the resume driver's executions — the route runs
the same handleResourceSideEffects pass (persistence only; an out-of-band
route has no live event sink, so mid-turn chip pushes are a follow-up).
* Extract intents from group text on every path, sync and async
The turn-model intent filter only fires for span-scoped subagent lanes, but
this surface also delivers subagent text through the legacy block path — so
<intent> tags flowed through unparsed and rendered as prose rows. Groups now
extract intents from their accumulated text at append time: the last complete
tag becomes the card's status line and every complete tag is stripped from
the rendered prose. Covers span-scoped, legacy, and persisted-reload paths
for both synchronous and background delegations.
* Fall back to the live tool title for the agent card status line
Persisted data proved tool-first subagents (grok search agents) emit zero
prose, so intent tags never stream no matter what the prompt says. The
collapsed card now always narrates: the agent's own <intent> tag when present,
else the latest tool's display title while the lane is live.
* Catch subagent <intent> tags in the server relay
The relay's subagent text handler now runs the split-safe intent extraction
as chunks stream: the latest complete tag is stamped onto the lane's persisted
subagent block (subagentIntent) and stripped from the stored prose, so live,
persisted, and replayed views all agree. Per-lane carry handles tags split
across chunks; a never-closing tag flushes back as plain text.
* Drop the tool-title fallback: the status line is the agent's intent
With the intent protocol now injected into every spawn's task message, agents
open with an <intent> tag; the card shows that narration or nothing.
* Replace intents with live tool-title status lines on agent cards
Intent parsing is fully removed (turn model, relay handler, persistence
fields, group extraction). The collapsed card's status is the latest tool
call in its RUNNING phrasing — never the completed rewrite, which stays in
the expanded log. Parallel tools show the most recently started still-running
title with a +N for concurrent siblings; between rounds the last title stays
frozen; a closed lane shows the bare name. Nested agent cards compute their
own status recursively from their own items.
* Keep the main Sim lane live-expanded; collapse only real subagent cards
The mothership group is the turn's own narration, not a delegation card —
collapsing it hid main-lane text and tools until manual expand, which read as
mis-ordered streaming while async subagents interleaved. It keeps the
original live-expand behavior and no status suffix.
* Persist subagent lane lifecycle blocks from the span handler
Lane-scoped span events route to the span handler, which only recorded trace
side effects — no subagent start block was ever persisted (verified: a
seven-agent run stored 104 blocks with zero starts). Grouping then fell back
to keying lane content by agent NAME, so a respawned agent of the same type
merged invisibly into the first one's card until it resolved. The handler now
persists the start block (spanId-keyed and deduped, carrying the display
name) and stamps endedAt on close, giving every invocation its own card.
* Name agents in orchestration titles; '+ n more' overflow format
wait/tail/steer/interrupt titles humanize the slugified agent ids back to
their display names ('Waiting for the first of Digest Workflow Build + 4
more'), and the agent card's parallel-tool suffix uses the same '+ n more'
format.
* Harden in-band tool execution and resources
* Route in-band execution through the comprehensive tool dispatcher
The internal execute route used the bare server-tool router, which rejects
VFS tools with 'Unknown server tool: read/glob/grep' — so nearly every
background agent's first discovery call failed (102 in-band calls in one run,
dozens rejected). It now uses the relay's executeTool dispatcher: registered
handlers (VFS, function execute) with permission checks and param
normalization, falling back to the app tool router — the same surface
foreground execution gets.
* Harden chat stream transition handling
* Harden VFS provenance and resource writes
* Standardize tool environment references
* Harden browser panel and chat cleanup
* Descriptive, user-language tool titles across the board
House rules applied everywhere: use every argument the call carries, never
name internal machinery, and never lead with Getting (the Got rewrite is
deleted so it cannot return).
- Deployments name the workflow: Deploying {workflow} as API/chat app/MCP tool
- Workflow reads name the part: Reading {workflow} meta/state/deployment/notes;
generic reads always name the file (Reading {leaf}), never bare Reading file
- Block runs name block and workflow: Running {block} in {workflow}, Running
from {block} in {workflow}, Running {workflow} until {block}, and
Enabling/Disabling {block} in {workflow}
- The six split-table tools get per-operation verbs (Adding column {name},
Updating rows, Wiring automation, Creating view {name}) instead of a wall
of Querying table
- The manage quartet drops X-action system-speak for gerunds
- get_* internal names become user language (Checking run settings, Tracing
block inputs, Reading the deployed version); web_fetch says Fetching
- Scheduled-task titles removed entirely (feature deleted from the Go catalog)
- New verb rewrites: Fetched, Traced, Wired, Configured, Looked, Rotated
* Deploying {workflow} as chat, not as chat app
* Loader gerunds; mv names both ends; mkdir names the folder
search_integration_tools -> Finding the right integration;
load_integration_tool -> Loading {integration} tools; load_skill ->
Loading skill {name}; run_enrichment -> Looking up {subject}. mv prefers the
model's phrasing, else reads 'Moving {files} to {destination}'; mkdir reads
'Creating folder {name}' from the path.
* Overflow counts read '+ n', dropping 'more'
* Unify workspace find and search
* Scale desktop title bar with page zoom
* Serialize account and organization truth into the copilot VFS
Workspace standing, membership, billing, org role, access-control
restrictions, published-block provenance, and fork topology were reachable
only through three parameterless tools (or not at all). They are ambient
read-only facts, so they belong in the VFS where they are greppable, cost no
tool round-trip, and every agent that can read gets them — the same move that
retired get_blocks_and_tools and list_user_workflows.
Adds account/{workspace,workspaces,members,billing}.json (always mounted) and
organization/{organization,access-control,custom-blocks,forks}.json (only when
the workspace is org-hosted). Every file projects an existing use case or util
after getOrMaterializeVFS's access assert — no new queries, no new
authorization. One relation per file, cross-referenced by id-and-name stub, so
overlapping facts cannot disagree. Volatile content (billing, access control,
forks) is lazy, so numbers are read-time fresh and unasked-for reads cost
nothing.
Projection follows the viewer: member emails are admin-only, fork detail
requires workspace admin on a forking-enabled org, and the whole organization/
namespace is absent for a personal workspace — which is itself the answer.
Retires get_account_billing, get_enterprise_context, and list_user_workspaces
along with their handlers; display titles stay for transcript replay.
* Fix insert_text refusing an editable field focused inside a frame
describeFocusedEditable descended shadow roots but not frames, while
activeElementReadback descends both. Focus inside a same-origin frame therefore
surfaced to the first as the FRAME element — not an input, not contentEditable,
not a canvas, no textbox role — so it fell through to 'not-editable' and
insert_text refused a field that press_key had just typed a character into.
Two functions answering 'what is focused' with different answers is the bug;
the descent loops now match exactly.
The refusal also names what actually held focus (tag, role, contenteditable).
A bare 'not-editable' gave the agent nothing to act on, so it guessed at the
cause — a real run spent twenty rounds on the wrong theory and had to be
stopped by the user.
* Keep retired browser takeover renderable in history
The tool is gone from the catalog, so its generated constant went with it and
every path that referenced it stopped compiling. Deleting those paths instead
would have silently downgraded every past transcript containing a takeover card
to a generic tool row, and dropped the no-timeout budget that an in-flight
takeover still needs while a rolling deploy finishes.
retired-tools.ts gives the literal a documented home that says what it is and
why it survives its tool.
* Follow the agent into a tab it opened to work in
browser_open_tab created the page with activate: false, so the agent worked in
a tab the user could not see while the panel sat on a page where nothing was
happening. The panel now follows a tab the agent deliberately opened.
Scoped to that tool only. A page spawning its own tab (popup, target=_blank) is
the site grabbing the view rather than the agent choosing a workspace, and
stays in the background as before — two existing tests pin that and caught the
first version of this change, which moved both.
A tab the user claimed still wins over both: the work starts in the background
instead of pulling the page out from under them mid-read.
* Make the browser tools agree with each other
An audit of the module found the frame-descent bug was one instance of a
pattern: six independent definitions of 'is this editable' and seven of 'what
is focused', disagreeing with each other. A tool refusing what its sibling
accepts on identical page state is invisible at runtime — the agent follows a
snapshot that says one thing into a tool that says another.
- browser_type now accepts role="textbox" like browser_insert_text does. The
snapshot advertises those elements as [textbox] with a ref, so refusing them
meant rejecting exactly what the outline told the model to type into. Both
the native and synthetic paths, and their descendant scans.
- pressKeyOnPage descends shadow roots and frames like every other focus
reader. It was dispatching synthetic keys at the shadow host or <iframe>
element, where they bubble but never reach the editor, while reporting
success — and contradicting the activeElement reported beside it.
- not-editable and ambiguous-editable name what was found: the element's tag
and role, and the candidate fields. Both had the data and discarded it, which
is what turns one blocked step into twenty rounds of guessing.
- obstructedAfterNavigation requires a dialog that ARRIVED with the
navigation. It compared against nothing, so every SPA route change under a
persistent role=dialog reported a successful click as obstructed. The test
that covered this asserted the false positive; it now pins both directions.
- browser_insert_text observes the top document when typing inside a frame,
like every other input tool. A submit that navigates the top page was
invisible to its frame-scoped observation.
* Let hover actually see what it mounted
Four independent defects made browser_hover blind to the most common thing a
hover produces — a row's action bar — so it reported no effect on a hover that
worked, and the agent fell back to clicking pixels off screenshots.
- The popup scan matched only role=tooltip/menu/listbox. Slack's message
shortcuts bar is a labelled toolbar/group, so it registered as nothing at
all. Added toolbar, menubar, labelled group, and [popover].
- The baseline was captured BEFORE prepareElementSurface scrolled the target
into view, so scrollChanged was always set by the tool's own probe. That
pinned every unproductive hover to 'background DOM churn' instead of the
honest 'nothing happened', and hid scrolling the hover really caused.
Re-baselined once the scroll settles and before the pointer moves.
- The MutationObserver attached only on the first observation, while the roots
list is rebuilt every call and grows as shadow roots mount. Components that
appeared later were never observed, so their DOM changes raised no revision.
Roots are now observed as they show up.
- observationTruncated was computed and never read, so a scan capped at 12k
nodes reported 'nothing appeared' with the same confidence as a complete
one — and portalled overlays live at the end of <body>, exactly what the cap
drops. Hover now says the page was too large to scan and to confirm visually.
* Stop the browser agent acting on the wrong element, and say why it refused
Four findings from the module audit, the first of which could silently do the
wrong thing rather than merely fail.
- A ref whose node is gone is re-adopted by structural resemblance, matching on
ORIGIN only so a pushState between snapshot and act does not kill every ref.
That leniency also let a ref to a row control in one view rebind to the
identical control in a view the app had since navigated to — acting on the
wrong message, signalled by nothing louder than recovered: true. Adoption now
requires the same path; a view swap reports the ref stale, and the caller
re-snapshots. Revalidating a still-connected node stays lenient, because that
is literally the node the model chose.
- A hit INSIDE the requested element is its own nested control, not an overlay.
Both produced 'covered by X — close or move the overlay', advice that cannot
be followed because there is nothing to close. Nested hits now say so and
point at retargeting.
- browser_click_at, browser_insert_text, and browser_drag listed targetChanged
in their effect formulas, but none passes an elementId, so no targetState is
ever captured and the term was always false — coverage that read as real.
Removed, with a test pinning the dependency.
- The seven effect formulas are deliberately NOT collapsed into one predicate:
drag must trust domChanged where others must not, hover must ignore field and
focus changes, click counts focus only for editables. Forcing one would make
each tool wrong differently. The differences are now documented in one place
next to the shared computation, so divergence is a declared policy rather
than an accident.
* Let edit_workflow configure block retries
* Updates
* Always focus the resource the agent is working on, and its browser tab
The resource panel had a carve-out: an already-open browser session declined
to replace another selection and only got an attention marker, so agent
browser work happened off-screen. The panel now follows the agent to whatever
it touches — browser included — and an event can still opt out explicitly.
The browser panel also follows the agent BETWEEN tabs: the store already
tracked automationTabId (and the strip marked it), but the visible tab never
changed. It now switches when the agent's target tab changes, so watching the
agent never means hunting for the tab it moved to. Keyed on the target
changing rather than on it being set, so a user who browses elsewhere
mid-run is only pulled along when the agent itself moves.
* Never paint a browser snapshot at stale geometry (the modal-open flash)
Opening a modal locks scroll, which removes the window scrollbar and reflows
the panel — so a capture taken before the lock describes a rect the panel no
longer occupies. The handshake painted that frame anyway and only then
retried, so the replacement landed visibly offset from the page it stands in
for: the flash. A capture is now checked against the host's live rect before
it is painted; a mismatched frame is skipped and re-captured at the settled
layout instead (modal retries go 2 -> 3 to absorb the extra settle).
* Name the workflow in deployment and workflow-scoped tool titles
'Checked deployment status' never said which workflow — nor did the deployed-
state read, run settings, block outputs/inputs, redeploy, promote, or the
global-variable write. These tools carry a workflowId (often defaulting to the
current workflow), so only the client can resolve a name: the enrichment layer
now resolves it for the whole workflow-scoped family and passes it as
workflowName, which every workflow title already reads.
Titles: Checking {workflow} deployment status, Reading deployed {workflow},
Checking {workflow} run settings, Reading {workflow} block outputs, Tracing
{workflow} block inputs, Redeploying {workflow}, Promoting {workflow} version
{n} to live, and 'Adding workflow variable {name} in {workflow}' — each
falling back to its unnamed form when no workflow resolves.
* Name the block that ran; never fall back to a raw block id
run_block and set_block_enabled carry only a blockId, so their titles would
have printed an opaque UUID ('Running 7f3a2b91-… in Invoice Sync'). The
enrichment layer now resolves blockId against the workflow store the same way
it already did for run_from_block's startBlockId, and the base titles no
longer accept an id as a name — an unresolved block reads 'Running block'
rather than a UUID.
* Add the missing Removing -> Removed rewrite
The table work introduced 'Removing automation'/'Removing enrichment' with no
past form, so those rows kept their present tense after completing.
* Name the target resource in the remaining tool titles
Table tools keep their operands nested under args and identify the table by
id, so their rows said 'Adding rows' with no hint where: enrichment now lifts
the nested args and resolves tableId against the cached workspace table list,
giving 'Adding rows to Runtimes', 'Adding column status in Runtimes',
'Reading views of Runtimes'.
Also: a block-schema read names the block instead of the file ('Loading
Slack', 'Loading Google Sheets tips'); browser type/insert show the text they
send, middle-ellipsized; downloads name the file; library-docs searches name
the library and query; knowledge-base searches include the query; generated
media names its output file; and diff_workflows, list_deployment_versions,
and publish_custom_block joined the workflow-name enrichment set.
* Bubble nested agents' tool calls into the parent's status line
The collapsed status only scanned a group's OWN tool items and skipped nested
agent groups, so a parent that had delegated froze on its last own tool while
its child did the actual work — the line described nothing that was running.
Status now walks the whole subtree: any tool at any depth counts, the most
recently started running one is shown, and the rest become the same '+ n'
overflow. With nothing running it falls back to the last tool at any depth,
so an idle parent still reflects where its subtree got to.
* Align nested tool call status rows
* Revert branch-local KB connector error-message edits
Restores apps/sim/connectors/ to staging state. Two copilot-focused commits
on this branch (
|
||
|
Justin Blumencranz
|
0b71717241 |
perf(icons): reduce and guard SVG path precision (#6839)
* perf(react): reduce SVG path precision * fix(react): preserve Sim wordmark precision * fix(icons): preserve Quartr scale * test(icons): ratchet SVG path precision * fix(icons): make precision exceptions local * perf(icons): enforce three-decimal paths |
||
|
Vikhyath MondretiandClaude Opus 5
|
2ced737976 |
refactor(sub-blocks): make a registered selector the single source for a remote option list, and close the fork-sync reconfiguration gap (#6878)
* fix(workspace-forking): stop double-labelling a custom block's inputs, and derive their controls from the canvas
Two problems with how a repointed custom block's inputs render in the sync modal.
The field title printed twice. The row wrapper already draws the label and its
required marker for every dependent field — `DependentFieldSelector` takes a
`title` only to phrase its placeholder and renders a bare combobox. The
custom-block branch used `ChipModalField`, which owns a label of its own, so every
input showed its name twice. It now renders bare controls like its sibling does.
The control was chosen by re-reading the raw field type instead of asking the
function that already answers this. `subBlockTypeForField` decides what a Start
field becomes on the canvas; the modal had a parallel switch that had already
drifted, rendering a `file[]` input — an upload on the canvas — as a plain text
box, which would write a bare string into a field expecting file references.
`subBlockTypeForField` is now exported and the modal derives from it, so the two
cannot disagree about what a field IS; the modal only decides how that kind draws.
A file input is explicitly `unsupported` rather than falling through: it renders
disabled, saying it is set in the workflow, instead of inviting a value that
cannot work. A test walks every type a Start field can declare and asserts the
modal's choice follows the canvas's, so a type added later surfaces here rather
than silently becoming a text box.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(workspace-forking): resolve a custom block's inputs against the target environment, and stop a re-sync wiping its uploads
A repointed custom block's inputs are configured at sync time, but the modal
drew them as bare text fields against no environment at all:
- `{{SECRET}}` had no completion, and no way to know which secrets exist in the
workspace the value is written INTO.
- `<block.output>` had no completion. The canvas dropdown reads the workflow
open in the editor; on the fork settings page there is none, and the workflow
that matters is the target's.
- A `file[]` input has no control here (it is an upload on the canvas), so it
had no stored override — and the block was rebuilt from overrides alone, so
every sync silently dropped the target's uploaded files.
`WorkflowReferenceScope` lets a surface supply the workflow a reference resolves
against. Absent a provider, the hooks read the live editor stores exactly as
before, so the canvas is unchanged. The scope splits graph from values on
purpose: reachability cannot change with the text being typed, and the
validation hook runs in every reference-aware sub-block editor at once, so
subscribing it to live sub-block values would re-render all of them on every
keystroke. A test pins that split.
`replaceCustomBlockInputs` now seeds from the target block when it is ALREADY
the mapped type, layering the configured values on top. That keeps an input the
modal cannot offer a control for, and leaves a field the user simply did not
touch alone; a field they explicitly emptied stores `''`, which is an override
and still wins. Under a DIFFERENT current type nothing is carried over — those
values are keyed by another block's field ids, which is the orphaning this
function exists to prevent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(workspace-forking): stop a required file input deadlocking Sync
Both PR bots flagged this and they were right. A repointed custom block's
`file[]` input renders as a disabled control — it is an upload on the canvas,
and there is nothing to type here — but the Sync gate still demanded a
non-empty value for every REQUIRED dependent. So a custom block with a required
file input turned Sync off permanently, while the field's own hint told the
user to go set it in a workflow they could only reach BY syncing.
`isForkSyncConfigurableField` is the one predicate for "can the modal put a
value in this field", used by the gate and by the per-kind status badge so the
two cannot disagree. Skipping the gate is only safe because the sync no longer
clears the field: the target keeps what it has, and a genuinely missing value is
still caught by the block's own required-field validation at run/deploy time —
the same fallback every other unconfigured required field already relies on.
Also gives the disabled control an `aria-label` (the row's visible label is a
sibling, not associated), closing the second review note.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(sub-blocks): make a registered selector the single source for a remote option list
`dropdown` and `combobox` could only load a remote list through a per-block
`fetchOptions(blockId)`, which resolves its credential by reading the live
workflow store. That works on the canvas and nowhere else — which is why the
fork sync modal cannot offer those fields, and why every one of those fetchers
turned out to be a hand-rolled duplicate of a selector that already exists
(`triggers/gmail/poller.ts` calls the very contract `gmail.labels` wraps).
Both controls now accept `selectorKey`, resolved through the registry inside
`useFetchedOptions`. Deliberately NOT a second code path: the registry is
presented through the same two function shapes the props already describe, so
the existing lifecycle — request-id guards, dependency-scope reset, label
hydration — is reused verbatim, and paginated selectors drain through the same
`loadAllSelectorOptions` that search/replace and value resolution already use.
`isDynamic` replaces the `fetchOptions &&` test the controls used to decide
whether the fetched list or the static `options` array is authoritative; that
question outlives the prop it was asking about.
No block or trigger changes yet, so nothing moves off `fetchOptions` in this
commit: subblock `type`, `multiSelect`, and the stored value shape are all
untouched and no existing workflow is affected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(triggers): move every credential-scoped option list onto a registered selector
Each of these `fetchOptions` resolved its credential with
`readSubBlockValue(blockId, 'triggerCredentials')` — a live-workflow-store read
— and then called the very selector contract a registered selector already
wraps. They were duplicates that only worked on the canvas.
Migrated: webflow sites/collections (x4 triggers), clickup workspaces, gmail
labels, outlook folders, and all six hubspot pickers. 425 lines of duplicated
fetch logic deleted.
The missing piece each one needed was `canonicalParamId: 'oauthCredential'` on
its credential subblock: `buildSelectorContextFromBlock` keys the context on a
subblock's CANONICAL id, so without it `context.oauthCredential` was never
populated and the block had no way to reach its credential except the store —
which is what forced the hand-rolled fetcher in the first place.
Five new hubspot selectors. `hubspot.pipelineStages` reads the pipelines
contract and narrows, because HubSpot returns stages inside the pipeline
payload rather than behind an endpoint of their own; sharing the one response
is also what keeps a stage list from ever describing a pipeline its sibling
picker is not showing. `objectType`/`customObjectTypeId`/`pipelineId` join
SelectorContext, and `resolveObjectType` keeps HubSpot's own `contact` default
so an untouched dropdown still lists properties for what it visibly shows.
Subblock `type`, `multiSelect`, and stored value shapes are unchanged, so
existing workflows are unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(triggers): move the table trigger's column picker onto table.columns
`fetchTableColumns` resolved the workspace from the active-workflow store and
the table id by reading two subblocks by name, then refetched the table list to
find one table's schema. The registered `table.columns` selector takes both from
the context — `tableSelector`/`manualTableId` already carry
`canonicalParamId: 'tableId'`, so the canonical pair resolves on its own — and
reads the table detail query directly.
Deletes the helper and the four imports it was the only user of.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(managed-agent): move its four pickers onto registered selectors
All four read one route distinguished only by `resource`, with the credential
pulled from the store by name. They are now `managedAgent.agents` / `.vaults` /
`.memoryStores` / `.environments`, and `lib/managed-agents/subblock-options.ts`
is deleted entirely.
The environment filter (cloud vs self_hosted expose different fields, so mixing
them offers choices the rest of the form cannot honour) moves into the selector
with `environmentType` on the context.
Also decouples two things `canonicalParamId` was conflating. It is both a
block's serialized PARAM NAME and the key `buildSelectorContextFromBlock` reads,
so making this block's pickers resolvable appeared to require renaming its
shipped `credential` param to `oauthCredential` — a rename that would change the
serialized shape of every existing managed_agent block, and one that
`blocks.test.ts` correctly refused. A picker should not be able to force a param
rename, so the context now reads a credential off the subblock TYPE when no
canonical id supplied one. It only fills a gap: a block that declares
`canonicalParamId: 'oauthCredential'` has already resolved it, including the
basic/advanced active-member logic the type check cannot express.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(sub-blocks): delete fetchOptions — a sub-block's options are a selector or derived, never both
Completes the migration. `fetchOptions`/`fetchOptionById` are off `SubBlockConfig`,
off both controls, and out of `useFetchedOptions`, leaving exactly two ways a
sub-block gets its options:
selectorKey — a registered selector. The ONLY way to load a remote list.
Parameterized by an explicit SelectorContext, so it works on the
canvas, in the fork sync modal, and anywhere else.
options — a static array, or a pure function of the block's own values.
No I/O.
Reading the remaining callsites showed most of the "derived" ones were nothing of
the kind — they were workspace-scoped remote fetches wearing a local-looking
signature. Those became seven `workspace.*` selectors (credential providers,
credential groups + their per-group providers, secret names, raw secret names,
sandboxes, trigger types) plus `providers.openrouterEmbeddingModels`. Only the
agent block's three capability dropdowns were genuinely derived; `options` now
takes the block's values so they can say so directly. The parameter is optional,
so every existing zero-argument options function is untouched.
`imap.mailboxes` is the one selector whose account is typed rather than stored.
Its password is deliberately absent from the query key: a query key identifies a
resource, a credential authorizes access to it. `oauthCredential` is safe there
because it is only an id — a typed password is a secret, and keys are cached and
surfaced by devtools. Host, port, TLS and username already identify the mailbox
list uniquely; the password rides the body exactly as before.
`selectorExcludeSelf` replaces the one thing a shared `sim.workflows` selector
could not express. It is a declared flag rather than a blanket rule because the
answer differs per field: the Sim trigger never receives events about its own
workflow, while the Logs block legitimately reads the logs of the workflow it
runs in.
Deletes `lib/workflows/subblocks/options.ts` and `triggers/editor-state.ts`
entirely — every caller was a `fetchOptions` resolver. The live-registry test for
the trigger vocabulary moves to the selector that now owns it, keeping its
lazy-import cycle guarantee under test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(workspace-forking): make every fork-clearable sub-block reconfigurable at sync time, and lint that it stays so
`clearDependentsOnRemap` wipes every transitive dependent of a remapped parent,
and a credential mapped between environments changes value on EVERY sync — so a
dependent the sync modal could not offer was re-emptied on every push, with
nowhere to set it that stuck. Setting it in the target did not survive. 36 fields
were in that state.
The selector migration closed most of it; this closes the rest. The collector now
also emits plain text dependents (`short-input` / `long-input`), which need no
selector — just somewhere to type — and the modal's no-selector branch renders
them through the same control it already drew for custom-block inputs. It
deliberately does NOT emit the manual half of a selector-backed canonical pair:
that pair already represents the field once, and its manual member is verbatim by
policy, so offering both would show one concept twice and invite writing into the
inactive half.
`forkDependentControl` replaces the direct `customBlockInputControl` call in the
view, because `fieldType` now means two different things: a custom-block input
declares a Start FIELD type (`string`, `file[]`), while every other no-selector
dependent is a canvas SUB-BLOCK whose own type says it. They agreed by accident
before; now they are classified separately.
`check:fork-dependent-coverage` fails when a sub-block under a
credential/knowledge-base/table anchor is none of: selector-backed, a canonical
pair member, a preserved name-based type, or text. 656 dependents, zero
uncovered, no baseline — verified to fail by seeding a regression. Picked up
automatically by `check:audits` (all 30 green).
Documented in `/add-block`, `/add-trigger`, and `.claude/rules/sim-integrations.md`,
including the two rules the checks enforce: a secret never enters a selector's
query key, and a fork-clearable dependent must be reconfigurable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(sub-blocks): stop selector-backed fields rendering undefined options, and pass derived values to ComboBox
Both found by Bugbot on the migration commit; both real, both mine.
A selector-backed field carries no static `options` — that is the point — but
`Dropdown` and `ComboBox` still read it on first paint, before any fetch resolves,
and `allOptions.map(...)` is unconditional. Every field moved to `selectorKey`
(Function sandboxes, Managed Agent pickers, OpenRouter embeddings, Logs
workflows, the migrated triggers) would throw on mount. The type said the prop was
required, so nothing caught it: the callsites pass `config.options`, which is
optional on `SubBlockConfig` and now genuinely absent.
Fixed on the controls rather than by restoring `options: []` to every migrated
sub-block: the absence is correct, so the component owns the default. `options`
is optional on both prop types and falls back to a shared empty array, which also
keeps a stable identity for the memo.
`ComboBox` never got the `options({ values })` wiring `Dropdown` received, so
agent's reasoning-effort, verbosity and thinking-level lists — all comboboxes —
silently stayed on their generic fallback instead of narrowing to the selected
model. Wired the same way, reading the block's own values from the store.
`selector-backed-subblocks.test.ts` pins the invariants against the real registry:
a named selector exists and can list, a selector-backed field never also declares
static options, and a field whose selector is gated on context declares the
`dependsOn` that rebuilds it. That last one immediately caught a third bug —
`clickup.triggerWorkspaceId` had no `dependsOn`, so its list would have loaded
once, empty, and never refetched once a credential was picked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(selectors): restore the credential-group provider label resolver, and probe getQueryKey for missing dependsOn
Two findings from a final adversarial pass over the migration, both the same
class as the three the review bots caught: something a `fetchOptions` sub-block
declared that its replacement selector quietly does not.
`credential-group.providerFilter` had a `fetchOptionById`;
`workspace.credentialGroupProviders` had no `fetchById`, so the canvas card
summarising several stored provider ids lost every label. The field is
multi-select, which is exactly when a label has to resolve without the full list.
The `dependsOn` assertion in `selector-backed-subblocks.test.ts` only probed
`enabled` against three hand-listed context fields, which is why it caught
`clickup.triggerWorkspaceId` and would have missed the rest. It now probes
`getQueryKey` as well — a selector's key names every context field its RESULT
depends on — and derives the sub-block-sourced set from
`SELECTOR_CONTEXT_FIELDS` rather than a literal. Verified by deleting a real
`dependsOn`: it fails naming the field and the fields it depends on.
Also checked and NOT changed: `display.ts` and the copilot dropdown validator
both guard `options` before use, so stripping `options: []` does not reach them.
The validator's behaviour does shift from "reject every value" (an empty
`validIds` array matched nothing) to "skip validation", which is a relaxation
rather than a regression. `function.sandboxId` kept its `dependsOn: ['language']`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: re-record the page-graph baseline after staging's growth consumed its allowance
CI's "Repo audits" step failed on `check:tool-registry-boundary`. Measured before
touching anything, because the reported growth (+32 and +42 modules on two
routes) looked like this branch had dragged the selector registry somewhere new.
It had not. Recording a baseline on clean `origin/staging` and diffing against
this branch attributes the growth precisely:
this branch: +1 to +4 modules per route, +35 total across 25 routes
staging: the rest
Staging's six merged commits landed both failing routes at exactly their
tolerance — knowledge/[id] at +31 of an allowed +31, layout at +41 of +41 — so
`check:tool-registry-boundary` passed there with nothing left over. This branch's
+1 tipped both past the line. The next PR to touch anything would have tripped it
just the same, whatever it contained.
The +1..+4 is the selector consolidation's real cost: `selectorRegistry` is one
static object, so a page reaching any selector reaches every provider, and this
branch adds four (hubspot, managed-agent, imap, workspace). That is the same cost
the 27 existing providers already impose, and it is what buys one option-list
mechanism that works off the canvas.
Also tried deferring the workspace provider's data-layer imports to fetch time.
Reverted: this checker follows dynamic imports, so the numbers did not move,
leaving only a Promise.all-of-imports shape that reads worse than the 27 sibling
providers it sits next to.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(workspace-forking): actually apply text dependents, and resolve labels against the full selector context
Both from Bugbot; both real, both mine.
**Text dependents never persisted.** `applyDependentOverrides` allowlisted
`dependsOn && selectorKey`, so the plain text fields the collector started
emitting were offered in the modal, stored, and gated on by the Sync button —
then dropped on apply. The field stayed wiped on every push and the typed value
went nowhere, which is the exact treadmill the feature existed to end.
The cause was the rule being written twice. `reconfigurableDependentIds` is now
the single definition of "a dependent the modal can offer AND the sync can write
back", used by the collector and by the apply side. A test asserts the two agree
by round-tripping through `applyDependentOverrides`, and fails against the old
allowlist.
**Provider labels stayed raw ids.** `useDynamicSubBlockOptionDisplayName` called
`fetchById` with a `workspaceId`-only context, which silently fails any selector
scoped by a sibling — `workspace.credentialGroupProviders` needs the group before
it can name a provider, so the `fetchById` restored last round returned null
every time. It now builds the block's real context with
`buildSelectorContextFromBlock`, the same one the canvas uses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(queries): scope the sub-block label cache by the selector's own context
Follow-on to
|
||
|
|
e3a4874ece |
feat(integrations): add Bitbucket Cloud (#6860)
* feat(integrations): add Bitbucket Cloud * fix(bitbucket): enforce selector workspace slugs * fix(bitbucket): overfetch small pipeline log tails * fix(bitbucket): harden provider edge cases * fix(bitbucket): accept provider diff redirect specs * fix(bitbucket): stop advanced-field leakage and harden log, status, and selector paths Splits the `closeSourceBranch` advanced subBlock into per-operation ids. Advanced fields serialize without evaluating their condition, so a value set on Create Pull Request reached Merge Pull Request and closed the source branch unprompted. Also: - read step logs through the byte-capped server transport and map an empty-log 416 to an empty result, keeping a genuine 416 an error - trim a step log's partial leading line after the character cap rather than before, and never return an empty log when the retained window held content - surface Bitbucket's `error.detail` alongside `error.message` - treat commit-status `key`/`state` as nullable so one malformed row cannot drop a page - match repository `full_name` case-insensitively and reject dot segments in a workspace slug before the outbound request - type `reviewerAccountIds` as the comma-separated string it is - trim optional Bitbucket query strings; correct the token lifetime to two hours --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Theodore Li
|
1372977d07 |
feat(setup): publish standalone self-hosting package (#6849)
* feat(setup): publish standalone self-hosting package * fix(setup): refresh discovered compose installs * improvement(setup): unify repository command * fix(setup): harden standalone package launch * Update README.md * fix(setup): isolate standalone compose installs * fix(setup): restore default stopped installs |
||
|
Vikhyath Mondreti
|
483ff12b6f |
feat(billing): align enterprise reporting periods (#6851)
* feat(billing): align enterprise reporting periods * fix(billing): harden enterprise reporting flow * fix(outbox): preserve handler compatibility * fix(billing): bound enterprise provisioning reads |
||
|
Waleed
|
4f9d5f33b0 |
improvement(search): search every folder, and document real API error bodies (#6861)
* improvement(search): search every folder, and document real API error bodies Search on Files, Tables, and Knowledge was ANDed with the open folder, so a query only ever matched that folder's direct children — and the query was not cleared when you entered a folder, filtering the folder you just opened down to the same matches. A non-empty query now searches the whole workspace, a Location column names each result's folder, and opening a folder ends the search. Also gives GET /api/v2/files a `recursive` flag, and replaces the single shared OpenAPI error example — which showed `BAD_REQUEST` under every status tab — with one real body per status. * fix(search): discard the search term on clear instead of masking it `useSearchFilterValue` returned the debounced term whenever the input was non-empty, so clearing only hid the settled needle. The mask lifted on the next keystroke while the debounce still held the pre-clear term — opening a folder and typing within the window searched the whole workspace for the query the user had just abandoned. A clear now resets the settled term rather than hiding it, adjusted during render so the reset is visible to the render that follows the clear. The initial state is seeded from the first value so a deep-linked `?search=` still filters on the first render. |
||
|
|
681a8ec427 |
feat(resources): empty-state graphics for knowledge, tables, logs, and files (#6828)
* feat(resources): empty-state graphics for knowledge, tables, logs, files, skills Four of the resource pages (knowledge, tables, logs, files) had no empty state at all — `Resource.Table` painted column headers over a blank scroll area and stopped there. Skills had a `: null` branch for zero data. Adds a graphic per resource, drawn in the editor vignette's recipe: take the product's own primitives, shrink them, strip the content to skeletons, and let the composition bleed off the frame edges. - Knowledge — a document fanning into the chunks it is embedded as, using the editor's 6px smooth-step connector language in --workflow-edge - Tables — a sheet of cells running off two edges with one cell in an edit ring - Logs — runs stacked newest-first, their trace spans staggered into a waterfall - Files — a folder held open with one file still above its dashed landing slot - Skills — a skill card opened far enough to show the tools bundled inside `Resource.Table` gains a sanctioned `emptyState` slot rendered below the column headers when `rows` is empty, so the chrome guarantee still holds. Each page shows the graphic only for true zero-data — never for a search or filter that matched nothing, never inside an empty subfolder, and (logs) never before the first page of runs lands. Also ports the shared `EmptyState` frame from the editor branch so this branch stands alone, and adds a review-only /empty-states-preview gallery route. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * improvement(tables): redraw the empty-state graphic in the house grayscale Matches the workflow editor's vignette and the landing feature graphics, which between them use no brand colour at all — every one of them is built from neutral tokens. Two corrections: - The blue edit ring is gone. Nothing in the reference graphics carries a hue, and it was the loudest element on the page. - `--surface-4`/`--surface-5` are near-white in light mode (#f5f5f5/#f3f3f3), so skeleton geometry built on them dissolved on a white card. Bars now mix `--text-secondary` into transparent at graded strengths — a real mid-grey that inverts with the theme, which is the idiom the editor vignette already uses for the one bar it needs you to see. Also drops the full-composition mask. The editor vignette keeps its block fully opaque and fades only the connector strokes leaving the frame; masking everything is what made the miniature read washed rather than deliberate. The card is crisp now and the continuation is drawn the way a real table draws it — an overflow fade at the edge the columns run off. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * improvement(tables): strip the empty-state graphic to ruled lines and a corner fade Minimal pass. The card is gone — no border, no fill, no header shading, no type squares. What is left is the grid itself: hairline rules in `--border-1`, ink bars at two strengths, and the one cell held in an edit ring. With no card fill the grid sits directly on the page, so it can dissolve into the background instead of ending at a border. The fade is the landing page's own idiom — two gradients intersected (`mask-composite: intersect`), crisp at the top-left and gone through the bottom-right, the same construction `workflow-graph-preview` uses. Two placement notes: - The grid is offset right of frame centre. A diagonal dissolve puts the visual mass toward its opaque corner, so centring the geometry would leave the graphic reading left of the copy beneath it. - The selected cell sits in the quadrant the fade leaves fully opaque. A selection ring dissolving mid-stroke reads as a rendering fault, not a detail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * improvement(tables): make the selected cell opaque and a shade darker The ring mixed `--text-secondary` into `transparent`, so the grid rules running underneath showed through its own stroke. Mixing into `--bg` instead holds the same apparent value while staying opaque, and still inverts with the theme. Raised 32% -> 46% so it reads as chrome rather than more content, and added a stacking context: neighbouring cells are later siblings, so their rules were painting over the ring's right and bottom edges. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * improvement(tables): round the empty-state grid's crisp corner 6px on the top-left only — the one corner the fade leaves intact, and the same radius the workflow editor's vignette uses. The other three dissolve, so there is nothing there to round. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(knowledge,tables): redraw knowledge's empty state and add docs/create chips Knowledge gets the same treatment tables just went through: no brand colour (the `--brand-knowledge` accent is gone), no card chrome, ink mixed from `--text-secondary`, and the landing page's intersected corner fade. The graphic is a document and the chunks it is embedded as. Its fade is held back further than the tables grid on both axes — the document has to stay whole for the graphic to mean anything, so only the chunk grid may trail off. The three chunks the edges actually land on are the only filled ones; filling the whole first column left a chunk with no edge feeding it. Both empty states now carry two chips in the frame's action slot — a docs link and the create action, each running the same handler as the header's primary chip and inheriting its disabled state. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * review(knowledge): three candidate depictions, and lead with the create chip Chip order swapped on both empty states — the primary action reads first, the docs link second. Adds a review-only `knowledge-alternates.tsx` rendered in the preview gallery, because the document-to-chunks graphic is not landing. Three directions: - A. the embedding mesh — the landing hero's own knowledge-base panel already draws a base this way (`stage-kb.tsx`), so this is the house depiction rather than a new invention - B. a stack of documents — the most literal reading, at the cost of colliding with what the files empty state wants to draw - C. a query and the passages that answered it — depicts what a base is for, which is what the description copy actually promises Delete this file and the gallery entries once one is chosen. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(knowledge): draw the empty state as an isometric set of volumes Replaces the document-to-chunks diagram, which read as a workflow graph rather than as a knowledge base. Built on the landing page's iso-illustration recipe rather than a new one: `ISO_STROKE` contours (`--text-subtle` mixed toward `--text-muted`) at the shared 3.2 stroke width, faces filled from the three-tier surface ramp brightest-on-top, round caps and joins. Geometry is authored in a large unit space so that 3.2 lands as a hairline once scaled to empty-state size — the same reason the landing marks draw 3.2 into a ~526-unit viewBox. The projection and faces are computed rather than hand-authored as path data, so the volumes stay coherent when the geometry is retuned. No corner fade here. The fade belongs to repeating structures that mean the same thing cropped — the tables grid keeps its meaning with two columns or four. A discrete object does not, which is also why the workflow editor's vignette keeps its block fully opaque. Drops the three candidate depictions now that the direction is settled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * improvement(knowledge): brand the front volume instead of laying a page beside it Drops the loose page on the ground and puts the knowledge-base mark on the front volume's cover — the same `Database` glyph the sidebar and the page header use, so the empty state names its own resource. The mark is laid into the cover's plane rather than drawn over it. The cover is the face at max x, spanned by the volume's depth across and its height up; walking those two edges gives the face's basis vectors in projected space, and an affine matrix built from them maps flat artwork into the face. So the glyph skews with the isometric, and because both vectors derive from the box, retuning the volumes carries the mark with them instead of stranding hand-fitted path data. Its stroke is pre-divided by the same factor the matrix scales by, so the glyph's contours land at the volumes' weight rather than four times it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * improvement(knowledge): bore through the front volume, and fade the set back Replaces the mark on the cover with a hole through it. The bore is authored as a plain circle in the cover's own plane and skewed into an ellipse by the face matrix. Its far mouth is the same circle stepped back through the volume: boring straight back is a world step of `-w` along x, and solving the cover-plane matrix for the local offset that produces it gives `(+w, -w)`. The sliver of near mouth the far mouth fails to cover is exactly the wall you see down the hole, so the depth falls out of the geometry rather than being drawn by hand. Down the hole the near mouth is floored in a tone darker than any outer face — the wall turns away from the light — and the far mouth is painted in the cover tone of the volume standing behind it, because looking through a hole in the front volume lands on that volume's face, not on the page. Corners stay square. Rounding was tried and reverted: rounding each face separately notches every corner where three faces meet, and rounding the silhouette instead cost a clip per volume for a softness the set did not want. The tables grid's corner fade is applied along the other diagonal. There it dissolves toward the bottom-right because a grid keeps its meaning cropped; here the set recedes up and to the left and the front volume carries the bore, so anchoring at the bottom-right eats into the back of the stack and reads as more volumes behind. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(resources): logs and files graphics, and settle the set as one collection Logs is an activity feed — newest run lifted onto its own card, older ones settling behind it. The relative stamps are the only literal text in any of these graphics; everything else stays skeleton, so nothing here has to be translated or kept true. Files is a folder with sheets standing proud of its front panel. Depth comes from the surface ramp rather than shadow, which would need separate light and dark recipes where the ramp inverts on its own. The tab's diagonal is filleted at both ends and every outer corner shares one radius — mixing radii, or running the diagonal into square junctions, made the corners fight at this size. Consistency pass across the set: - Titles are the resource name alone. "No tables yet" earned nothing the description does not already say. - The knowledge mark is mirrored so its bore faces left. Rebuilt on the geometry rather than flipped, since a flip would have put the shading on the wrong side. Its contours are thinned and mixed toward `--border-1`: the landing marks are the focal art of their section, but this one sits beside a ruled grid whose lines are 1px, and full-weight contours read as ink next to it. - The logs feed is sized to the same ~148px footprint as the rest. The frame centres graphic and copy together, so a taller graphic pushes its title out of line with the others' and the set stops reading as one thing. - Every empty state carries its create action and a docs link, each running the same handler as the header's primary chip. - Fades run whichever way the subject recedes: the tables grid to the bottom-right, the knowledge set up and right, the logs feed down, the folder up. Fixes a duplicate React key in the knowledge mark — the volumes stack along y now, so keying on `box.x` gave every one of them `0`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * revert(skills): drop the skills empty state Removed at request. The skills list goes back to rendering nothing for zero data, which is what it did before this branch. Takes `vignette.tsx` with it — the shared stage and skeleton bar were left over from the first pass, and skills was the last thing still importing them once the other four graphics were redrawn. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * cleanup(resources): fix empty-state flashes and share the iso ramp Drops the review-only preview route and gallery, which the branch always meant to delete before merging. Three ways the zero-data graphic painted over a workspace that has content: - The gate read the instant URL search term while `rows` is filtered by the debounced one, so clearing a search that matched nothing showed the full "you have nothing yet" state for one debounce window. - Nothing gated on the list still loading. Knowledge and tables hydrate from a server prefetch that is allowed to seed nothing, and the files list deliberately seeds nothing above 300 rows — so the emptiest-looking screen was shown to the fullest workspaces. - The filters are part of the query key and every list keeps the previous key's data, so `isLoading` is false across a filter change. Only the placeholder gate suppresses the graphic during that refetch. Also folds the re-declared isometric fills and stroke back onto the shared `iso-illustration-style` source they were copied from, so a change to the iso ramp reaches this mark too; only the stroke width still diverges. The static face paths move to module scope, the bore interior becomes a named component so its note is TSDoc rather than a JSX comment, and the four identical docs chips become one. * simplify(resources): reuse the iso recipe and let the frame own its layout Hides the empty-state graphic behind `error` as well. A failed load also leaves `rows` empty, and inviting someone to create their first item is the wrong answer to a request that did not complete — all four pages only logged the error, so the zero-data copy was what a failed load actually rendered. `iso-illustration-style` moves out of the landing route group to `components/iso/`. Importing it from a workspace route was the only workspace-to-landing edge in the app, one directory away from an `iso-marks` barrel that pulls ~10KB gzipped of illustration components — a hazard for whoever needs the second constant. The contour recipe is now shared too: `createIsoLineProps` takes an optional stroke width, so the knowledge mark stops re-declaring it and only its weight diverges. `EmptyState` owns the action row's layout, so the three pages with two chips drop their wrapper div and every empty state's chips sit identically. Its unused `className` prop goes with them. Also drops the `height` prop that had one caller passing its default, and the `CORNER` constant that promised single-sourcing the path's four bare literals did not honour. * simplify(resources): decide list emptiness in one place "This list holds nothing" was derived in four pages, each with the same seven clauses under the same nine-line comment. Adding the `error` gate one commit ago took four identical edits, and the skills empty state that was reverted off this branch would have made it five copies. `isResourceListEmpty` now owns the rule and the reasoning behind each gate. Logs omits the folder argument because it has no folder navigation; the other three pass theirs. `Resource.Table` also wraps the slot in its own growth box, so the empty state centres because the table says so rather than because the node handed to it happened to carry `flex-1`. * chore(audits): re-record the page module-graph baseline The four empty-state graphics and the shared frame add **+10 modules** to each of the five routes that render them — measured against `origin/staging`, not against the recorded baseline: files/[fileId] 1958 -> 1968 files 1958 -> 1968 knowledge 2167 -> 2177 logs 1727 -> 1737 tables 1817 -> 1827 The baseline itself was last recorded in #6697, and staging has drifted up to +29 on tables since — inside the max(25, 2%) tolerance on its own, but close enough that this +10 tipped it over. So the failure was the stale baseline meeting a small real addition, not a heavy import. The other 29 entries move only by that accumulated drift. The graphics stay eagerly imported on purpose: an empty state is the first thing a new workspace paints, and deferring ~4KB gzipped behind a chunk request would trade a shared, already-fetched module for a visible pop on the one screen where the product has to look like it works. * fix(resources): hold the empty state until folders resolve Folder rows share the list with resource rows, so a workspace whose only contents are folders has an empty `rows` until the folder tree lands — and got the "create your first item" graphic in the gap. The resource list's own loading gates never covered it because the folder tree is a separate query. `useFolderNavigation` already exposes `foldersResolved` (`isSuccess && !isPlaceholderData`) for exactly this hazard — it guards the ancestry index against evicting a folder id it has not loaded yet. Knowledge and tables pass it straight through; files reads the same two flags off `useWorkspaceFileFolders`, which it calls directly. Logs omits it, as it has no folders. --------- Co-authored-by: andresdjasso <andresdjasso@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Vikhyath MondretiandClaude Opus 5
|
521348b529 |
feat(secrets): record which secrets each run resolves, and surface it per secret (#6823)
* feat(secrets): record which secrets each run resolves and surface it per secret
Redaction stops a value at a boundary but cannot stop code that never emits it —
a Function block can print a key one character at a time and nothing ever matches
the secret. That is undecidable in general, so this adds the other half of the
posture: attribution.
Every run now records which configured secrets it actually resolved, under whose
identity, through which surface (workflow, Sim agent, MCP). The data already
existed in ResolvedSecretTraceRegistry.addActiveEntry and was persisted only for
paused runs; this persists it for every terminal path.
Execution logs cannot answer this. They store the whole available encrypted
environment rather than what a run referenced, they evidence a secret only where
value-matching redaction happened to fire, and they expire under
logRetentionHours — while "who has touched this key" outlives any single run.
- secret_usage: per-UTC-day rollup keyed by workspace, secret, scope, owner,
source, workflow, actor. A one-minute schedule touching three secrets would
otherwise write thousands of rows a day, which is also why this is not
audit_log. workflow_id/actor_user_id use '' sentinels rather than null so the
unique key works on Postgres 14 without NULLS NOT DISTINCT, and are not FKs:
they are historical facts, and an onDelete would rewrite a key column.
- secret_owner_user_id is part of the key. Two people can hold a personal secret
under one name and a shared personal secret resolves for a caller who does not
own it, so name and scope alone do not identify a secret. It is NOT the actor:
a scheduled run resolves the workflow owner's personal slice under the
workspace's execution actor.
- Direct environment reads are now detected in JS (TypeScript AST), Python
(tokenizer-checked) and shell (quote/heredoc-scanned), so a secret read as
environmentVariables['K'] or $K enters the run's provenance instead of going
unredacted. Each detector prescans for names that are actually configured
secrets before paying for a lex or quote-frame pass.
- Copilot integration tool calls are covered: resolveCopilotEnvReferences
substitutes {{SECRET}} into user-only params, which is a real use.
- See usage lives behind a credential-admin gate, using the same predicate that
reveals the value; members get a disabled chip explaining why.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(audit): register the secret-usage route in the validation baseline
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): keep rollup metadata with its run, and stop shadowed bindings faking usage
Review round 1.
- record.ts: last_execution_id/last_trigger were assigned unconditionally while
last_used_at was chosen by greatest(), so two runs completing out of order split
one row between them — the newer run's timestamp beside the older run's execution
id, making "View log" open a run the row does not describe. Both are now guarded
on the timestamp actually advancing, so the row's metadata always belongs to the
run that owns its timestamp.
- javascript.ts: a local binding named environmentVariables (declaration, parameter,
destructured binding, or bare reassignment) made reads off the user's own object
look like mounted-secret reads. Any such binding now disables detection for the
file; the AST already had parent pointers, so this is a kind check during the
existing walk.
- python.ts: same class of bug with no parser available, so the rule is an allowlist
— every mention of the binding must be a literal subscript or .get(), otherwise
detection is off for the file. This also subsumes the cross-line attribute case
(other.\n environmentVariables['K']), which the previous space-and-tab look-behind
missed.
Under-reporting is the safe direction here: a trail that claims a use that never
happened is worse than one that misses a use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(db): format the generated migration snapshot
CI runs lint:check across every workspace; the drizzle-kit output in packages/db
had never been through biome, so the branch was green locally (where lint had
only been run inside apps/sim) and red on CI. Whitespace only — both files are
byte-for-byte identical once parsed, and drizzle-kit still reports no pending
schema diff against the reformatted snapshot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): detect every rebinding of the environment identifier, not just declarations
Review round 2. A bare `for (environmentVariables of rows)` has no declaration to
key off, so the previous check missed it and reads of the loop value were still
recorded as secret usage.
Rather than extend the hand-rolled node-kind list, this reuses the pair the same
file already applies to reject a placeholder in a write position:
isDeclarationIdentifier covers declarations, parameters, destructured bindings and
imports, and isWriteIdentifier covers every assignment operator, ++/--,
destructuring targets, and for-in / for-of initializers.
That also closes four forms neither the review nor the original check named:
logical (||=) and nullish (??=) assignment, and object and array destructuring
assignment. Six of the eight added cases fail against the previous check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): apply the rebinding rule to shell, and say when a run's log is gone
Review round 3, plus the docs that were left claiming the old behavior.
- shell.ts: a script that writes a configured name (API_KEY=local, export/local/
readonly, read, for, unset) expands its own value from that point on, not the
mounted secret, so recording it claimed a use that never happened. Every mention
of the name must now be a `$NAME` / `${NAME}` expansion, matching the allowlist
shape the Python detector already uses. Applied per name rather than per file:
JavaScript and Python shadow one object holding every secret, whereas rebinding
one shell variable says nothing about the rest.
- The usage trail deliberately outlives execution logs, so a row routinely names a
run whose log has been pruned. The read now left-joins workflow_execution_logs on
its unique execution_id and reports availability, and the panel renders the chip
disabled with the platform tooltip instead of linking into an empty Logs view.
Three states: no run to link, a run whose log is gone, and a live link.
- Docs said a direct environmentVariables/$KEY read does not activate masking,
which this branch changes. Corrected in credentials.mdx, function.mdx and the
logging FAQ, and the recognition limits are now written down: runtime-built
names, reassigned bindings, and reads that cannot be told apart from text.
Added a "See usage" section covering who can see it and why an empty trail
means "nothing recognized" rather than "never used".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): writing a name is not reading it, and a bare mention is not a rebinding
Review round 4.
- javascript.ts / python.ts: `environmentVariables.API_KEY = 'x'` and
`delete environmentVariables.API_KEY` touch the name without ever reading the
mounted value, but the detectors matched the member access and recorded a use
that never happened. JavaScript now asks the same isWriteIdentifier the
placeholder rewriter uses (its parameter is widened to ts.Node — the body
already walked generic nodes, so this is a type change, not a behaviour one)
plus a delete check; Python excludes a subscript followed by `=` and a `del`
target.
- shell.ts: requiring every mention of a name to be an expansion also fired on
text that binds nothing — a comment naming the key, or `echo "API_KEY=$API_KEY"`
where the literal is an argument rather than an assignment — and dropping those
cost masking on a genuine read. It now looks for actual writes: an assignment at
command-word position, a binding builtin, `printf -v`, or a `for` target.
The two directions are not symmetric, which is why this errs toward detecting
the read: missing a write records a use of a secret the script only had in its
environment, a misleading audit row and nothing more, since masking still
searches for the real value and will not find it. Over-detecting a write
suppresses masking on a value that does reach the log.
This also makes the code match what the docs already described — skipping after
a rebinding, not after any mention.
13 tests added; 11 fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): an update reads before it stores, and a del target may be parenthesized
Review round 5. The first of these is a regression from round 4.
- javascript.ts: reusing isWriteIdentifier to answer "is this a read" was wrong.
That predicate answers the rewriter's question — is this a target the
substitution must refuse — so it treats every assignment operator alike, which
is correct there and wrong here: `+=`, `||=`, `??=`, `++` and `--` all load the
current value before storing, so they are genuine reads and were silently
losing their masking. Only a plain `=` stores without reading. Replaced with a
purpose-named predicate, and isWriteIdentifier's parameter is narrowed back to
ts.Identifier now that nothing else needs it widened.
A test committed last round asserted the wrong behaviour for `+=`; it has been
corrected rather than left to pin the bug.
- python.ts: `del (environmentVariables['K'])` slipped past a check that looked
only at the characters immediately before the match. It now isolates the
enclosing logical line and tests whether that is a del statement, which also
covers `del((x))`, `del(x)`, `del a, x`, and a del after a semicolon.
12 tests added or corrected; 10 fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): stop excluding Python writes, which kept leaking in the unsafe direction
Review round 6. Greptile found that `del environmentVariables[environmentVariables['K']]`
had its inner access — which computes a key, so it is a genuine read — skipped
along with the delete, leaving that value unmasked.
The narrow fix was another textual rule. Instead this removes the write and delete
exclusions from the Python detector entirely, because they were optimizing the
wrong direction.
`resolvedSecretNames` feeds `outputSecretMatcher`, an exact-value matcher over the
output. Naming a secret the code never read costs nothing there: the matcher scans
for a value that does not appear. Failing to name one that was read leaves it
unmasked. The two error directions are therefore not comparable, and the
exclusions bought only audit-trail tidiness while every heuristic they needed has
so far leaked into the dangerous side — first a parenthesized target, now a nested
read. A `del` or an assignment is reported like any other access.
JavaScript keeps its exclusion: a real AST answers the question per node, with no
text to misread, and it has produced no such hole.
Net 30 lines removed from python.ts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): report recognized reads instead of proving they are not reads
Review round 7. Greptile flagged both directions at once — false usage from
reporting a write target, and unmasked secrets from the file-wide shadow flag —
so I traced what the signal actually drives before choosing.
The chain: the compiler's names feed outputSecretPlaintextsByName and the
exact-value matcher, NOT context.resolvedSecretNames, which starts empty. After
execution activateOutputSecretProvenance scans the output and adds only names
whose plaintext actually appeared; those become __resolvedSecretNames, which
tools/index.ts turns into recordResolved calls, which is what the usage trail
reads.
So a compile-time false positive produces no usage row on the ordinary path — it
only hands the matcher a value the code never emits. It does produce one on the
!projection.safe fallback, where the system already over-approximates by design.
A false negative, by contrast, keeps the value out of the matcher entirely, so a
genuinely read secret is never masked on any path.
That asymmetry decides it, so every "prove this is not a read" mechanism is gone:
- javascript.ts: the file-wide shadow flag. A helper declaring its own
environmentVariables discarded genuine reads of the mounted binding everywhere
else in the file — Greptile's security finding, and real.
- python.ts: the allowlist requiring every mention to be a subscript or .get().
Same hole: passing the dict to a function suppressed unrelated reads.
- shell.ts: the rebinding check. It had the same hole in a form nobody flagged —
`echo "$API_KEY"; API_KEY=local` dropped the first read, which is of the real
secret.
What stays is the question of whether the text is code at all — strings, comments,
single quotes, quoted heredocs — plus the receiver check that `other.environment
Variables['K']` is a different object, and JavaScript's node-precise write/delete
exclusion, which cannot suppress a read elsewhere.
Net 215 lines removed across the three detectors and their tests. Docs updated:
the rule is now stated as reporting rather than proving, and that See usage may
occasionally list a secret the code had available but did not read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(secrets): drop the last write-vs-read special case
`environmentVariables` is a plain object deserialized from the run payload
(route.ts:206), not a handle on the stored secret. Assigning to it changes
nothing outside the sandbox and is discarded when the run ends, so separating a
write from a read bought almost nothing while leaving JavaScript as the one
language still trying to prove a read is not a read.
Every language now follows the same rule: report a recognized read of a
configured secret name. The only exclusions left are facts rather than
inferences — the text is not executable (string, comment, single quote, quoted
heredoc), the receiver is a different object, or the name is not statically
knowable.
Docs note that assigning to the binding does not edit the secret.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(secrets): ship only the fields the trail actually shows
Five fields crossed the API and reached no reader: usageDate, firstUsedAt,
actorEmail, workflowId and actorUserId. The panel renders the timestamp, the
trigger, what used the secret, the actor's name, the run count and the run link;
everything else was projected, serialized and discarded.
first_used_at is dropped from the table as well. Nothing read it, and inside a
per-day bucket "first used that day" says nothing next to "last used that day" —
so it was a column written on every run for no question anyone asks. The upsert
loses its least() with it. Migration regenerated; the identifier columns behind
the joins stay, they simply are not returned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): report referenced code secrets, not only ones that surface in output
The Function route activated a secret's provenance — and therefore its usage row
and downstream masking — only when the exact value appeared in the result,
stdout, or error. That gate made the trail miss silent use entirely: a key that
authenticates an API call and is never echoed reported nothing, and so did the
founding scenario of this feature, a key exfiltrated character by character. The
innocent run that echoed a key got a row; the run worth catching did not.
Activation now follows the referenced set the compiler already computes: resolved
{{KEY}} bindings plus recognized direct reads, filtered to configured values —
the same set the unsafe-projection fallback already activated. An extra name only
hands the output matcher a value that never appears; configured-but-unreferenced
values are still never included. The output-scan activation path and its surface
helper are deleted rather than kept alongside.
One old test pinned the gate ("does not activate a referenced secret that does
not cross the Function result"); it now asserts the reverse, with the reasoning
attached. Two new tests pin the char-split exfiltration and the silent API-call
case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): shell escaping is backslash parity, not adjacency
Review round 8. `\\$API_KEY` is an escaped backslash followed by a LIVE expansion
— bash prints `\` plus the value — while `\$API_KEY` is an escaped dollar and
stays literal. Checking only the character adjacent to `$` read every even run as
escaped, dropping a real read from usage and masking alike; verified against
bash before fixing.
The scanner now counts the run of backslashes before the `$` and skips only odd
runs, the same parity rule logicalLineEndAfterContinuations in this file already
applies to line continuations. Six-case parity table added; the three even-run
cases fail against the previous check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): recognize destructured environment reads
Review round 9. `const { API_KEY } = environmentVariables` delivers the value by
name with no property- or element-access node in the AST, so the member-access
walk missed it entirely — and a missed read leaves an emitted value unmasked,
the dangerous direction.
The AST walk now also recognizes the declaration form (shorthand, renames,
defaults, string-literal keys), the assignment form ({ KEY } = env), and a
...rest element — which names no key but takes every value, so it reports every
configured name; the alternative left `const { ...all } = env; return all`
entirely unmasked. A computed key stays unrecognized, the same runtime-name
boundary as a computed subscript, and a receiver that is not the bare identifier
is not attributed.
Nine cases added; the six positive ones fail against the previous walk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): one receiver rule for destructured reads, parentheses included
Review round 10. Two accurate findings, folded into a generalization instead of
two more special cases:
- A parameter default (function f({ API_KEY } = environmentVariables)) and a
binding-element default are the same by-name delivery as a variable
declaration. The detector now keys on the ObjectBindingPattern itself and
checks its parent's initializer, so every declaration position follows one
rule instead of per-kind arms.
- Parentheses group without changing the receiver, so (environmentVariables) is
unwrapped before the identifier check — in the destructuring arm AND the
member-access arm, which had the same hole unreported.
Declined the for-of-over-array-literal finding: the receiver there is a
container, not the environment object, and following data flow through
containers has no fixed point — the same documented boundary as aliasing and
computed keys. A test pins the boundary so it reads as chosen, not missed.
Eight cases added; the seven receiver-rule cases fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): a dot in prose is not a qualifier, and a literal computed key is a subscript
Review round 11. Both findings were implementation-narrower-than-rule, fixed by
consulting authorities the detectors already had rather than adding new ones:
- python.ts: the receiver walk crosses whitespace so a parenthesized `other.` on
a previous line is seen — but it landed on a comment's final period
(`# Load the value.`) and discarded the genuine read on the next line. The
landing position is now checked against the same lexer ranges that filter the
candidates, which is also why the receiver check moves after lexing.
- javascript.ts: `const { ['API_KEY']: key } = environmentVariables` is the
element-access rule in pattern position, so a computed key holding a string
literal resolves like a literal subscript; any other computed key keeps the
runtime-name boundary a computed subscript already has.
Eight cases added; the comment-period case and all three literal-computed-key
cases fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Waleed
|
9328e66d69 |
feat(account): let users delete their own account (#6831)
* feat(account): let users delete their own account Adds a GDPR self-serve account deletion path: a preflight that reports what deletion would remove and every reason it would be refused, and a confirmed delete that erases the account and everything only it can reach. Deletion refuses while the account is still entangled rather than reassigning its content. Most tables reference user.id with ON DELETE CASCADE, and those cascades do not distinguish content in the account's own workspace from content it created inside somebody else's, so each blocker names the existing action that untangles it (leave the workspace, leave the organization, cancel the plan) — all of which already hand work over on their own tested paths. * fix(account): make deletion atomic and re-check privacy at delete time Reorders the teardown so nothing irreversible happens before the deletion is certain: anchors are handed over first (the fallible step, while everything is still recoverable), the workspace and user deletes now share one transaction, and the object-storage purge runs only after that commits. The workspace delete also re-checks inside the transaction that each workspace is still private, so a membership granted between the preview and the delete aborts the whole thing instead of destroying the new member's access. * fix(account): close deletion gaps found in review - Run the whole teardown in one transaction. The billing and ownership handovers now take the caller's transaction, so a refused deletion can no longer leave a workspace reassigned for a deletion that never happened. - Fail closed on a subscription read error. getHighestPriorityPersonalSubscription defaulted to returning null, which read as "no plan" and would have erased an account Stripe was still billing. - Erase the account's profile picture. It is personal data under our own storage prefix; an external provider avatar is left alone. - Enforce the storage purge cap while collecting keys rather than after, so an oversized account cannot exhaust memory before the cap applies. |
||
|
Waleed
|
1e2572dd8a |
fix(desktop): guarantee the update feed resolves the newest release (#6827)
The stable channel reads a GitHub release list it shares with web-app releases, SDK tags, and legacy prereleases, but only ever looked at the first 30 entries. Once enough unrelated releases stack on top, the feed 404s and every stable shell silently stops updating. Walk pages (100 per page, up to 5) until one yields a release for the channel, and fail the feed rather than serving an older build when a page cannot be read. Also point the update gate's manual download at a new /api/desktop/update/download redirect, which resolves through the same channel selection. It previously opened GitHub's repository-wide latest release, which can be a tag carrying no desktop artifact at all. |
||
|
Justin BlumencranzandWaleed Latif
|
0c34e69fdc |
feat(files): upload and safely extract ZIP archives (#6782)
* feat(files): support zip extraction * fix(files): harden zip extraction safety * fix(files): batch extraction notifications * fix(files): defer rollback storage cleanup * fix(files): restore reliable drag uploads * fix(files): use explicit archive extraction route * fix(ci): account for archive extraction route * refactor(files): bound every archive extraction and trim the extractor's option surface `maxMaterializedItems` was opt-in, so only the new unzip route bounded its output tree — the copilot `materialize_file` and `POST /api/tools/file/manage` extract paths had no cap on folder creation at all. An archive within MAX_ARCHIVE_ENTRIES can still imply far more folders than files, so the cap now defaults to MAX_WORKSPACE_FILE_BULK_AFFECTED_ITEMS and applies to all three callers. `materializedRootFolderCount` was a hand-maintained number that had to agree with what an opaque callback would create, and the callee could not check it; drift surfaced only as an over-limit archive slipping past the cap. It is now derived from whether `prepareRootFolder` ran, so the contract is just "the callback creates exactly one folder". Also single-sources the ArchiveError -> HTTP status map (it was copied into both the internal error policy and the tools route), drops IdempotencyService config that only `executeWithIdempotency` reads (the extraction lease uses atomicallyClaim/release, so no result is ever stored), hoists the duplicated predicates in purgeCreatedWorkspaceFile and archiveWorkspaceFileFolderIfEmpty so a lock and its write cannot diverge, and names UPLOAD_SESSION_LOCAL_PUT_MAX_BYTES rather than overloading the multipart part size as the local single-PUT ceiling. Adds coverage for the two guards nothing exercised: the re-validation of the segments `prepareRootFolder` actually returned, and the default cap applying with no caller opt-in. UI: the drop overlay used --surface-4 unconditionally, which renders grey over the light-mode canvas; matches the canonical overlay's --white/dark:--surface-4 and swaps arbitrary px type sizes for named tokens. * fix(files): bound the extraction write loop so it cannot outlive its lease Cursor Bugbot flagged two related holes, both rooted in the write loop being unbounded: 1. `maxDuration` is a Next.js route-segment config that serverless platforms enforce and self-hosted deployments do not. A slow extraction (up to 1000 sequential uploads) could therefore outrun the six-minute lease, and `IdempotencyService` reclaims an expired in-progress claim — so a second unzip of the same archive could start beside the first. 2. Nothing rolls back a process killed mid-pass-2, so a timeout stranded the destination folder and every file written so far. `decompressArchiveBufferToWorkspaceFiles` now takes an `AbortSignal` and checks it between entries in both passes, and the extraction use case supplies a 180s deadline. The abort unwinds through the existing all-or-nothing rollback, so the work stops on our terms with the tree cleaned up, well inside both the route's 300s budget and the 360s lease. That closes (1) outright — the holder can no longer outlive its lease on any platform — and converts (2) from a stranded partial tree into a clean rollback for the slow case that actually triggers it. A SIGKILL still cannot be caught; that needs a durable job and is out of scope here. The overrun surfaces as a caller-fixable 413 naming the archive rather than an opaque 500 from the raw DOMException. * fix(files): only remap the deadline abort itself, and stop overclaiming rollback Two follow-ups on the budget deadline, both reported by Cursor Bugbot: `deadline.aborted` stays true for the rest of the request once the timer fires, so it cannot decide whether *this* error was the abort. An `ArchiveError` or storage failure thrown mid-entry after the timer fired was being relabelled as a timeout and returned as a 413, hiding the real cause. The catch now matches the thrown value against `deadline.reason` — `throwIfAborted()` throws exactly that object, so the check is identity-exact and cannot capture an unrelated failure. The message also claimed a rollback that has not necessarily happened: the budget covers the archive download too, so it can fire before the first write, when there is nothing to roll back. It now says the unzip was cancelled and claims nothing about what was written. Including the download in the budget is deliberate — the lease it has to fit inside starts earlier still — so the TSDoc says that rather than "the extraction itself". --------- Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Waleed
|
c8f559ae77 |
fix(workflows,connectors): close pre-merge audit findings (#6783)
* fix(workflows,connectors): close pre-merge audit findings Recover subblock values orphaned by the id renames in this release, and stop truncated knowledge-base listings from reporting themselves complete. - Add operation-scoped subblock id migrations so a saved workflow's stored value survives a rename. Cloudflare create/update DNS record, ServiceNow read record, and Okta deactivate/delete previously lost their stored value: the create path substituted a seeded default (an A record where the user chose CNAME, and unproxied where they chose proxied), and the update path silently no-opped while reporting success. A migration is used rather than a legacy-id fallback so no subblock id carries two value spaces at runtime. - Webflow, Zendesk: a listing that stops for a reason the connector cannot rule out now reports as capped instead of exhausted. A malformed envelope, an unfollowable continuation link, or an absent collection list previously read as a complete listing and let deletion reconciliation hard-delete every document past the truncation point. - Sentry: pin the listing window in the request rather than inheriting the server default, so the range cannot silently narrow into hard deletes. - Fork sync: a parent re-pick no longer writes a blank over a hidden optional dependent's stored target value, and a required field stays on screen once it is filled. Add hook-level coverage for the submitted payload. - Fork file copy: a file whose name is already taken in a reused target folder is de-duplicated instead of dropped. - Delete an orphaned Shopify OAuth route that built a credential from unsigned cookies. It had no writer, no caller, and no inbound link. - Tailwind: drop two content globs that scanned 5.4k files to emit one unused rule, keeping the ones that fix brand tile icon color. - Correct the API route-count baseline, add an Evernote docs redirect, align library copy with the language rules, and fix a stale turbo filter. * fix(connectors,forking): trim the audit fixes to their minimum A legitimacy review found several changes closed no live defect, and two introduced problems of their own. - Zendesk: narrow the cursor fix to a signal change. Treating a missing meta envelope as truncation had also made the walk follow links.next and keep paginating, and the ticket cursor has no page-depth valve, so a source advertising a next page with no meta could loop without terminating. The page-fetch set now matches the previous behavior; only the flag is new. - Zendesk: drop the search next_page branch. The existing count check already caps every case where a missing key could lose documents. - Webflow: drop the empty-collections flag. The sync engine already blocks the first sync on an empty listing and reconciles only when a second sync agrees, which handles a transient fault better and still removes documents when a source is genuinely emptied. The flag short-circuited that and suppressed reconciliation permanently. Restore the previous loud failure on a non-array envelope, and drop the unreachable collection-id filter. - Webflow: soften a docstring that claimed pagination.total is always present. It is documented optional, so its absence proves nothing either way and treating it as unprovable truncation is the fail-safe reading. - Sentry: drop the pinned statsPeriod. Sentry's issue search floors every query at 90 days in the executor regardless of the request, and the endpoint this release moved away from hit the same floor, so there was no window to close. Keep the tests and the docstring recording that. - Fork copy: drop the renamed counter, which no caller reads. - Repair check-block-registry, which stopped exempting migrated subblock ids when the migration map became an array — `in` was testing array indices. - Drop mdx from a Tailwind content glob that emits nothing, and loosen an exact compiled-SQL assertion to the invariant it was pinning. * fix(migrations): keep a ServiceNow write body off the read projection Review findings from the first round. - A legacy ServiceNow block can hold a Create/Update Record JSON body under `fields` while its stored operation is Read Records: the id served both value spaces before the rename, and a subblock value is not cleared when the operation changes. The scoped migration moved that body onto `readFields`, where it would reach the wire as sysparm_fields. Migration entries can now carry a `whenValue` predicate for the case where the stored operation alone cannot separate two value spaces, and the ServiceNow entry uses it to move only a plausible comma-separated projection. - Type the fork copy test harness instead of using `any`, without weakening it: every predicate shape it does not model still throws rather than matching. - Correct the dependent-omission comments. Omitting a parent-invalidated field preserves the target's stored value on Save and across an undo, where the parent nets out unchanged; on a Sync the written state is source-derived, so what it prevents there is an explicit blank reaching the fields the remap's clearing pass does not cover, nested tool params in particular. Okta's migration scope is left as-is: `okta_remove_user_from_app` and the sendEmail split shipped in the same release, so no saved block can hold legacy state for it, and widening the scope would promote an activation-era value onto the deactivation switch. Tests document the boundary. * chore(forking): move the fork-sync changes to their own PR The dependent-omission fix and the fork file-copy de-duplication are reviewed separately in #6787. They are the only changes here that overlap #6776, and they carry their own design tradeoff, so they should not ride along with the unrelated audit fixes in this PR. * fix(migrations): separate a ServiceNow write body from a projection by parsing The guard tested for a `{` or `[` prefix, so a stored scalar body — `true`, `"short_description"`, `42` — read as a field list and was promoted onto `readFields`, where it would go out as sysparm_fields. A Create/Update Record body is JSON and a projection is a bare comma-separated field list, which is never valid JSON, so parsing is the whole test rather than a guess at its opening character. Ambiguity still resolves to "not a projection", leaving the value where the Create/Update control owns it. * test(connectors,credentials): tie two assertions to what they actually prove - Webflow: a non-array collections envelope reaching `for...of` throws, which is the intended loud failure. Assert the spec-mandated TypeError plus a single request and no write-back, rather than matching V8's wording. - Credentials: the second guard test cannot observe "not deleted" — the proxy driver replays canned rows — so name it for what it does verify, that the reference check carries no workspace predicate and an empty RETURNING logs nothing. Making the driver decide the outcome would fake the database. - Drop `vi.importActual`; a plain `drizzle-orm/pg-proxy` import works now that `drizzle-orm` is un-mocked. * fix(migrations): identify a ServiceNow projection by its own shape Recognising a write body was the wrong way round. A saved body is not always well-formed: it can be a half-typed draft or carry an unquoted block reference, so neither "opens with a brace" nor "fails to parse as JSON" identifies one — and a body misread as a projection is moved to readFields with its original key dropped, losing the draft. Match the projection instead: a comma-separated list of ServiceNow field names, which are word characters plus the dot of a dotted walk. A brace, quote, colon, angle bracket or interior space fails that shape. Parsing then removes the bare scalars that satisfy it by accident. |
||
|
Waleed
|
ae2147645c |
fix(cli): resolve findings from a full command-surface audit (#6788)
* fix(cli): resolve findings from a full command-surface audit Exercised all 147 commands against a live deployment. Fixes the defects that surfaced, plus the docs and generator drift they exposed. Transport - Stop following redirects. A bare domain that 301s to www silently converted POST to GET and dropped the body, so reads worked while every write failed with a misleading validation error and login returned 405. Both the client and the device flow now explain the redirect and name the endpoint to configure, rather than carrying credentials off-origin. - Report a non-JSON response as one instead of printing the HTML page. - Name the personal-API-key remedy on a workspace-key refusal, reading the machine-readable code the API actually sends. - Drop union-branch noise from validation errors that contradicted itself. - Show paging progress on stderr for multi-page fetches. Output - Clamp record values for table only. text is the format built for pipes, and it was truncating signed URLs and tool source mid-value. - Infer timestamp, duration, bytes and boolean formatting for API-owned keys so undeclared commands stop printing raw ISO and float ms. Skips user-defined table cells and leaves json/yaml on the raw payload. - Render a declared-but-absent field as an em dash; billing credits were vanishing silently. Paths, naming and validation - Percent-encode folder paths per segment and decode them for display, so a folder reads and types as the name shown in the app. - Reject a malformed endpoint where it is set and where it resolves, instead of crashing with a URL parse trace. - Request the detail level logs list's own columns need; its workflow column could never populate. - Rename three commands that described themselves wrongly and align two flags with their siblings. Old spellings still work: hidden, warned on stderr, and kept out of help and docs. - Verify whoami against the API, separating a bad key from an unreachable endpoint, and report the workspace by name. - Correct the --yes help text, which advertised skipping a prompt that does not exist. Docs - Teach the docs generator that a flag required by the runtime is required, and that hidden commands are not documented. * fix(cli): clear the paging progress line when a page fails Progress is written without a trailing newline so it can be overwritten in place, and both paging loops cleaned it up only on success. A page that threw part-way through left `fetched 1200…` on the line the error was then printed onto, so the two ran together. * fix(cli): name a working API root when an endpoint redirects The suggested endpoint was the redirect target's origin, which drops a path prefix. A self-hosted deployment reached at https://host/sim was told to set https://www.host — not an API root, so following the advice replaced one broken endpoint with another. Derive it by stripping the request's own path from the target instead, so a prefix survives, and say nothing about --set-endpoint when the target resolves to the endpoint already configured: a trailing-slash or path normalization redirect keeps the origin, and naming the value the caller already has explains nothing. The login poll shared both faults and now shares the helper. |
||
|
Waleed
|
025ea4d2bd |
fix(docs): serve JSON-LD in the HTML, fix sidebar spacing, and tighten the CLI guides (#6763)
* docs(cli): use -g for the install, and cut the prose that was not pulling weight
`--global` is valid but `-g` is what every comparable CLI documents, and the
long form only came from the package README. Also drops the yarn tab: it read
`yarn global add sim`, which works on Yarn 1 only — Yarn 2 removed global
installs, so that command fails for anyone on a modern Yarn. Adds `npx sim` for
running without installing.
The guides had accumulated design rationale that belongs in code comments rather
than user docs — why the filter grammar is JSON, why the config section naming
is asymmetric, why an unexpected error keeps its stack trace. Surveying how gh,
Vercel, Turborepo, Deno, Bun and Supabase write theirs, none carry that kind of
justification, and callouts are reserved for content whose absence produces a
wrong result rather than for general asides.
So: 1016 lines to 763, and 12 callouts to 3. The three that remain are the
pairing-code check, that `sim logout` does not revoke the key, and the
`--limit 100` default on `batch-delete`/`batch-update`, which silently truncates
a larger match. Troubleshooting drops the entries whose error message already
contained its own fix and keeps the seven whose cause is not obvious.
* fix(docs): render JSON-LD as native script tags so it reaches the HTML
All four structured-data blocks — WebSite, TechArticle, BreadcrumbList,
SoftwareApplication — were rendered with `next/script`, which never emitted a
script tag. Measured on a production build, `/api-reference/getting-started`
contained zero `<script type="application/ld+json">` elements; the payload
existed only in the `__next_s` client-injection queue and the RSC flight data,
so anything reading the served HTML saw no structured data at all. React was
also logging "Encountered a script tag while rendering React component" on every
page.
`next/script` is for loading and executing JavaScript. JSON-LD is data, and
Next's own guidance is a native `<script>` in the component. `serializeJsonLd`
already escapes the `<` character to its unicode form, which is the
sanitization that guidance calls for, so only the element changes.
Same build, after: three valid tags per page with `WebSite` in `<head>`, and the
injection queue gone entirely.
* fix(docs): scope the flush-separator rule to a container's first separator
`[data-separator]:not([data-separator] ~ [data-separator])` was meant to keep the
first sidebar group flush against the top padding, but `~` only reaches siblings,
so it also matched the first separator inside every expanded folder. Under
Self-Hosting, "Install" lost its top margin and crowded the "Architecture" link
above it — 25px of gap where "Configure" and "Operate" below it had 40px.
`:first-child` expresses the intent directly. Only the four sidebar roots open
with a separator; every nested folder starts with a page, so the intended case
still goes flush and nothing else changes.
* fix(docs): move the flush-separator rule onto the separator component
Keeps the styling with the component that owns it, per the repo standard, and
lets the global rule be deleted outright rather than corrected — `global.css`
now only loses a rule in this PR. Tailwind's `first:` variant compiles to the
same `:first-child` selector, so behavior is unchanged: the build emits
`.first\:mt-0:first-child{margin-top:0}` and the prerendered HTML carries the
class on the separator.
|
||
|
Waleed
|
fed891f69d |
docs(cli): add a CLI docs section generated from the command tree (#6762)
* docs(cli): add a CLI section, generated from the command tree
The `sim` CLI shipped with no coverage in the docs site. Adds a fourth
top-level tab for it, and moves Academy last.
The command reference is generated. `sim` exposes 147 leaf commands across
33 groups, most of them derived at runtime from the v2 route contracts, so a
hand-written reference would be wrong the week after it was written. The
generator walks the command tree `buildProgram()` hands to commander — the
same tree the terminal parses — rather than re-deriving it from the contract,
which would be a second implementation free to describe commands nobody can
invoke. `check:cli-docs` is a zero-arg `check:*` script, so the existing audit
runner picks it up and stale pages fail CI.
Generating against the real tree surfaced a collision it had been hiding:
`bulkUpdateKnowledgeDocuments` and `updateKnowledgeDocument` both derived to
`sim knowledge documents update`. Commander resolves a duplicate to the first
match, so the bulk form shadowed the single-document one and its flags were
unreachable while still appearing in `--help`. The bulk form is now
`batch-update`, matching how `tables rows batch-delete`/`batch-update` already
handle the same REST overload, and the generator fails on any duplicate path
so the next one cannot land silently.
Five hand-written guides cover install, auth, configuration, output formats,
and scripting. Also corrects two commands in the package README that do not
exist as documented (`tables columns <tableId>`, and `--sort score:desc`,
which is JSON).
* docs(cli): document every flag from the contracts, add troubleshooting and a single-page reference
The command reference was structurally complete but said almost nothing: 223 of
377 flags rendered as "Set sort by" because the CLI only ever read flag help
from its own contract overrides, and fell back to restating the flag name.
The prose already existed. The v2 route contracts carry 931 `.describe()` calls
and the OpenAPI specs publish all of them — 327 parameters and 282 body
properties, 100% coverage — but the generated operation table dropped every one,
carrying only a per-operation summary. It now carries the field descriptions,
the path-parameter descriptions, and positional help, so `--help` and the docs
explain a flag the same way the API reference does. Placeholder descriptions are
now zero, and 147/147 commands, 377/377 flags and 130/130 arguments are
documented.
`check:cli-docs` fails on a request field with no `.describe()` rather than
letting it render as documentation that says nothing.
Also in this pass:
- Commands are root-level sidebar entries under a Commands heading rather than
a folder, and headings are the command's description, so the table of
contents distinguishes entries at the first word instead of repeating
"sim knowledge documents …" fourteen times. A guard fails the build if two
descriptions on a page collide, since they would share an anchor.
- A single-page `Complete reference` carrying all 147 commands, for in-page
search and for agents fetching `/cli/reference.mdx`. It keys on exact command
paths because descriptions are only unique within a group.
- A troubleshooting page, with every message copied from the source.
- Table columns are sized by a local component; the flag column was starved
while descriptions kept most of the row empty.
- The prerelease install channels are dropped from the docs and the package
README, which is what npm renders.
* fix(docs): match the CLI tab by path segment, and escape backslashes before pipes
`pathname.includes('/cli')` also matches `/integrations/clickup` and
`/integrations/clickhouse`, so both existing integration pages lit the CLI tab
and unlit Documentation. Matching is now per path segment. Anchoring to the
start would not work either — a non-default locale prefixes the path, as in
`/ja/cli` — so the segment is matched wherever it sits.
Table cells now double a backslash before escaping pipes. A value ending in one
turned `a\` + `|` into `a\\|`, which the table parser reads as an escaped
backslash followed by an unescaped pipe, splitting the cell early. Nothing in
the command surface contains a backslash today, so this was latent rather than
visible.
The reference page's global options table is two-column and was being wrapped in
`CommandTable`, which sizes the second column for the `Required` cell of the
three-column tables and crushed the description into 5.5rem. It now matches the
overview page, which leaves that table unsized.
|
||
|
Waleed
|
6a29a9e2f4 |
feat(mssql): add Microsoft SQL Server integration (#6739)
* feat(mssql): add Microsoft SQL Server integration
Add a Microsoft SQL Server block backed by six tools (query, execute,
insert, update, delete, introspect), mirroring the existing PostgreSQL
and MySQL integrations.
Connections go through the `mssql` (Tedious) driver: `connectionTimeout`
is top-level while `encrypt`, `trustServerCertificate`, and
`instanceName` live under `options`, and `port` is omitted when a named
instance is used. Values are bound as `@paramN` via `request.input()`;
no user value is interpolated into SQL. Identifiers are bracket-quoted
after validation and WHERE clauses run through the shared injection
guard.
Introspection reads INFORMATION_SCHEMA plus the `sys.indexes` catalog
views for tables, columns, primary keys, foreign keys, and indexes.
The icon is a placeholder database cylinder drawn with `currentColor`
until the real brand mark lands.
Requires `bun install` for the new `mssql` / `@types/mssql` deps.
* feat(mssql): use the SQL Server brand mark on a white tile
* fix(mssql): pin the validated IP and correct the introspection catalog reads
Tedious exposes `options.connector`, a hook that replaces its own
resolve-and-connect path, so the connection can be pinned to the address
`validateDatabaseHost` already approved instead of re-resolving the
hostname. `server` stays the hostname because tedious derives the TLS
`servername` from it independently of the connector, so SNI and
certificate validation survive the pin. This brings MSSQL in line with
the PostgreSQL and MySQL tools.
Named instances are dropped: tedious resolves them with a UDP SQL Server
Browser lookup issued outside the connector, and node-mssql deletes
`port` whenever `instanceName` is set, so no configuration leaves a
named instance pinned. A named instance is reachable through its static
TCP port.
Introspection fixes:
- index key columns now filter on `key_ordinal > 0`; INCLUDEd columns
and partitioning columns both report `0` and were being returned as
key columns, ordered ahead of the real ones
- foreign keys resolve through `sys.foreign_keys` /
`sys.foreign_key_columns` rather than
`INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS`, whose join to
`TABLE_CONSTRAINTS` has no row when a key references a unique index
and so dropped the key entirely
- `is_unique` is a `bit`, which tedious maps to a boolean, so it is
coerced rather than compared
- schemas come from `sys.schemas`, which needs only `public` and carries
no metadata-visibility caveat
The WHERE-clause guard also covers `WAITFOR TIME`, `OPENQUERY`,
`OPENXML`, the legacy `master..sys*` compatibility views, and extended
and OLE-automation procedures beyond `xp_cmdshell`.
Regenerates the docs and catalog artifacts the icon change left stale.
* chore(mssql): commit the lockfile entries for mssql and its tedious dependency tree
* fix(mssql): make the Query operation genuinely read-only
The block label, tool description, and docs all present Query as SELECT-only
while the route ran whatever T-SQL it was given, so an agent picking
mssql_query because "it is only a SELECT" could delete rows. Screen the
statement for mutating keywords with string literals stripped, which also
catches the WITH ... DELETE form that a leading-token check would miss.
Also switch the tool barrel to absolute imports per the repo convention.
* fix(mssql): compose the shared WHERE guard and close the semicolon-less batch gap
The local validateWhereClause re-derived an older copy of the shared patterns
and scanned raw text, so it missed a bare 1=1 and false-positived on prose in a
quoted value. Delegate to validateSqlWhereClause, which masks string literals
first, and keep only the SQL Server surfaces it has no reason to know about.
T-SQL needs no statement terminator, so every semicolon-anchored stacked-query
check reads straight past `id = 1 DROP TABLE dbo.users`. Screen for a bare
statement-introducing keyword to close that; word boundaries leave ordinary
column names like updated_at and deleted_at untouched.
Export maskSqlStringLiterals so the dialect layer masks the same way the shared
guard does rather than carrying a weaker single-quote-only copy.
* fix(mssql): screen administrative T-SQL and reject batches in the read-only path
The previous round left two keyword lists maintained separately, and both were
short: DBCC, KILL, CHECKPOINT, USE, and DENY were in neither, so
`SELECT 1; DBCC SHRINKDATABASE(...)` and `id = 1 DBCC SHRINKDATABASE(...)`
both got through. Collapse them into one MSSQL_STATEMENT_KEYWORDS shared by the
query and WHERE screens so a keyword cannot be covered in one place and missed
in the other, and add the administrative commands.
Also reject any second statement after a semicolon in the Query path outright.
That closes SELECT 1; <anything> structurally instead of by naming the anything,
so the blacklist no longer has to be exhaustive to hold.
* fix(mssql): reject SQL comments in the read-only query path
A block comment placed inside a keyword splits it as far as a lexical scan is
concerned, so keyword coverage cannot settle whether the server rejoins the
halves. Refuse comments in the Query path instead of modelling the tokenizer.
A SELECT sent through this operation has no need for one, and Execute Raw SQL
still accepts them. Masking leaves comment markers intact, so a literal
containing -- still passes.
* fix(mssql): close the masker-desync bypasses and correct the catalog reads
Every T-SQL screen runs over the shared literal masker, which was written for
the MySQL dialect. Three ways to desynchronise it let real SQL hide inside what
the masker believes is a string, two of which survived the existing even-quote
check:
- a backslash before a quote. T-SQL has no backslash escape, so the server
closes the literal where the masker swallowed the quote and runs the rest as
code. `a='x\' DELETE FROM dbo.t WHERE b='y'` holds four quotes and masks the
DELETE out of the keyword screen entirely, so the read-only Query operation
would run it.
- a double quote inside a bracketed identifier, which the bracket rule missed
because it only looked for single quotes.
- any unbalanced double quote or backtick, which the parity check did not cover.
All three now fail closed. Introspection also filters hypothetical and disabled
indexes, which were reported as if they were live, and resolves the referenced
side of a foreign key through sys.schemas so a cross-schema reference is no
longer an ambiguous bare table name. Values bound through request.input are
serialized when they are nested JSON, which the driver otherwise rejects with a
bare "Invalid string.".
* test(mssql): cover the block param merge and the operation-to-tool map
Asserts on the merged `{ ...inputs, ...buildParams(inputs) }` the generic tool
handler forwards rather than the mapper's return, since a key the mapper omits
keeps its raw subBlock value through that merge. Pins the TLS toggles to their
string form end to end — a switch subBlock would serialize `'false'`, which is
truthy, and the route contract would coerce the user's off into on — and checks
that duplicate subBlock ids agree on their seeded default.
* fix(mssql): release a pool whose connect failed, and allow a keyword with no trailing space
Only a pool handed back to the route reaches its `finally`, so a pool whose
connect rejected leaked its tarn resources — one per attempt when a bad
credential is retried. It now closes itself, and a failure to close cannot mask
the connect error the caller needs.
The read-only screen also anchored on `\s` after the opening keyword, which
refused valid reads like `SELECT*FROM dbo.users` and `SELECT(1)`. A word
boundary accepts those while still refusing `SELECTX`, and cannot loosen the
screen — the keyword and batch checks run over the whole statement regardless.
* chore(mssql): regenerate tool metadata and the integration catalog after rebase
Artifacts rebuilt with the generators rather than hand-merged, so they carry
both the mssql entries and the tools that landed on staging in parallel.
* fix(mssql): reject a parenthesised or negated constant tautology in a WHERE clause
The shared guard recognises `OR 1` but not `OR (1)`, `OR ((1))`, `OR NOT 0`, or
`OR NOT (FALSE)` — a parenthesis or a NOT between the operator and the constant
hides it. Both patterns require the constant to be the whole parenthesised term,
so a real disjunct such as `OR (1 = priority)` is untouched.
This narrows the gap rather than closing it, and is not meant to close it: an
always-true expression is not lexically decidable in general, which is why the
WHERE screen stays documented as defense-in-depth rather than a boundary.
* chore(mssql): regenerate tool metadata after rebase onto staging
Rebuilt with the generators so the artifacts carry the servicenow and
crowdstrike tools that landed on staging alongside the mssql entries.
* fix(mssql): screen trigger and state statements, and drop the space anchor on Execute
DISABLE and ENABLE were missing from the shared statement list, so
`SELECT 1 DISABLE TRIGGER dbo.audit ON dbo.users` passed the read-only screen as
a semicolon-less batch and turned auditing off. SET, BEGIN, COMMIT, and ROLLBACK
are added with them, since session and transaction state are reachable the same
way. FETCH is deliberately left out: OFFSET ... FETCH NEXT is the standard paging
clause, and screening it would reject the ordinary paged SELECT.
Execute Raw SQL anchored its allowlist on `\s`, which refused `EXEC(@sql)` —
the ordinary form of dynamic SQL, on the one operation meant to run it. It now
uses `\b`, matching the read-only screen.
|
||
|
Justin Blumencranz
|
7f936dc02a |
feat(tooling): enforce docs freshness and modernize agent skills (#6756)
* feat(docs): fail CI when generated integration docs are stale * fix(docs): don't flag delete-then-recreated trigger pages in check mode * docs(skills): require docs:check in the integration authoring skills * chore(skills): migrate agent commands to native skills * fix(skills): clean orphaned Claude projections |
||
|
Waleed
|
9e67655b23 |
feat(servicenow): semantic incident, change, catalog, approval, CMDB, and knowledge tools (#6747)
* feat(servicenow): add semantic incident, change, catalog, approval, CMDB, knowledge, and directory tools
The ServiceNow block only exposed generic Table API CRUD, so every real task
started with "which table is that on?". This adds 27 semantic tools that wrap
the same Table API plumbing under the names customers actually use.
- Incidents: create, get by number or sys_id, search, update, resolve, close,
and append a work note or customer-visible comment.
- Change: create, get, list, update, move state, and list change tasks through
the documented Change Management API.
- Service catalog: browse items, order one via the Service Catalog API
order_now endpoint, and list or get requested items.
- Approvals: list pending approvals for an approver, approve, and reject.
- CMDB: search CIs on any class, read a CI with its inbound and outbound
relations through the CMDB Instance API, and list cmdb_rel_ci rows.
- Knowledge: search and read articles through the Knowledge Management API.
- Directory: find a user by email or user name and list group members, which
is what fills assigned_to and assignment_group.
Reference fields are the usual source of confusion, so every semantic read
defaults to sysparm_display_value=all — a reference comes back as both its
sys_id and its label — and every semantic write exposes
sysparm_input_display_value so a display name can be written instead of a
sys_id. Coded state values are exposed as labelled dropdowns built from one
constants module rather than raw integers.
The shared instance-URL, Basic Auth, sysparm, envelope, and error handling now
live in tools/servicenow/utils.ts, and the existing eight generic tools were
moved onto it rather than keeping their own copies.
* fix(servicenow): stop per-operation subblock defaults colliding on a shared id
Subblock initial values are seeded into block state keyed by subblock id, so
two subblocks sharing an id leave one stored value and the last definition
wins. Three ids were duplicated with differing defaults:
- `displayValue` was defined twice, unset for the generic Table API tools and
`all` for the semantic ones. The semantic definition won, so a new block set
to Read Records or Aggregate Records sent `sysparm_display_value=all` — a
wire change to two already-shipped tools.
- `state` was defined four times. The Approval State definition won, so every
new block carried `state=requested`, which Create Incident wrote to the
incident and Move Change State used instead of its own `-5` default.
Give the colliding controls their own ids and map them back to the tool params
per operation, so the generic tools keep their original request shape and each
semantic operation keeps its own default.
Also correct descriptions that overstated what the API does: the LIKE operator
is not documented as case-sensitive, List Requested Items has no requester
filter, and the Change Management API task shape differs from the Table API.
Adds tool tests covering the refactor invariants for the eight pre-existing
Table API tools and the display-value separation.
* feat(servicenow): read a change request's real next states from the instance
The change tools describe state transitions using the base-system codes, which
only hold on an instance that has not customized its change model. ServiceNow
publishes an endpoint that answers the question directly for the record in
hand, so use it rather than keep assuming.
GET /api/sn_chg_rest/change/{sys_id}/nextstates returns the states reachable
from the change request, the instance's own state-value-to-label map, and, for
model-driven changes, each transition with the conditions it has and has not
met. The tool flattens the per-target-state grouping ServiceNow returns (each
transition already carries from_state and to_state, so nothing is lost) and
derives the states whose conditions currently pass.
Also record the sourcing for the coded values in constants.ts: the change
states and close codes are published as a table, but the incident state codes
are not — only 6 (Resolved) appears in the docs — so mark the rest as defaults
rather than guarantees. Note that sysparm_input_display_value also reinterprets
date and time values in the caller's timezone instead of GMT, which matters for
the change start and end dates.
* docs(servicenow): stop asserting undocumented coded values in placeholders
The additional-fields examples used hold_reason with a coded value of "1".
ServiceNow documents the On hold reason choices by label only — Awaiting
Caller, Awaiting Change, Awaiting Problem, Awaiting Vendor — and publishes
neither the column name nor the codes, so the example was asserting something
unsourced. Use a field whose value is caller-supplied instead, and record the
On Hold requirement on the incident state control using the labels the docs
actually give, including that Awaiting Caller makes Additional Comments
mandatory.
* fix(servicenow): drop phantom parent fields from the catalog order output
order_catalog_item read parent_id and parent_table off the order_now response.
Those fields belong to submit_producer, a different Service Catalog endpoint;
the documented order_now result is sys_id, number, request_number, request_id,
and table. Both outputs were therefore always null.
* fix(servicenow): correct what knowledge search returns as an article id
Search results carry a table-prefixed identifier — "kb_knowledge:9e528db1..."
— not a bare sys_id, while GET /knowledge/articles/{id} accepts only a bare
sys_id or a KB number. The output described it as a sys_id and the tool
description told callers it was what they needed to fetch the article, so
chaining the two tools on that field would fail. Point callers at the KB
number instead. Relevancy score is documented as a number, not a string.
* docs(servicenow): cite the page that actually documents approval statuses
The approval state constants pointed at the classic-approvals landing page,
which does not list the statuses. Approval status is documented separately and
names four — Requested, Approved, Rejected, and Not Requested.
* fix(servicenow): stop constant interpolation leaking into tool descriptions
The docs generator and the client-facing integration catalog read tool
descriptions from source rather than from the evaluated module, so a
template literal like `state ${INCIDENT_STATE.RESOLVED}` shipped to users
verbatim: `apps/sim/lib/integrations/integrations.json` and the published
ServiceNow integration page both rendered `${INCIDENT_STATE.RESOLVED}`
instead of `6`. Inline the base-system coded values in the description
text; the constants stay in use everywhere behavior depends on them.
Also drops an escaped `\'` in the `inputDisplayValue` description for the
same reason, and adds a standing guard test asserting no subBlock id
carries two different seeded defaults — the invariant behind the
per-operation defaulting bug, now checked structurally rather than only
through the four per-operation cases.
* refactor(servicenow): type the shared response boundary instead of any
`parseServiceNowResponse` returned `any`, so every tool reading `data.result`
did unchecked property access — a shape change on the instance side would have
produced a wrong-typed output silently rather than a type error.
Introduces `ServiceNowEnvelope` (`result?: unknown`) as the parser's return
type and narrows the record index signatures from `any` to `unknown`. Adds
`toRecordObject`, `readString`, and `readNestedNumber` so the tools that read
individual fields narrow deliberately at the point of use.
This surfaced five genuinely unchecked reads: Order Catalog Item, Get Knowledge
Article, and Search Knowledge were declaring `string | null` / `number | null`
outputs while emitting whatever the instance sent, and Get Change Next States
assigned an unvalidated object to `Record<string, string>`. Each now coerces or
drops a non-matching value rather than passing it through.
* fix(servicenow): publish the shared tool params and stop offering inert controls
The docs generator reads tool source rather than importing it, so the shared
`params.ts` consts the semantic tools spread were dropped from every published
Input table — 27 of 35 ServiceNow tools listed no instance URL, username, or
password at all. Follow a spread into the module it is imported from so those
rows are published; ten other integrations gain the rows they were missing for
the same reason.
Two controls were dead on arrival: Additional Fields was offered on Move Change
State and Add Incident Comment, and neither tool read it. Wire it through the
change transition, which needs it, and drop it from the comment tool, whose body
is exactly one journal field.
Every coded-value control was a select-only dropdown, so a customized instance's
state or close code was unreachable — sharpest on Move Change State, whose
target state is required and whose real codes come from Get Change Next States.
Make them comboboxes.
Also correct two doc claims ServiceNow does not publish (the incident state
citation pointed at a page that does not exist and compares the legacy
incident_state field; closing an incident is not documented as requiring
itil_admin), replace Record<string, any> with checked narrowing that surfaced
two unsound widenings, and document that List Change Tasks returns a fixed
{value, display_value} shape under `tasks` rather than `records`.
* fix(servicenow): stop one subblock id from carrying two value spaces
Subblock values are stored per block keyed by id, so an id reused across
operations keeps its value when the operation changes. Incident and change
shared `state`, and `closeCode`, `closeNotes`, `comments`, and the knowledge
search phrase were each reused for a different value space — so an incident
state could be written onto a change request, an incident close code sent as a
change close code, or an encoded query searched as knowledge text.
Give each value space its own subblock and republish it to the tool param from
the operation that owns it, the way targetState and approvalState already work.
The generic Table API ids stay exactly as they are, since renaming one would
orphan the stored value of every workflow already using those shipped tools.
The previous guard only compared seeded defaults, which is why this class stayed
hidden; the new one asserts against the merged params a tool actually receives.
* fix(servicenow): point the canvas sentences at the renamed subblocks
The split of the colliding subblock ids left the operation sentences anchored on
ids that no longer exist, so those clauses would silently drop from the card.
* fix(servicenow): validate collection members and split the fields projection
toRecordArray cast every member of a successful response, so a null or scalar in
a collection was handed to the next block as a record while the tool reported
success and its declared output said that could not happen. Members that are not
plain objects are now dropped, and knowledge articles and change transitions get
the same narrowing. The two response types that described an unverified inner
shape now say what is actually checked.
The 'fields' subblock also carried two value spaces: a JSON body on Create and
Update Record, a comma-separated projection everywhere else. Operations added
since read a separate returnFields control, so a body can no longer arrive as a
projection or the reverse. The shipped ids are untouched, since renaming one
orphans the stored value of every workflow already using those tools.
|
||
|
Waleed
|
0cd87bad2f |
feat(resources): multiselect on tables and knowledge, spring-loaded folders (#6721)
* feat(resources): multiselect on tables and knowledge, spring-loaded folders Tables and Knowledge lists get the checkbox multiselect Files already had — selection, shift-click ranges, select-all, and a shared bulk action bar for move and delete. Dragging a resource onto a folder row and resting there now opens that folder, so nested filing is one gesture (macOS Finder spring-loading). Works on Files, Tables, and Knowledge. Selection, the action bar, the drag payload, the drag ghost, and drag teardown are extracted to shared modules; Files migrates onto them rather than keeping its own copies. Bulk move and delete land as single authorized operations that take folders and resources together, so a mixed selection commits once instead of fanning out. Fixes two latent UI bugs: the drop-target outline referenced --accent, an HSL-channel token only valid via hsl(), so it silently rendered as currentColor; and rows painted hover and selected with the same surface token, making the two states indistinguishable. * chore(audits): re-record route ratchet after merging staging * fix(bulk): reject a move target inside the moving subtree and report contained folders deterministically * improvement(resources): neutral drop affordance, longer spring delay, and a body drop target * fix(resources): drop into the open folder on Files, return on an unused spring-open, and guard bulk caps * fix(drag): end a drag on pointer resume instead of an idle timer * feat(resources): drag onto breadcrumbs to move back up, and round the drop ring * fix(resources): tint the list region on drop instead of ringing it * refactor(folders): share the spring-navigation lifecycle so Files returns too * fix(breadcrumbs): accept a drag on the open-folder crumb too * fix(files): keep the view in a spring-opened folder when an OS upload lands there |