24 Commits

Include renames

Author	SHA1	Message	Date
Waleed	479ae8c138	fix(cli): seven defects found by live-testing the CLI against staging (#7176) ... * fix(catalog): resolve an unversioned tool id against the visible set `tools get github_comment` answered NOT_FOUND while `github_comment_v2` worked, though the toolId help promises an unversioned name resolves to the newest version. A superseded tool stays in the registry, so `resolveToolId` short-circuits on the exact hit and returns it unchanged; the visibility gate then refuses it because no visible block exposes a v1 tool. 204 base names were unresolvable this way. Resolution now walks the visible set newest-first, the way blocks already do. `resolveToolId` is untouched — execution depends on an exact id returning that exact id, and none of the 5182 visible ids change under the new path. * fix(files): resolve an archived folder path through its active ancestors The archived folder listing built its path map from the archived rows alone, so a folder whose parent is still active came back as its own name. Deleting `a/sub` and restoring `a/sub` therefore disagreed — restore only matched the truncated `sub` — and the path and parentPath fields were wrong. The extra read is taken only for the archived scope; active and all keep their single query, which a test now pins. Restoring by path also stopped guessing. Archiving, recreating and archiving again leaves two archived folders with the same canonical path, and the resolver took the first match, silently restoring the wrong one. It now refuses and names the folder-id form. * fix(v2): answer a folder-list miss with an empty page A parentPath naming no folder returned 404 on the workflow, table and knowledge folder lists, and an empty page on files. The rule the codebase already publishes is the empty page: V2_FOLDER_FILTER_MISS is appended to the folderPath filter on six list surfaces, and resolveFolderPathFilter documents why a list must not become an existence oracle — a 404 claims the collection is missing and breaks a walk when a folder is deleted mid-pagination. Both TSDocs asserted the sibling folder lists already behaved that way. They did not; that premise is corrected here too. Mutations keep every 404. The miss short-circuits before the row query, because an unfiltered parent id lists the whole workspace. * fix(cli): gate activating a deployed version `workflows activate create` switched which version production serves with no confirmation, while `rollback` refused without --yes. They are the same application operation under two transitions, so gating one and not the other was an accident of naming. The destructive-operation classification in the client tests listed activate as non-destructive, which is what kept its sweep from noticing. Moved, so two independent tests now hold the gate. * fix(cli): name the profile in the suggestion configure prints Refusing a root global printed a command to save it — without --profile, so following it verbatim wrote the default profile and left the named one untouched. The neighbouring suggestions in this file already carry the flag. Resolution matches resolveProfile, so SIM_PROFILE is covered too, and the profile name is redacted like the value beside it. * fix(cli): fail a row delete that matched nothing `tables rows batch-delete` exited 0 when none of the named rows existed, while the table equivalent exited 1 on the same shape. Only the id-list selection is checked: a filter answers without a requested count, so the guard self-excludes and an idempotent sweep still exits 0 on its second run. * fix(cli): show the -- escape for an id that opens with a dash Short ids draw from a 64-character alphabet containing one dash, so 1 in 64 open with one and commander reads it as an unknown option. It reaches `audit-logs get` and the custom-tool commands, and the escape was documented nowhere. The hint is appended only for a lone dash followed by two or more characters carrying an uppercase letter or digit — a shape no flag on this surface has — so a misspelt flag keeps commander's own suggestion. * chore: regenerate the API reference and CLI surface * fix(cli): quote a profile name a pasted command would otherwise split The suggestion configure prints is meant to be pasted, and it interpolated the profile name bare. Profile-name validation is creation-only by design — the validator says so, because a hand-written `[profile my stack]` has to keep resolving — so a name carrying whitespace, or a `;` that would end the pasted command and start another, reaches this message unchecked. Names that already satisfy the creation rule stay bare; the rest are single quoted, embedded quotes included. Redaction runs first, so a control character becomes a space and is then quoted rather than splitting the command.	2026-08-27 15:14:48 -07:00
Waleed	6bbabbbac6	fix(cli): close follow-up gaps (#7137) ... * chore: run the orphaned migration-safety test, and stop publishing real-looking ids One of the eight script tests was reachable from no entrypoint, so it had never run in CI — it passes, it was simply never invoked. This is the second time that hand-maintained list has drifted from the files beside it; the audit runner's own header records the first. The guard against a third is a `check:*` script rather than a test, because the runner derives its list from that namespace and so picks the guard up by name — a test would have had to be hand-added to the very list it guards. A published spec sat outside the generator's manifest and so outside its drift check, and carried six example ids with the texture of real generated ones rather than the pandigital placeholders the rest of the repo uses. The new check globs the directory instead of reading the manifest, since the manifest is what omitted the file. The one pre-existing borderline id is allowlisted with a reason: loosening the threshold to admit it would have hidden one of the six. * fix(cli): close the gaps black-box testing the shipped CLI found `sim profiles <anything>` still exited 0, so a probe reading the exit code to ask whether a command exists was told yes — the one group the earlier guard missed. The exemption was written for commands that are both a group and a leaf, but only `files restore` takes an operand; `profiles` takes none. Registering its listing as a default subcommand puts it back among the pure dispatchers the existing guard already covers, so the guard itself did not need widening. Three commands refuse a workspace API key and said nothing, while their menu siblings said so — reading as though they accept one. They are hand-written, so they never reached the code that appends the note. That note now comes from a helper taking the operation, so a command names the operation it invokes and the two cannot disagree, and a test fails if a hand-written command ever calls a restricted operation without it. A blank numeric value in a request body still became a real zero, the same coercion already fixed for query strings: the guard keyed off the slot when the distinction is the field's declared type. Twenty-one fields across fifteen operations were affected. An empty body string still clears a description. Blank values for the root endpoint, workspace and profile flags fell back to what was configured instead of being refused, and a whitespace workspace was accepted verbatim. A hand-written profile name carrying padding listed as reachable but resolved to defaults rather than erroring. Two schema descriptions named request fields that no flag spells, and a rejected value was echoed unredacted by four messages while their siblings redacted it. A write now re-emits a section header it was not asked to touch byte for byte. The blank-line normalisation around it is left alone: making the writer position-faithful is a change to its model, not a fix. * fix(scripts): match example uuids case-insensitively in the spec audit The pattern only recognised lowercase hex, so an uppercase id in a published spec was never examined and the audit reported success without having looked at it. Matching case-insensitively is not enough on its own: hex is case-insensitive, so a mixed-case id counts `A` and `a` as two digits and reports twenty distinct ones rather than sixteen. That inflated count clears the threshold the texture test uses to recognise a hand-authored placeholder, so a real id could have passed for one. The allowlist is an exact-string lookup and would likewise have missed an uppercase spelling of an entry. Both checks and the lookup now take a normalised id, while the finding still reports the spelling as it appears in the file. * fix(cli): stop a refusal being swallowed, and gate example ids by name A blank root flag was refused everywhere except `profiles`, where the catch that lets a broken profile still list absorbed it and the command exited 0 after printing the table. The refusal now carries its own error class, which is what the listing rethrows on — the two are distinguished by type rather than by matching message text, and a genuinely broken profile still lists. The unknown-profile message redacted the name the caller typed but not the suggestion or the list of configured names beside it, which come from the same file and are equally attacker-influenced once it has been hand-edited. Those are redacted now, as is every other message in these two files that quotes a name read out of the config, and the profile listing flattens the names it renders the way it already flattened the error column. The example-id audit judged a uuid by its digit texture, on the premise that a real one essentially never looks hand-authored. Measured against ten million generated ids, 0.81% of them do — one in 124, where this change alone replaced six. Requiring each digit exactly twice takes that to zero but rejects all fourteen placeholders now in the specs, so it is no cheaper than the alternative. The audit now holds the eighteen ids the specs actually use, which is one file rather than the twenty-seven a reserved format would touch, and a new id fails until someone lists it — which is the review the check exists to force. * fix(scripts): match the uuid sentinels exactly rather than by shape Accepting any id built from at most two distinct hex digits let something through that was never on the approved list. A generated id essentially never has that shape, so the practical risk was small — but this check had just stopped being a shape test and become a list, and a structural exception is the one thing that undoes that. The two ids it exists for are the nil and max sentinels, and both are matched by value now.	2026-08-26 17:53:39 -07:00
Waleed	c930830310	fix(v2,cli): second audit pass over the v2 API and CLI (#7126) ... * fix(chat): resolve a caller-supplied conversation id through its owner The v2 chat route used the caller-supplied conversationId verbatim, with no existence, owner, or workspace check, against a store keyed by bare text with no owner column. A caller who knew another user's conversation id reached that conversation. Ids now resolve through the same owner-scoped loader the web chat path uses, and anything unresolvable answers one uniform 404 before any lifecycle work runs. Omitting the id mints a server-issued conversation. The contract also accepted any 1-128 character string for a column typed uuid, so a malformed id raised a driver error and rendered 500 while an unknown but well-formed id rendered 404 - a shape oracle, and a 500 on ordinary input. The ownership predicate had no coverage anywhere: the route test mocked the module and the lifecycle test drove a chain mock that ignores its where clause, so deleting the owner condition left both suites green. It is now asserted by composition and by condition count, which is what catches a dropped condition. Also renames the reply's model identifier away from a term the project's own copy rules forbid on a user-facing surface. * fix(v2): conceal workspace absence, and stop archived tables faulting their page Two reads answered a caller more than they were entitled to know. A workspace a caller cannot reach at all returned FORBIDDEN while one that does not exist returned NOT_FOUND, so a workspace-key holder could enumerate which workspace ids exist by diffing the two. Both now answer the same absence, using the concealment policy the billing routes already use. A refusal from inside the workspace - a member whose role is too low - still answers FORBIDDEN, because that caller already knows the workspace exists. Separately, archiving a folder cascades onto its tables but leaves each table pointing at the archived folder row. The archived listing resolved those paths strictly, so one such row faulted the whole page and no cursor could step past it - which also made the ids undiscoverable and left restore unreachable for exactly the tables that need it. The archived scope now resolves leniently to the root, where a restore would place them, matching the shipped workflows behavior. Active listings still fault loudly on a dangling folder. * fix(knowledge): validate upload processing options without stranding live sessions recipe and lang were accepted as free strings up to their length caps, silently discarded, and echoed back nowhere, so a typo was unobservable: uploading with a misspelled recipe returned 200 and quietly used the default. Both are now validated at the boundary and a bad value answers 400 naming what is accepted. The accepted recipe set deliberately includes the sentinel every first-party caller sends today alongside the three real chunker recipes, and the three are derived from the chunker's own union so removing one there is a compile error here rather than a silent 400 in production. The same schema also parses metadata read back off a persisted upload session, so tightening it would have thrown out of resume and complete for any session created before this - a 500 on work that could then never finish. The read-back path now drops a value it no longer recognises instead of rejecting it; the request boundary stays strict. Neither field reaches chunking, so nothing here moves chunk boundaries, embeddings, or search results. * fix(v2): honour a requested stats window, and answer a claimed graph id with a conflict Log statistics accepted a start and an end, filtered the totals by them, and then built the series against wall-clock now. Bucket width was computed over a span the caller never asked for, and every bucket past the requested end was structurally empty - so a bounded historical query returned a wrong-width series with fabricated trailing buckets, under a window label that disagreed with the request. Each edge now honours the bound it was given and keeps its previous derivation when omitted, so an unbounded request is unchanged. Separately, block, edge and subflow ids are global primary keys while the delete that precedes a state replace is scoped to one workflow. An id owned by another workflow survived that delete, the insert violated the key, and because callers pass their own transaction the driver error escaped unclassified as a server fault. The write now refuses such an id up front with a conflict naming it, and re-classifies the same violation if one races past the check, since the lock covers only the workflow being written. The dry run checks the ids a commit would insert and reports the warnings a commit would report, which is what its own contract already promised. * fix(secrets): let a workspace secret change its metadata without resending the value Restoring redaction cost more than removing it. The only way to flip a secret back to redacted was to re-send the plaintext, because the write required a value and omitting it fell into an interactive prompt that cannot run in CI. A workspace secret can now change its description or visibility on its own; the stored value is never re-encrypted or rewritten, a write that names no existing secret answers not-found rather than creating one, and a personal secret still requires a value because it has no other writable field. The path parameter was also one shared schema across the write and the delete, so a single description had to cover both and the delete documented an argument that could create and replace. Split, mirroring the credentials pair. The metadata write is a new update against the credentials table, so its scope is asserted by composition and by condition count: an unscoped update would let one workspace flip another workspace's identically-named secret out of redaction, and the cache invalidation would then carry that flag into the other workspace's runtime catalog. * fix(v2): say what an error means in terms the caller can act on A size-limit refusal collapsed every value under a kilobyte to "0 Bytes", so a 28-byte file over a 27-byte ceiling read "is 0 Bytes, above the 0 Bytes limit" - self-contradictory, and useless for choosing a value that would work. Errors and field descriptions also told callers to invoke raw HTTP endpoints. These strings serve the REST reference and the CLI's own help equally, so they now name the operation and its object rather than a method and a path. A sweep test walks every v2 schema description and holds the line, with the remaining offenders in files this change does not own recorded explicitly rather than left to be rediscovered. Listing the editors of a built-in skill claimed the skill did not exist, while reading the same id succeeded - a well-formed request for a real resource is not malformed, so the list answers an empty roster and only the mutations refuse. Bulk folder deletion recorded only the leaf name in its audit trail while the single delete recorded the full path, leaving two same-named folders under different parents indistinguishable after the fact. Bulk chunk enable, disable and delete each treated an unmatched id differently behind one sentence of documentation. They now follow one rule. A workspace-scoped list refused with the name of a resource the caller never addressed, which reads as an empty workspace rather than an unreachable one. * fix(cli): stop a config value forging a section it was never meant to write The config file is written by joining names and values into INI lines, and nothing checked what was in them. A profile name carrying a newline and a section header wrote a section that merged into a different profile and took over its endpoint - and the next command sent that profile's stored API key there. A workspace value could do the same from the other side, since only the endpoint flag validated its input. The refusal now lives at the writer, the single place untrusted text enters the document, with the flag-level checks kept for the better message. Either alone blocks the forgery; the pair is deliberate. Rejecting rather than escaping, because the format has no escape syntax and these files are hand-edited and read by other tools that would not decode one we invented. The forbidden set covers control characters and the two Unicode line separators, which the previous guard missed - those parse as an unreadable line, so the key silently vanished on read and the next write appended a duplicate while the command reported success. Login also wrote the key before the settings, so a malformed response from the deployment could leave a key on disk with no endpoint beside it, and the next command would send it to the default host. Settings are written first, and the response is checked before anything touches disk. Name validation applies only when creating a profile, so a hand-written one that predates the rule keeps working. * fix(docs): tell the reader which key a command needs, and stop the ids contradicting the CLI Around sixty v2 operations refuse a workspace API key, and the CLI's help said nothing about it - the caller found out from a 403 after the request went out. The restriction is already stated in the API spec, so the generator now reads it from there and the command description carries it. The sentinel sentences are imported from the spec's own constants rather than copied, so a reword cannot silently unmark every command, and the test pins the count as well as named operations because a reword confined to one family would otherwise slip past. The generated reference also rendered an empty default as a sentence pointing at nothing - "Defaults to ." - for every repeatable filter. Omitted now, while false and zero still render, which is the trap that shape of check usually walks into. The hand-written guides used a workflow-shaped id for workflows that the CLI's own help says never names one, and five other families were equally wrong. All of them now match the scheme the CLI declares, consistently per entity across pages, with the shared ones taken from that help text so the two read as one voice. The page documenting every flag was linked from nowhere; both landing links pointed at the overview instead. And the generator's test file was absent from the hand-maintained list CI runs, so its guards never executed. * fix(cli): stop a page-size default capping a destructive filter Every request field named limit inherited the pager's default of 100, but only a cursor-paginated command interprets that flag. The two filter-based row mutations declare no cursor, so the default went onto the wire as a row cap: a filter matching 250 rows deleted 100, exited 0, and said nothing - while the confirmation the user had just answered promised every matching row. The flag's own help offered 0 for everything, which those endpoints reject; the unbounded form is the field being absent. The pager's default now applies only where the pager runs, and the tests pin the omission on the request body rather than in help text. A cap typed alongside an explicit row list was silently ignored; it is now refused on the client, where refusing costs nothing to already-installed versions. Lists also truncated at a hundred with no signal in any format, and the two inventory endpoints that do report truncation had that field dropped on the way out - so a caller reconciling against a clipped list could not tell. One note now goes to stderr while stdout stays a bare array, and a flag raised on a later page survives the fold. Also: a folder whose name contains the separator no longer prints a path that resolves to a different folder; validation errors name the flag the user typed instead of the wire field; an unknown subcommand with --help exits non-zero instead of printing the parent's help; a fractional or negative page size is refused rather than floored; an empty query filter is refused rather than silently returning everything; and the two spellings of the missing-workspace message became one. * fix(cli): gate destructive table imports and fix follow-mode rendering A `tables import --mode replace` empties the table before its first batch, so the only warning was in the describe. It now confirms, and the wording tells the truth per mode: cancelling a replace leaves a prefix of the new file with the originals already gone, while an append re-adds its rows if the file is imported twice. `--yes` skips the gate, and the gate runs before the file is opened. Import and export cancellation carried no describe at all; the import one now confirms, the export one records why it deliberately does not. Follow-mode output truncated cells to whatever the first row happened to measure, so a longer status or workflow name arrived clipped with no signal. Cells now clamp at a shared ceiling and pad to the lock, and the log columns carry width floors so a short first page cannot pin a column narrower than its own values. Interrupting a staged download left the staging directory behind; it is now removed on SIGINT and SIGTERM before the signal is re-raised. `--select-output` without `--follow` selected from a response that does not carry outputs, and said nothing. It is refused client-side, with a separate message for `--async`. Its describe now names what the path addresses. `secrets set` always read a value, even when only metadata flags were passed. Off a TTY that was an immediate refusal, so a metadata-only edit exited 1 in CI for a value it was never asked for; on a TTY it stopped to prompt, and the prompt rejects an empty entry, so there was no way to say "leave the stored value alone" short of re-typing the secret. The read is now skipped and the field omitted, which is what lets a metadata-only edit run unattended. On a TTY, setting only a description no longer prompts. Passing both spellings of the reveal flag is refused rather than silently resolved. Four mandatory hand-authored flags now say so, `billing logs` names its key-type scope, and the dispatch list declares its columns. * fix: close the gaps an adversarial review of this branch found A conflict handler added earlier in this branch was dead code. It read the Postgres error code off the thrown object, but the driver error arrives wrapped with the real one on `cause`, so the check returned false on its first line and the 409 never fired. Its test passed only because it threw a flat shape production never produces. It now reads through the cause chain with the shared helpers, compares the constraint name exactly instead of matching a substring of the SQL, and its test throws the real wrapped error. Resuming a conversation checked its workflow and its workspace but not its type, so a conversation created by the web surface could be continued as a CLI turn. It now refuses through the same uniform 404 as every other mismatch, which closes the same omission on the web posting path. Minting one no longer leaves a blank untitled row at the top of the Chat list. The pre-write check on a minted API key refused fewer characters than the writer does, so a key the check accepted could still fail at the write — after the endpoint beside it was already stored, pairing a new endpoint with the previous key. The two had drifted because the set was spelled three times; there is now one. A description claimed a processed count reported only the chunks that changed. The update returns every row it matched, so re-enabling chunks that were already enabled counts them all. Two OpenAPI sentences promised no conflict detection and no persistence warnings in a dry run, both of which the same branch had just made false. A described window was wrong whenever a start was supplied without an end. Listing the editors of a built-in skill answered a read with a modification refusal on the internal surface. Archived table listings could reach the strict folder projector again through a third scope value the input type still allowed. A metadata-only secret write skipped the guard its personal-scope twin has. The internal document boundary still took the two processing fields as unbounded strings. Truncation was reported only from the response envelope, so a clipped file body, row search and workflow-stats list said nothing. A staged download stopped watching for signals before it finished removing its directory, and cleared every listener for the signal rather than its own. Three tests asserted a contract constant against itself; they now drive rendered help, real argv, or real render output. * chore: regenerate the API reference, CLI surface, and CLI docs The published reference still marked a secret value required and described the delete parameter as one that also creates, the CLI surface still lacked the marker that says which operations refuse a workspace key, and the reference rendered an empty sentence for every repeatable filter whose default is an empty list. * test(cli): use the package's own delay helper in the staging poll The audit bans a hand-rolled setTimeout promise. `sim-cli` does not depend on the shared utils package, and its own idiom is `node:timers/promises`. * fix: act on a second review round, and correct two earlier claims The conflict pre-check read block ids from the wrong side. The writer inserts each block's own `id` field while the check read the record key, and the two can diverge because preparation copies a value under its key without reconciling them. Edges already read the value and subflows are genuinely keyed by the record key, so only blocks were wrong — collecting every family from the values, as first suggested, would have broken subflows instead. A minted API key carrying leading or trailing whitespace passed the pre-write check but failed the writer, leaving the new endpoint on disk beside the previous key. It is refused up front now rather than trimmed: a key is opaque, so trimming would store a value the server never issued and turn a loud failure into an unexplained 401 later. The endpoint normalizer does trim, which is what made a padded `--endpoint` fail only after the browser flow had already minted a key. A metadata-only secret write raced with deletion returned 500, because the follow-up read that only assembles the response body threw an unclassified error; it now reports the same not-found the non-racing miss already gave. An unusable output format in the environment silently printed a table instead of refusing. Two validation messages printed control characters verbatim. A dry run now reports the preparation warnings its own commit path returns. The chat route created a titled conversation and never wrote a message, so it appeared in the Chat list promising content it did not have. Both sides of a successful turn are now persisted; a failed turn still writes nothing, so a question is never stored without its answer. Two claims of mine were wrong. The earlier commit message said `secrets set` sent an empty value that overwrote the stored secret — it did not; the prompt refuses off a TTY and rejects empty on one, so the old behaviour was a clean refusal. And the delay helper commit said this package's idiom is `node:timers/promises`; the package carries its own `sleep`, which is the audit's sanctioned home and has five callers. It uses that now. Also: a test asserting a deadlock stays unclassified could not fail, since every candidate rejects it; it now pins a unique violation carrying no constraint name. Workflow ids spelled with the file prefix are corrected in the remaining fixtures, leaving the genuine file ids alone. * fix: close a credential-misdirection path this branch had opened Making the endpoint normalizer trim handled whitespace around a value but not a control character inside one, and the URL parser removes those from anywhere in its input — so a value that reads as one host could resolve to another, and the profile's key went with it. The flag and environment paths never touch the config writer, so its guard did not cover this. The normalizer now refuses the same character set the writer does, which also keeps the invariant that nothing it blesses can be refused by the write that stores it. Comparing the parsed URL back against its input was the alternative and is wrong: the parser rewrites percent-encoding, case, internationalized hosts and default ports, so legitimate endpoints would be refused. The blank-query guard tested for exactly empty, so a whitespace-only value still reached the wire — as a real zero on a numeric filter, an explicit false on a boolean one, and as an encoded space the server then rejected. It now refuses any value that is blank once trimmed, while a body string keeps its meaning, an explicit zero still sends, and a value with content around its whitespace is passed through untouched rather than trimmed. A graph-id conflict reported 409 on the v2 route and fell through the older persistence wrapper as an unclassified 500. That wrapper now classifies orchestration failures through the cause chain, which also fixes a pre-existing case where a workflow archived between authorization and the locked read reported 500 rather than 404. Persisting a chat turn claimed its row by id alone, so a conversation soft-deleted mid-turn still received the messages and was bumped back up the list. It now requires a live row. A turn whose caller hung up after the model had already answered persisted nothing, though the work was done and billed; it now persists and still reports the connection as closed. An empty workspace id from the login response was read as no workspace at all. A published description still promised a language-tag standard the schema does not enforce. The test asserting that a turn is stored before the final event drained the whole response first, so it held whichever order the code used. It now reads the stream incrementally and fails if the write moves after the event.	2026-08-26 15:05:00 -07:00
Theodore Li	d86fdc152f	feat(api): publish agent tool input schema (#7109) ... * feat(api): publish agent tool input schema * fix(api): bound agent tool inputs	2026-08-26 14:37:50 -04:00
Waleed	4508ec75d2	fix(cli): resolve blockers and majors from a full command-surface audit (#7083) ... * fix(cli): resolve blockers and majors from a full command-surface audit Audit of all 222 CLI leaves against a live deployment, plus fixes for every defect it confirmed. Blockers: - An unrecognized --profile resolved to built-in defaults, so a typo silently targeted production and transmitted the API key there. - sim logs follow sent an undeclared query key and failed on every invocation. - sim workflows run exited 0 on a failed run, so CI reported success. - knowledge connectors documents update matched rows already in the target state, making exclude and restore permanent no-ops. - PDF text layers below the OCR threshold are transcribed by a model and stored verbatim with no record that it happened. Majors include: rollback --version was swallowed by the program-level flag and silently did nothing; nullable string flags could not send null despite their help promising it; six protocol commands discarded excess arguments, dropping files on upload; sim chat crashed with EPIPE when piped to head; tables import dropped malformed CSV rows without reporting them; audit-logs required an organization id no API surface exposed; secrets could not opt out of redaction or read a value from a file; bulk deletes and moves exited 0 having done nothing; and MCP registrations were destroyed by undeploy rather than restored. Adds extraction_method to documents so OCR output is distinguishable from parsed text, and reports the applied scope on billing logs so the two ledger questions are no longer indistinguishable. * fix(mcp): bound MCP restore by server and re-check uniqueness under the lock Two gaps in the archive/restore lifecycle this branch introduced. The candidate query bounded archived rows and deduplicated to one per server afterwards, so several archived generations stacked on one server consumed the whole budget and every other server the workflow had been published on fell out of the result with no warning. Deduplication moves into SQL so the bound applies to servers, preserving most-recently-updated-per-server. The live-registration check ran before the server lock was acquired, so a concurrent tool create could land in between and the restore would un-archive a second live row for the same server and workflow, violating the partial unique index and rolling back the whole deployment. That check now runs under the lock alongside the tool-name, capacity, and metadata-budget checks it belongs with. * fix(review): address round-three review findings across CLI and server Restore now picks candidate servers by recency: DISTINCT ON forces its own key to lead the sort, so bounding on that statement kept the lexicographically lowest server ids and left a workflow's most recently used servers archived. Deduplication and bounding are now separate stages. CLI: a total miss on tables move reported only in notFound exited 0; unsetting a key or removing a profile mutated the first duplicate INI block while reads merged later ones, so the removal appeared to succeed and did nothing; an import that rejected cells but no rows showed a clean progress line; and the --run-id help implied idempotency it does not provide. Server: a run with no recorded output projection let block-name selectors past the new validation; the billing window comparison still fired on a bound that parsed but failed the shared schema; CSV rejection accounting reached only the streaming path, so buffered and synchronous imports still dropped records silently, through to the Copilot tool that reports them; case-insensitive tag name uniqueness now serializes on the knowledge-base row the delete paths already lock; and a failed sync claim reports the lifecycle reason rather than always claiming a sync is in progress. Reverts an over-scrub from the previous commit: workspace-file-imports is consumed only by Copilot, so naming save_upload and glob there is the correct remediation rather than a leak, and the sweep that guards against leaks now exempts it explicitly. Corrects two contract descriptions that promised a bulk tag save would rename or relocate an occupied slot, which it deliberately no longer does. * fix(cli): remove flags a caller cannot use, and correct three that misled Removes surface that should not have shipped: - `files uploads get` is hidden. Its `--upload-token` was required, and the token is minted and consumed inside a single `files upload`, which completes or aborts its session before returning. Nothing in the CLI could produce the value, so the command answered every invocation by asking for something unobtainable. The same flag is dropped from the two table-import commands, where a CLI-created import is already queryable without it. - The `--no-<flag>` companion that sent JSON null is gone. `--no-X` means "send boolean false" on thirty-seven other flags, and one spelling should not carry two meanings. `--description ''` already clears the displayed value, and the help now warns that the literal word null is stored as text rather than suggesting a substitute, because on the OAuth client fields null revokes a stored grant and an empty string does not. - The document extraction method column and its contract field are reverted. Nothing read them, they were null for every existing document, and the name collided with the parser metadata field that already exists. Corrects flags that misled: the workflow move destination is `--to`, matching its two siblings rather than meaning the opposite of `--folder` one command over; `files list --recursive` is a bare flag like the four folder deletes rather than a twelve-alias string; the dispatch row cap takes a count instead of its wire object; `--yes` no longer claims to be required on commands that accept `--dry-run`; cancelling every run on a table is confirm-gated; and the retry-processing negation, which the route rejects, is suppressed. Extends the guard that missed all of this: it swept only `--x-` prefixes over generated commands, so a header spelled without one was invisible to it. It now derives every header name from the operation table and sweeps the assembled program. * fix(mcp): budget MCP restore against the workflow's live server fanout Restore bounded its candidates at the per-workflow server limit counting archived rows only, never subtracting the memberships the workflow already holds live. The fanout validation a few lines later in the same deploy transaction counts live servers against that same limit, so a workflow with both live and archived registrations could restore past it and roll the whole deployment back. The candidate query now bounds on the remaining headroom, and the budget is re-checked under the server locks and spent once per accepted candidate, so a create that lands between the count and the unarchive cannot push it over. Candidates that do not fit are dropped by recency, matching how the set is already selected, and stay archived with a warning naming the workflow, the server, and the reason — restore still never throws inside the deploy transaction. One residual is left open deliberately: a create on a server outside the candidate set is serialized by neither the locks nor the recount. Closing it would need a workflow-level lock, which would change the ordering every other writer here depends on, and the create path runs its own limit check. * fix(mcp): stop restore spending its budget on servers it cannot restore A server can hold both a live registration and archived ones for the same workflow: the partial unique index constrains only the live row. Such a server was counted twice — once shrinking the restore budget, once consuming one of its slots — before the liveness check under the lock skipped it. While the bound was the full server limit that waste was invisible; once the bound became the remaining headroom, every slot spent that way cost a registration that could have been restored. The candidate query now excludes servers the workflow is already live on, so the budget and the candidate set agree. The exclusion sits on the inner stage, before deduplication and the bound, and is a pre-lock optimisation only: the check under the server lock stays authoritative, because the query can go stale between reading and unarchiving. Candidates rejected for a tool-name collision, the per-server cap, or the metadata budget are still not replaced. That case is only knowable under the lock, so replacing it would mean fetching past the bound and locking servers outside the candidate set, widening an ordering every writer here relies on.	2026-08-25 18:43:48 -07:00
Waleed	32a15fde2f	improvement(access-control): wire tool and model permissions into copilot editing (#7080) ... * improvement(access-control): wire tool and model permissions into copilot editing Permission groups already supported a per-tool denylist (deniedTools) and model restrictions, but only the canvas honored them. The copilot edit path gated on block type alone, so Sim could build workflows using tools and models the user was not allowed to run — the executor refused them at run time instead. Enforce both at authoring time, and stop advertising what the viewer cannot use. * fix(access-control): use the path alias for the permission-groups type import * fix(access-control): close two denied-tool leaks in copilot discovery The VFS stamped every integration schema from the shared static map before the per-viewer loop re-authored the permitted subset, so a denied operation's schema stayed published. Skip the shared copy for integration paths; the viewer loop is the only projection that knows the denylist. Block metadata resolved denied operations from the catalog's `operation.toolId`, which the projection fills only from `tools.config.tool` — a block whose operation ids are its tool ids left it undefined and read as fully permitted. Resolve through the shared operation gate instead. * fix(access-control): key the copilot schema cache on permission policy The deferred integration-tool schemas now depend on the viewer's permission group, but the cache key encoded only identity and block visibility, so an admin's change to deniedTools took effect only when the entry expired. Resolve the config before the key and add a gate signature alongside the existing visibility signature, mirroring how block visibility already keys the same cache. The read moves out of the cached section rather than being added: what the entry caches is a user-tool schema per exposed integration tool, which dominates it.	2026-08-25 15:41:09 -07:00
WaleedandTheodore Li	656840a1b1	feat(api): make the platform operable headless over v2 (#6912) ... * feat(v2): download run output files by API key Adds GET /api/v2/workflows/{id}/runs/{runId}/files/{fileId}, closing the async-run loop for headless callers. A run's output carries UserFile URLs pointing at /api/files/serve/..., which rejects x-api-key outright, so an async run that produces a file previously had no byte path out for an API key at all. The file is addressed by the id the run reported and resolved against the run's own recorded execution data, from which the storage key is read. The request never supplies a storage key, so the endpoint cannot be aimed at bytes the run did not produce. Resolution deliberately reads the materialized-but-undisplayed recording, because the display projection strips exactly the `key`/`context` fields a byte read needs. Also hardens normalizeStartFile to derive a file's storage key only from a validated internal serve URL, discarding any caller-supplied `key`/`context`. A workspace API key has no human subject, so the executor resolves its actor to the workspace billing owner (preprocessing.ts -> resolveSystemBillingAttribution); verifyFileAccess then authorizes a workspace-context key as that owner, whose reach is not bounded by the key's workspace. Accepting an attacker-authorable key made that substitution exploitable as a confused deputy. Normalization is all-or-nothing, so a forged file now drops the whole files input. * feat(workflows): one graph-write door, principal-derived audit source, and v2 authoring endpoints Extract replaceWorkflowNormalizedState as the single persistence primitive for a workflow graph replace and route both the internal editor save and the Copilot edit tool through it, so neither can skip state preparation, the row lock, the lastSynced stamp, or custom-tool extraction by choosing a different entry point. Derive the audit source from the acting principal instead of hardcoding 'copilot', then widen workflows.variables.apply_operations and workflows.bulk.move to every principal kind. Add GET/PUT /api/v2/workflows/{id}/state, POST /operations, /duplicate, /restore, PATCH /variables, and POST /api/v2/workflows/move over surface-neutral application use cases; move the edit engine to lib/workflows/editing. * test(workflows): cover the graph-write primitive, audit source, and the v2 authoring surface Pin the two-doors fix (preparation runs, the row is locked, custom-tool extraction is post-commit and best-effort) and the false-audit fix (a session principal writes source: 'session', a delegated one writes its service). Both were verified to fail with the fix reverted. Add the application matrix for replaceWorkflowState, applyWorkflowOperations, readWorkflowGraph, and restoreWorkflow — role floor, principal-kind rejection before canonical load, asserted-scope concealment, lock, validation, atomic conflict, plan gate, and audit-then-notify ordering — plus route tests for every new endpoint. * test(workflows): pin the internal graph-write door and the v2 list scope Characterize saveWorkflowNormalizedState's statuses, messages, and notification after the persistence extraction, and cover the new scope filter on GET /api/v2/workflows including a cursor replayed under a different scope. * feat(api): add v2 block, tool, connector-type, and enrichment catalogs Adds six read endpoints under /api/v2 that publish Sim's code-defined catalogs: GET /blocks, GET /blocks/{blockId}, GET /tools, GET /tools/{toolId}, GET /connector-types, and GET /enrichments. These read like static reference data and are not. What a caller may place is decided per workspace by its permission-group integration allowlist, per organization by which unreleased blocks have been revealed, per deployment by ALLOWED_INTEGRATIONS, and per workspace again by the workflows it has deployed as blocks. So all six are plain defineWorkspaceOperation reads at minimumRole 'read' with workspaceApiKey 'allow' — the exact policy of credentials.providers.list — and every response keeps Cache-Control: private, no-store, because an unrevealed preview block's existence must not leak across organizations through a shared cache. Trigger blocks ride as ?capability=trigger rather than a second endpoint, and workspace custom blocks ride inside /blocks discriminated by `source`, so "what may I place?" stays a one-call question. The block projection is extracted out of the Copilot get_blocks_metadata tool and rewritten onto @/tools/metadata and @/tools/metadata-outputs. That cuts the tool's own @/tools/registry edge as a side effect: its module graph drops from 6,756 to 1,318, and the new routes land at 1,673-1,734, next to the shipped /v2/credentials/providers baseline of 1,668. Supporting changes: - scripts/sync-tool-metadata.ts derives hostedApiKey ('always' | 'conditional' | 'none') from each tool's `hosting`. The config itself stays excluded because it holds closures, but "does Sim host the key" is a first-order authoring question, so the answer is emitted. - getCopilotToolDescription takes hostedApiKey as an option instead of reading `hosting` off the tool, so both an executable ToolConfig and the generated metadata can answer it through one shared derivation. - principalUserId / allowedIntegrationTypes move out of lib/credentials/application/provider-catalog.ts into lib/integrations/principal-scope.server.ts. Two copies of the workspace integration gate would diverge first on the workspace-key path, which has no user for permission groups to key on. - scripts/check-tool-registry-boundary.ts walked page.tsx/layout.tsx under app/workspace only, so a route importing the executable registry passed green. It now walks a list of entry sources, seeded with the four catalog route subtrees and the shared projection barrel. Routes are covered per subtree rather than wholesale because 122 of ~1,130 route files legitimately execute tools. Registry sweeps parse every block, tool, connector type, and enrichment through its published response schema and compare against the wire round-trip. They caught a real drift while being written: an operation's inputs were typed as a union of the tool-param and block-input shapes, and the union resolved to whichever member matched first, silently dropping a block input's `schema`. * docs(api): stop publishing a 413 GET /workflows/{id}/state cannot emit * feat(v2): read upload-session state Adds GET /api/v2/files/uploads/{uploadId}. Only DELETE was exported, so a caller that lost track of a transfer could abort it but could not ask whether the session was still alive, already finalized, or failed — the resume story was missing. Runs on a new files.upload.read operation at minimumRole 'read' rather than reusing uploadCancel, which is a 'write': asking about a session must not require permission to destroy it. The GET is a control leg like every other, so it carries the signed upload token and re-authorizes the caller's present workspace permission through reauthorizeWorkspaceUploadPurpose instead of resolving the session on its id alone. * fix(api): reconcile v2 catalog and workflow-authoring integration Merging the catalog and workflow-authoring branches surfaced four issues that neither produced in isolation. - Route and OpenAPI counters were bumped to the same value on both branches, so git merged them as one change while the merged tree holds the sum. Corrects the route ratchet to 1142 and the workflows document to 29 operations (152 total), then regenerates the OpenAPI documents and the CLI surface from the reconciled contracts. - The seven new workflow operations were published in the spec but absent from the workflow API reference groups, which `check:openapi` rejects. - `route-policies.ts` reached `WorkflowOperationsNotAppliedError` through `apply-workflow-operations`, dragging the edit engine — and its diff and comparison dependencies, which reach a client OAuth hook — into every route that uses the shared workflow error policies. The class moves to its own leaf module, mirroring `WorkflowImportError`, and each importer now takes it from there. - The operations route test shadowed that class inside its module mock, so `instanceof` matched a fake and the assertion pinned a message the production class never emits. It now uses the real class and asserts the real message. * feat(v2): extract ZIP archives over the public API Adds POST /api/v2/files/{fileId}/extract and widens files.extract_archive from principalKinds ['session'] / workspaceApiKey 'deny' to admit personal and workspace API keys at the unchanged 'write' role. The widening is an authorization change, so the justification lives in the operation's TSDoc: extraction grants no capability an API key lacks, since every file it writes could be created one at a time through files.create and files.upload.create, both already 'allow' at the same role. It only collapses many calls into one. The previous ['session'] restriction read as an artifact of the UI having been the only caller. Delegated services stay out — no copilot or executor caller exists and admitting one is a separate decision. The response is counts plus the destination folderPath, never the extracted files: a large archive would otherwise materialize thousands of objects into one body. Callers page GET /api/v2/files?folderPath=... instead. The use case returns the internal display path and the adapter projects it to a v2 path, keeping the use case surface-neutral. * feat(v2): extract file text over the public API Adds GET /api/v2/files/{fileId}/text. Text extraction previously sat behind checkInternalAuth on /api/files/parse, a route that also mixes in external-URL fetching, execution-file upload, and multi-file aggregation, so it could not be reused. The parse call is lifted into a thin application use case instead. Runs on the existing files.read_content operation unchanged — it is already workspaceApiKey 'allow' at the read role, and turning bytes it already authorizes into text grants no further reach. `degraded` is a required, non-optional boolean on the response. The legacy doc and ppt parsers deliberately return best-effort or placeholder content rather than throwing, so an omittable flag would let a client that never checks it treat guessed text as extracted text. It is reported honestly rather than converted into an error, because the parsers' behaviour is deliberate and characterization-tested. The read is bounded on its input at 25 MiB before extraction rather than on its output after, given the parsers' documented DoS history; a caller may lower the ceiling but never raise it. * feat(v2): restore archived folders and list the archived set DELETE /api/v2/files/folders archives recursively, so a recursive delete was unrecoverable over the API: the archived files stayed visible through GET /api/v2/files?scope=archived, but nothing could rebuild the folder structure. Adds POST /api/v2/files/folders/restore, path-addressed like the rest of the v2 folder family, and a `scope` selector on the folder list so a caller can find the archived path to hand it. `scope` extends the files folder-list query rather than the shared v2ListFoldersQuerySchema: only workspace files have an archived folder set, so adding it to the shared schema would give tables, workflows, and knowledge a parameter they ignore. GET /api/v2/files/folders is a FULL_SET_LIST, not paged, so no cursor binding changes — list-pagination.test.ts passes unchanged. Restore resolves the archived folder from its path by scanning the archived set rather than walking the live tree, which by definition does not contain the folder being restored. The folder-restored analytics hook now reports the folder actually restored rather than the requested selector, which carries no id on a path-addressed surface. * feat(v2): bulk-download a file selection as a zip Adds GET /api/v2/files/bulk-download, an adapter over the existing downloadWorkspaceFileItems use case and its internal binary route. Path collision: a static segment beside [fileId] permanently shadows a file whose id equals it, and workspaceFileIdSchema does accept [A-Za-z0-9_-]+. Rather than invent a new shape, this follows the existing bulk-delete sibling: the hyphenated form cannot be produced by either minted id shape (UUID v4 or wf_<shortId>), so the shadowed id is unreachable in practice. Documented on the contract so the reasoning is not lost. Folders are addressed by path, matching the rest of the v2 file surface. The paths resolve against the folder set the selection already loads, so it costs no extra query, and a path matching no folder is rejected rather than silently dropped — a misspelled folder must not yield a zip of whatever else was selected. The empty-selection and folder-count guards now account for folderPaths, which a path-only selection would otherwise have tripped. Selections are comma-separated only: v2 rejects a query parameter sent more than once, so a repeated-parameter form would never reach the schema. Pinned by a test so the contract cannot advertise a form the boundary rejects. * feat(v2): expose run output files and optional inline bytes on the runs read GET /api/v2/workflows/{id}/runs/{runId} now reports the files a run produced, each with the downloadPath that fetches its bytes, and can inline them as base64 on request. Gated by includeOutput, matching `output`'s nullability: a caller that did not ask for output does not receive a file list it did not request. The async execute request's rejection of includeFileBase64 is deliberately left alone — at submit time the run has not happened, so there is nothing to inline; reading a finished run is the first moment the question means anything. Inlining is capped per file at the executor's 16 MiB inline ceiling, which a caller may lower but never raise. A file above it answers 413 naming that file's downloadPath, so the caller is told exactly how to get the bytes rather than being left stuck. The descriptor deliberately omits the storage key — files are addressed by id and the key is re-derived from the run's recording — and omits an expiry, which the recording does not carry and which would be fabricated if published. The route becomes headSafe: false, since inlining reads object storage. The builder enforces that this requires the use case to expose authorize(), so HEAD still answers from a real authorization rather than from authentication alone. * feat(v2): permanently delete an archived file DELETE /api/v2/files/{fileId} only archives — the OpenAPI says its stored bytes are never removed — so there was no way to actually destroy a file over the API. Adds the repository primitive, application use case, operation, and DELETE /api/v2/files/{fileId}/permanent. A distinct path rather than a flag on the ordinary delete: a query parameter that turns a recoverable archive into an irreversible destruction is set by accident, and the two acts carry different minimum roles, which one route declaration cannot express. The file must already be archived; a live file answers 409 naming the archive step, so no single request can turn a live file into lost bytes. minimumRole 'admin', which forces workspaceApiKey 'deny' since the workspace-key ceiling is 'write' — the desired policy anyway: unattended credentials should not destroy bytes. Row first, then object. The two legs commit independently, so one can survive a crash between them: deleting the row first leaves at most an orphaned object for the storage sweep, while the reverse would leave a live row pointing at bytes that no longer exist — a file that lists and opens but can never be read. A failed object delete is therefore reported as objectDeleted: false rather than thrown, because the request has genuinely succeeded once the row is gone. Both directions are pinned by failure-injection tests, verified to fail when the order is reversed. Audited as a distinct FILE_PERMANENTLY_DELETED action, not a reuse of FILE_DELETED, which records the recoverable archive step. * feat(api): v2 log analytics, itemized cost, filters, and sortable query Adds the aggregate and rich-read halves of the public logs surface, and fixes three defects the existing reads carry. Aggregate analytics. `GET /api/v2/logs/stats` returns time-bucketed run counts, success rate, error count, mean latency, and the window bounds, per workflow and for the workspace. The first-party route was a raw handler with inline SQL and inline aggregation, so it is split into a repository (`lib/logs/stats-queries.ts`), a pure aggregator (`lib/logs/stats.ts`), and an application use case. That route keeps its legacy authorization — it answers a caller without workspace access with a zeroed 200, where v2 conceals the workspace as a 404 — and consumes only the two surface-neutral halves. `segmentCount` had no `.int()`, `.min()`, or `.max()`, so `0` divided by zero and `1e9` allocated two billion-element arrays: both caller-reachable 500s. Bounded on both contracts. `workflows` is capped, with the workspace totals still computed from every workflow and the cut reported as `workflowsTruncated`. Detail reads gain the itemized `cost.items` ledger (`null` and `[]` are distinct answers and both reachable) and `workflowInput`, restoring a v1→v2 regression. The list gains `workflowName` and `status` filters, and `includeJobRuns`, which unions Chat and Sim-agent job runs into the sequence behind a new `kind` discriminator — without it a job run is indistinguishable from a run whose workflow was deleted. A filter no job row can answer drops the branch outright rather than meaning two things across the union. `POST /api/v2/logs/query` carries the additional sort columns. `GET /logs` is untouched: its single `order` param rests on there being exactly one sortable column, and both escapes from that are ruled out, so the rich read gets its own endpoint — the split the table surface already ships. It uses the shared keyset scheme with the two nullable sort columns read through a sentinel, since a keyset cannot compare against null. `folderPaths` now covers a folder's whole subtree on the public path, as it already did everywhere else; it previously omitted every nested run with no error. The path strings did not change, so a folder-scope version is stamped into the cursor and in-flight tokens restart rather than silently skipping rows. Also fixes `folderName`, which ILIKEd `workflow.name` — a copy of the clause above it — and so searched workflow names instead of folders. `buildLogSortCursorCondition`'s `IS NULL` disjunct is documented and pinned: under `NULLS LAST` the null block is only reachable through it, so removing it as a duplicate-row fix makes those runs unpageable. Ratchets: route count 1142 -> 1144; logs OpenAPI operations 2 -> 4; total operations 152 -> 154. * feat(api): v2 tables run state, dispatch polling, batch update, bulk, archive Closes the headless gaps on the v2 tables surface. - Per-cell run state is now readable through an opt-in `includeRunState` on `GET /rows`, `POST /query`, and `GET /rows/{rowId}`. The default projection is byte-identical; a page whose sidecar outgrows its byte budget is a 413 rather than a silent truncation. - Run dispatches are addressable: `GET /tables/dispatches/{dispatchId}` publishes the column's full four-state domain so polling a finished run is not a 500, and `GET /tables/{tableId}/dispatches` lists what is in flight. - `POST /rows/batch-update` takes one distinct patch per row. Its transaction moved out of the Copilot-only module into a surface-neutral use case both surfaces now call. - `GET .../enrichment/{groupId}` publishes the provider cascade, cost, and timing behind one enrichment cell. - `POST /tables/bulk-move` and `/bulk-delete` reach the existing bulk use cases, which now accept folders by canonical path and resolve them inside the application layer. - `DELETE` is recoverable: `scope=archived` on the table list plus `POST /tables/{tableId}/restore`. * feat(api): expose knowledge chunks, tag writes, archive/restore on v2 Closes the knowledge cluster's remaining public-surface gaps. Chunks: list/read/create/update/delete/bulk under `/api/v2/knowledge/{id}/documents/{documentId}/chunks`. `queryChunks` gains an `id` tiebreaker on every sort so the list pages on a keyset rather than an offset — `tokenCount` and `enabled` are both non-unique, so a page boundary inside a run of equal values used to repeat or drop the tied rows. The internal offset caller is unchanged; the two positioning schemes share one read. Tag definitions: create, update, delete, next-slot, usage, and the document-scoped save and cleanup. Without them a caller could write a tag value into a slot with no definition and then had no way to name it, so tag-filtered retrieval was unbuildable end-to-end. `v2KnowledgeTagSchema` gains `id`, without which PATCH and DELETE are unaddressable. The document-scoped DELETE is pinned to `action: 'cleanup'`: the domain's `'all'` deletes the whole knowledge base's tag vocabulary from a document path. Archive/restore: `GET /api/v2/knowledge/archived` as a sibling route rather than a `scope` param — the two reads bind different operations and a v2 route declares one — plus `POST /api/v2/knowledge/{id}/restore`. `knowledge.restore` is a new workspace operation carrying `delete`'s policy, since an operation's inverse must not be harder to reach; the internal session route now delegates its workspace branch to the shared use case and keeps only the legacy personal one. Also: `POST .../documents/from-workspace-files` surfaces `addWorkspaceFiles`, so a file already in workspace storage no longer has to be re-uploaded byte-for-byte to be indexed; the `chunkingConfig` write widens to the first-party five-key schema with its refines and separator bounds, while the response stays `.catchall` so a legacy JSONB row cannot 500; and `CONNECTOR_MANAGED_RESOURCE_READ_ONLY` joins `FORBIDDEN_DETAIL_CODES` now that the bare 403 on connector-managed chunk writes is wire-reachable. Document upsert is deliberately not included. * feat(api): add v2 credential rotation and a gate-exempt capabilities endpoint PATCH /api/v2/credentials/{credentialId} rotates service-account secret material or renames a credential in place, preserving the credential id so existing workflow, deployment, paused-run, connector, and webhook references keep working. Re-posting to POST /api/v2/credentials answers 409, and delete-and-recreate mints a new id, so rotation previously had no door. The route is adapter-only: updateWorkspaceCredentialUseCase already owned the rotation, its audit projection, and credentials.update. It gains one additive assertedWorkspaceId field for the v2 workspace assertion, and the per-principal credential-type table that deleteCredentialUseCase already applied is lifted into requireManageableCredentialType so both operations share it. Without it a personal API key could rename an env_workspace row and toV2Credential's throw would surface as a caller-reachable 500. CredentialProviderOperationError now maps to 503 with Retry-After when the provider is unreachable, instead of the 400 its OrchestrationError('validation') base projected. A transient outage rendered as a permanent input error invites a caller to revoke a working credential. GET /api/v2/meta reports the calling key's rollout cohort, type, and expiry. It is the one route declaring the new typed gate: 'exempt' option, because the rollout gate and the unknown-path catch-all answer byte-identical 404s and a gated /api/v2/meta could never resolve that ambiguity. Authentication still runs first, so the only fact disclosed is one about the caller's own credential. * feat(api): publish deployment lifecycle and workflow-MCP v2 surfaces Adds the four deployment-lifecycle operations v2 was missing, and the workflow-as-MCP publishing surface, both as adapters over application use cases that already existed. Deployment lifecycle: - PATCH /api/v2/workflows/{id}/versions/{version} relabels a version. Deliberately not the internal route's body-shape dispatch between "rename" and "promote to live". - POST .../versions/{version}/activate promotes a version. Same use case as rollback under a different transition, on its own path because the two mean opposite things to a caller. - POST .../versions/{version}/revert overwrites the draft. Accepts the literal `active` alongside a version number. - PATCH /api/v2/workflows/{id}/deployment toggles unauthenticated public execution. `workflows.public_api.update` widens from session-only to session plus personal API key: it is an admin-role change the same accountable human may make from a script. Workspace keys stay denied. Its EE refusal now carries PUBLIC_SHARING_NOT_ALLOWED instead of a bare forbidden. Workflow MCP servers: - /api/v2/workflow-mcp-servers list, create, update, delete, plus publish and unpublish of a workflow as a tool. Named apart from /api/v2/mcp-servers, which registers the external servers Sim calls. - The six mcp_servers.workflow_deployments operations widen from ['delegated'] to admit sessions and personal API keys; roles and the workspace-key denial are unchanged. - The server list gains keyset pagination, matching its external sibling, since nothing caps how many a workspace publishes. - Server, tool, and workflow reads move out of the use case into lib/mcp/queries. Route ratchet 1150 -> 1160; OpenAPI operations 161 -> 171. * feat(api): extract chat deployments and publish the v2 surface Chat deployment was a shipped module with no public API and two authorization systems: `lib/workflows/application/chat-deployments.ts` had deploy/undeploy extracted, but only Copilot used them — the REST routes reimplemented workflow authorization inline, and `PATCH /api/chat/manage/[id]` additionally owned password encryption, the auth-type field-clearing matrix, identifier uniqueness, the redeploy-gating protocol with two 409s, a raw db.update, and a manual recordAudit. New `lib/chat-deployments` domain: - `chat_deployments.list/read/update/delete`, keyed on the deployment whose workspace is derived by joining its workflow. Creation stays `workflows.chat.deploy`, which is keyed on the workflow. - The PATCH extraction, including the field-clearing matrix and the asynchronous-cutover invariant the route had hand-mirrored from `performChatDeploy`. - One `buildChatDeploymentUrl`, replacing three constructions that had already drifted onto two different host helpers. There is no chat subdomain, so nothing publishes a host. - Repository reads moved out of the use cases into `lib/chat-deployments/queries`. Internal routes are now adapters over those use cases. `GET /api/chat` is deliberately not migrated: it scopes by `chat.userId` while every other chat operation authorizes by workspace admin, and reconciling the two is a product decision. `PATCH` keeps its 400 for an identifier collision through a typed `ChatIdentifierInUseError`; v2 reports the 409 the condition actually is. v2 surface at `/api/v2/chat-deployments`: list, create, read, update, delete. Workspace-scoped, keyset-paged, and a stored password is never readable — reads carry `hasPassword` only, and the session-only reveal endpoint deliberately has no v2 counterpart. Also: an email- or SSO-gated chat with an empty allow-list is now refused in the use case rather than only at the internal boundary, since it is unenterable; and the doc comment on `processHostedKeyCost` claiming a `usageLog` write is corrected — no such write exists. Route ratchet 1160 -> 1165; OpenAPI operations 171 -> 176. * fix(api): close three review findings, two of them caller-reachable - Run output files are filtered to keys under the run's own execution prefix. The recording they came from is not a trustworthy key source: the start block copies every caller-supplied input field verbatim into its output and `collectUserFilesById` accepts anything carrying the `UserFile` shape, so a caller could name any storage key and have the download and base64 paths — neither of which authorizes per file — serve it back. - `getBlock` reads own keys only. `BLOCK_REGISTRY` is an object literal, so `constructor`, `toString` and friends returned inherited functions that every consumer then treated as a block, turning a path segment into a 500. `getToolMetadata` already guarded this way. - A folder-scoped log page no longer unions in every job run in the workspace. The guard read `filters.folderIds`, which the public surface never sets — it carries the folder filter in `folderScope` — so the page contradicted the contract's promise that job runs are dropped whenever a filter they cannot answer is set. Also: the log cursor stamps `includeJobRuns` only when it is on, so its `.default(false)` no longer puts a constant in every fingerprint and rejects cursors minted before it existed; and the `folderName` subquery is scoped to the workspace and to workflow folders instead of scanning the whole `folder` table. * fix(api): close two more review findings, one an authorization bypass - `workflows.operations.apply` no longer admits a workspace API key. The use case authorizes against three per-user policies — the EE permission config, block visibility, and credential reachability — and all three take a human subject. An actorless key has none, and both substitutes fail open: attributing to the workspace billing owner evaluates the batch as the least-restricted account in the workspace, and passing no user makes `getUserPermissionConfig` return `null`, which every caller reads as unrestricted. Either way a workspace constrained by an allowlist was edited as though it were not. Personal keys keep the capability, so headless editing is unaffected for a credential that names a human. - `GET /workflows/{id}/state` reads its variables through `parseWorkflowVariables`, and the stored variable response schema drops the two assertions the column cannot honour. The column has carried a JSON string and a legacy array as well as the current record, the realtime `variable.add` op types `type` as `z.any()`, and the parser writes `name` through verbatim — so the input bounds on the read turned a stored workflow into a 500 on the endpoint that opens it. The write schema keeps them, which is where they can still be honoured. - `GET /workflows?scope=archived` projects folder paths tolerantly. Archiving a folder cascades onto the workflows inside it but leaves their `folderId` dangling — which is why restore has to null it — so the strict projector threw a bare `Error` and took the whole page down with no cursor able to step past the row. * fix(files): bind Start-block file keys to the executing workspace The Start block derived a file's storage key by parsing the caller's own `url`, which `isInternalFileUrl` matches on any host and `extractStorageKey` returns verbatim — so a request body could name any tenant's bytes. The key is now accepted only when its own layout names the workspace the execution runs in, and every file is dropped when the execution carries no workspace. Also bounds `includeFileBase64` with an aggregate response ceiling and a worker pool instead of an unbounded `Promise.all`, scopes the bulk download's authorization resource to the workspace when folder paths are requested, makes the folder-restore selector mutually exclusive at the type level, and names the bound in the `maxBytes` validation message. * fix(api): close v2 log review findings Cursor scope: `scope` on the workflow and table lists carries `.default('active')`, so it entered every fingerprint as a constant and refused every cursor minted before the param existed — with the "cursor does not match the requested filters" 400, which is actively misleading for a caller that changed nothing. Both now stamp the default as absent, so only a caller who asked for `archived` gets a new sequence. Dashboard stats: `maxWorkflows` capped the response, not the allocation. Segment series are now densified after the cut instead of before, so returning 200 series no longer materializes one `segmentCount`-length array per workflow in the window. The aggregate still sums every workflow, now from the sparse per-workflow maps. Cost keyset: `cost_total` is an unconstrained `numeric`, so its anchor travelled through `Number()` and was compared back at full precision — rows differing beyond float64 collapsed onto one anchor. Adds `decimalKey`, which carries the digit string and binds it `::numeric`. Run detail: `cost_total` is a backfilled projection, so a run predating the backfill reported `cost: null` even with a real ledger, making `items` unreachable for exactly the runs the ledger explains. Falls back to the ledger total. Also caps the log folder-path index reads at MAX_FOLDERS_PER_WORKSPACE like every other reader, publishing the folder-tree 413 on the four log operations; reverts a dead `status` widening in `v2CommaListSchema`; drops an unread `executionData` select; corrects the segment-count and searchLogs prose; and replaces the sort-cursor SQL-text assertions with a two-page walk over a fixture with a null block. * fix(api): close knowledge v2 review findings - widen knowledge.list_archived to the delete/restore policy so a workspace API key can discover what it may restore - escape LIKE wildcards on the now-public chunk search - derive tag slot capacity from TAG_SLOT_CONFIG per field type - type updateKnowledgeBase's chunkingConfig as ChunkingConfig and project every declared field explicitly - attribute a restore to the calling surface instead of a literal 'api' - gate 'knowledge chunks batch-update' behind --yes, since it can delete - present the tag-cleanup action from the parsed request rather than faulting on the domain result after the delete committed - unbind asserted-scope workspaceId from the nested knowledge cursors, matching the table-row lists - add executed-SQL coverage for the chunk keyset * fix(catalog): close the catalog and registry-boundary review findings The module-graph ratchet treated an entry with no baseline row as informational, so the six catalog routes and the projection barrel were unratcheted while the summary still read "within their module-count baseline". An unbaselined entry now fails --check, the summary counts only what was actually compared, and the baseline is re-recorded. The Copilot block-metadata tool — the reason the shared projection exists, 6,756 modules down to 1,321 — was in no guarded subtree. It is now an entry source and a catalog boundary root. Catalog behaviour: - hostedApiKey is gated on the deployment, so a self-hosted install reports none instead of promising 127 tools' keys it will never supply - block detail resolves an unversioned base type to its newest version and projects through the viewer's visibility, so it can no longer 404 a block the list contains or name it differently - offset-cursor ordering compares code units rather than the process locale - projections copy every array they publish instead of handing out the registries' own - an options function returning a thenable throws rather than silently widening the providers-store substitution across the event loop - a throwing block projection costs the Copilot tool one block, not all of them - the trigger-kind log returns to debug: chat/manual/api are entry-point kinds, not authoring defects Also sweeps the custom-block detail branch against its response schema, guards each projection module rather than the dead barrel over them, and drops a provably dead branch in processHostedKeyCost. * fix(workflows): close v2 workflow-authoring review findings - Read a blockless draft back as an empty graph. `PUT /state` of `{ blocks: {}, edges: [] }` — the contract's own published example — deletes every block row, and the loader answers `null` for a blockless workflow, so the following `GET /state` answered 404 while the list endpoint still showed the workflow. Existence is the workflow row's to decide; the null is now projected as an empty graph. - Rewrite the `readWorkflowGraph` authorization test so it can fail. It called `authorize?.()` and asserted only a negative, so deleting `authorize` or replacing it with a no-op both passed — the invariant the head-safe `HEAD` path depends on. - Route `setWorkflowBlockEnabled` through `replaceWorkflowNormalizedState`, the same door the other two graph writes use, instead of writing the normalized tables itself without state preparation or custom-tool extraction. - Count applied operations directly. Enablement refusals landed in the same skipped-item array and were subtracted from the operation count, which `Math.max(applied, 0)` then masked when it went negative. - Give a `disabled_ancestor` refusal its own member of the published skip enum instead of reporting it as `block_locked`. - Refuse an `atomic` batch whose credential or hosted API key would be stripped, and carry the dropped inputs in the 409 details. - Publish the whole lint report — `sources`, `sinks`, `orphanBlocks`, `emptyOutgoingPorts`, `invalidBranchPorts`, `invalidConnectionTargets`, `fieldIssues`, and the `kind` discriminator on unresolved references — rather than only free-text reference prose. - Stop reporting unresolved lint references as `inputValidationErrors`. `collectUnresolvedReferences` is read-only, so those values stay persisted; they were double-reported, and falsely as dropped inputs. - Replace two unfalsifiable negative-principal tests, which used a principal kind `Exclude`d from `PrincipalKind`, with a reachable one. - Nits: drop a stranded TSDoc block; assert the membership predicate in the selector-validator admin test; make the HEAD test assert a representation; assert the sanitized graph is what `replaceWorkflowState` writes; add a route test rejecting `baseGraph` in a v2 body; carry `principalAuditSource` on restore/duplicate/moveBulk audit; unify the two `base64MaxBytes` ceilings on `MAX_INLINE_MATERIALIZATION_BYTES`. * fix(api): close seven v2 review findings, one an authorization bypass Raise mcp_servers.workflow_deployments.update_server to admin: its body carries isPublic, and a public server executes with no Sim credential, so a write member could remove authentication from every workflow it publishes. create_server already grants the same visibility at admin. Pin the widened operations in registry tests — the six workflow-MCP ones, the four workflow widenings, and files.extract_archive. Reject secret fields on a credential that has no rotatable secret instead of dropping them behind a 200, classify a non-transient provider 4xx as caller error rather than a retryable outage, and reconcile a provider outage to 503 with Retry-After on all three surfaces. Give /v2/meta a declarative principal policy through a new defineOperation factory, carry the key expiry on the auth context instead of reading the api_key table from the application layer, and make the impossible principal branch an invariant error rather than a codeless 403. Enforce the rollout-gate exemption at definition time, against the one contract path it is reserved for, and remove the gate parameter from the exported admission helper so the builder is its only door. * fix(tables): bound the run-state sidecar, and close six v2 review findings Enforces the 2 MiB run-state ceiling INSIDE the sidecar drain rather than over its materialized result, refuses the unbounded query form paired with it, and normalizes the two stored blobs the v2 surface publishes from bare `as` casts. - The run-state budget now travels into `loadExecutionsByRow`, which drains row ids in bounded chunks and refuses before fetching the next one. The post-hoc `requireBoundedRunState` walk is gone: it measured a spike that had already happened, and re-serialized every entry to do it. - Both row reads that accept `includeRunState` cap the page at `V2_MAX_RUN_STATE_ROW_LIMIT`, and `POST /tables/{id}/query` additionally refuses the flag paired with `limit: 0`. - `runState.status` and the enrichment cascade blob are projected onto the published shape before presentation; both were caller-reachable 500s on a well-formed read. - `sim tables bulk-delete` now gates behind `--yes`, and the CLI sweep that should have caught it covers destructive non-DELETE forms. - `POST /tables/{id}/restore` is idempotent (200, no audit) like its knowledge sibling, and bulk folder selection deduplicates after resolution. - The batch-update backstop keeps the looser Copilot ceiling and says so in TSDoc: it is a backstop no surface reaches, because the contracts stop a v2 caller at 1000 and the Copilot tool stops itself at 5000. Each caller sees the bound that actually applies to it; neither surface's cap moved. * fix(chat-deployments): close v2 chat review findings Fixes the chat-deployments slice of the v2 review, several of which are regressions the application-operation extraction introduced. - Stop a `500` on schemaless JSONB: the response now declares a stored shape without bounds and `toV2ChatDeployment` projects `customizations`, `outputConfigs`, and `allowedEmails` onto it. The request schemas keep `.strict()` and their bounds. - Restore the specific validation message on `POST /api/chat` and `PATCH /api/chat/manage/[id]`, and the deleted test that pinned it. - Restore `chat_deployments.read` to workspace `admin`; the detail read serves the visitor gate. - Narrow the list projection so `chat_deployments.list` can stay a `read` operation reachable by a workspace API key: `allowedEmails`, `hasPassword`, and `customizations` are gone from the list entry and available only from the admin-gated detail read. Serialized field by field so a field added to the detail shape cannot reach the list by default. - Classify create-path failures: `performChatDeploy` carries an `errorCode`, so an in-flight deployment is a `409` and an invariant failure a `500` instead of every refusal being a `400`. - Delete the callerless `GET /api/chat`, which served the encrypted password column with no response contract. - Propagate undeploy infrastructure failures instead of concealing them as `404`, and return `ChatDeploymentView` from both delete paths. - Name `CHAT_AUTH_MODE_NOT_PERMITTED` on the create path. - Guard `getBaseUrl` inside `buildChatDeploymentUrl`, which otherwise throws on a self-host with no `NEXT_PUBLIC_APP_URL`. - Correct the published allow-list claim: a replacement `allowedEmails` is applied after the auth-type clear, so it does survive. - Assert `workspaceId` on the v2 detail routes and reconcile the concealment TSDoc with what the error policy actually renders. - Move `resolveActiveWorkspaceApplicationContext` to the workspaces domain so chat-deployments no longer imports workflow application code. * test(credentials): pin the reconciled provider-outage status The internal route alone answered 502 where the v2 surface, the shared status helper and `PROVIDER_OUTAGE_CODES`' own TSDoc all say 503. The test pinned the divergence; it now pins the reconciliation, including the `Retry-After` a 503 carries. Corrects a stale comment that still named 502 as the value callers see. * fix(executor): restore cloud-storage Start files, dropped by the key rule The ownership check accepted a key only when it could be parsed out of an internal `/api/files/serve/...` URL. But the server-side uploader for run inputs returns a *presigned cloud* URL whenever object storage is configured, whose path is the bucket key — so every chat-deployment attachment, API `files[]` payload and generic-webhook file field resolved no key, and because normalization is all-or-nothing the entire `files` input was dropped with no error. It passed locally and under vitest only because the uploader falls back to an internal URL when no object storage is configured, which is exactly why no test caught it. The test is ownership, not provenance: a key is accepted when its own layout names the executing workspace, whether it arrives directly or is parsed out of the URL. Neither field has to be trusted, since both are caller-authored and both are held to the same check. A payload whose key and URL disagree is refused rather than resolved in the caller's favour — a genuine uploader writes the two consistently, so only a forged pairing is turned away. `context` is now derived from the accepted key rather than read from the payload or the URL's `?context=`, so an owned key can no longer be labelled with a bucket its bytes do not live in — the hardening the previous comment claimed but did not perform. * fix(api): close four defects the fix pass introduced - `resolveLatest` built a `RegExp` from the caller's block id and read the registry with a bare lookup, so the catalog detail route — moved onto it by the version-alias fix — routed around the `ownBlock` guard added for exactly this. `GET /api/v2/blocks/%5B` was a `SyntaxError` 500 and `.../constructor` an inherited function. Matched by string comparison now, the way `tools/tool-ids.ts` resolves the same convention, and read through `ownBlock`. - The run-state byte budget was applied inside `queryRows` rather than at the callers that publish it, so the first-party table grid — which reads run state at five times the row limit and publishes no ceiling — turned a large page into a hard failure, with an error naming a parameter it does not expose. The budget is now an explicit option the public reads pass and internal callers omit. - Three graph-write CLI commands shipped ungated because the sweep meant to catch them matched only the names already enumerated, so it could never fail. It now forces every non-`GET` operation into a destructive or non-destructive list, and the three carry confirmations. - Unbinding `workspaceId` from the knowledge-documents cursor was right on the merits and wrong in effect: the value is constant per sequence, so removing it changed the fingerprint and refused every cursor already in flight. Restored there; the chunks list is new in the same change and keeps the cleaner reading. * fix(api): resolve a detail read to a version the viewer can see `getLatestBlockForViewer` took the newest version and then hid it, which inverted the contradiction it was written to close: `slack_v2` and `table_v2` are preview-gated while their v1 deliberately stays in the toolbar, so an unrevealed viewer got a `404` on a detail read for a type `GET /api/v2/blocks` was listing in the same breath. It now walks versions newest-first and answers with the first one visible to that viewer. Also: - The chat password guard ran after `performFullDeploy`, so a request that could never succeed burned a real workflow deployment version and then answered 400. Its two sibling gate guards already refuse ahead of the deploy; this one now does too. - The Copilot sub-block serializer published the registry's own `options` and `dependsOn` arrays by reference. Pre-existing, but the catalog projection this parallels copies every array it publishes precisely because they are process-global and shared by every request. * fix(api): restore the locked read-modify-write and the password validator Two findings verified as real regressions against staging, out of ten checked — the rest were pre-existing, latent, or false. `setWorkflowBlockEnabled` read the graph outside the row lock and wrote it back inside a later transaction. The editor's own save takes that same lock, so an autosave committing in the window was silently discarded: this operation writes a whole graph, not a delta. The persistence primitive now accepts a reader that runs after the lock is taken, and the toggle re-reads and re-decides there. Its lock predicate is also scoped to the workspace and to a live row again, so a workflow archived mid-flight is refused rather than written. The v2 chat-deployment contracts inlined their own password rule twice instead of using `chatDeploymentPasswordSchema`, losing the refusal of a whitespace-only password — which the internal contract rejects precisely because it strands the deployment behind a password the visitor form will not submit. Both sites use the canonical validator now. * fix(api): one folder projection, one dynamic-provider list, honest 413s - `toV2Folder` existed twice, and the second copy had been written without the name/path invariant — so a row the list read refuses loudly would have been served with a mismatched pair by the restore read. One definition, guard included. - The catalog projection restated `DYNAMIC_MODEL_PROVIDERS` and had drifted by one member. Derived from the canonical list instead. - The tables reads documented a `413` for run state that they cannot emit — the budget became opt-in, and the row limit is the bound now — so the claim is removed rather than declared. The workflow run read has the opposite problem: it genuinely emits one, on a single file *or* the run's inlined total, and declared neither. Now declared, and the sentence covers both. - Reclassifies the operations staging added into the destructive sweep, so the triage stays exhaustive. * fix(v2): classify storage and uniqueness failures, drop permanent file delete - remove the permanent file-delete endpoint; the platform offers no such action in the UI, and its manager wrote outside a transaction with no storage accounting - extract a generated document's text from its compiled artifact rather than its generation source, matching the download path; a `.pdf` source was a 500 and a `.docx` source returned generator JavaScript as clean content - report a run file whose object retention has already swept as 404 rather than 500, on both the inline base64 read and the download stream - report a knowledge tag that loses at a unique index as 409, naming whether the slot or the display name is taken - gate `workflows versions revert` behind a CLI confirm; it overwrites the draft graph and was classified non-destructive * feat(v2): report lint from both graph writes and add dry-run previews - `PUT /workflows/{id}/state` now returns the same `lint` report as `POST /operations`; an agent authoring a graph from scratch needs the findings at least as much as one editing incrementally - extract the report into one shared builder so the two writes cannot drift, and one shared presenter so the wire shape is identical - skip the credential/tool reference pass when the caller has no human subject, rather than resolving it against the workspace billing owner: that would misreport what the workflow can reach and disclose another person's grants. `lint.notes` says when it was skipped - add `?dryRun=true` to both graph writes: validates and lints, persists nothing, records no audit, notifies nobody. A query param, not a body field, since the body of a PUT is the resource itself - CLI: a dry run no longer demands `--yes`; requiring confirmation to preview a change teaches callers to pass `--yes` reflexively - CLI: name the graph commands for their verbs — `workflows state get`, `workflows state replace`, `workflows operations apply` — instead of the derived `state list` / `state update` / `operations create` - document when to use `rollback` vs `versions/{version}/activate` on both * chore(docs): sync generated docs manifest for the new CLI pages * feat(v2): add the missing workflow-MCP reads and align bulk naming - add `GET /workflow-mcp-servers/{serverId}` and `GET /workflow-mcp-servers/{serverId}/tools`. The resource could be PATCHed and DELETEd but never read, and its tools could be published and unpublished but never listed — the server list reports tool names only, so nothing published the `workflowId` that addresses a tool for deletion. Both mirror `mcp-servers` beside them, and carry that family's workspace-API-key denial rather than the wider `mcp_servers.read` policy - rename `POST /tables/bulk-move` to `POST /tables/move`, so tables matches the shipped `files` resource exactly (`move` + `bulk-delete`) - name the CLI commands for their operations instead of the derived `... create`: `tables move`, `workflows move`, `tables bulk-delete`, and `tables rows update-each` for the per-row batch, which sits beside the existing filter-based `tables rows batch-update` `POST /tables/{id}/rows/batch-update` keeps its name: a distinct payload per resource is precisely AIP-234 BatchUpdate, and `bulk-` would have collided one word away from the filter form. * fix(v2): correct documented statuses and a caller-reachable 500 - duplicating a workflow into a locked destination folder answered 500: `FolderLockedError` is a plain Error carrying `status = 423`, which the v2 error policy does not classify. Converted to OrchestrationError('locked') at the application boundary, matching the bulk-move path - restore workflow promised a 413 for an oversized folder tree that its response list never published; the cap is real, so the status now is too - move workflows and apply variables documented 409/423 they cannot emit: every per-item lock and conflict is reported in `failed`, not thrown - bulk download and delete knowledge tag can both 409 and did not say so; cleanup tag definitions cannot and did say so - apply workflow operations denies workspace API keys but never documented it - the dry-run responses are not byte-identical to a committed write: `needsRedeployment` describes the pre-write state and persistence warnings cannot appear. Reworded rather than overclaimed - read file text and get file upload are head-safe, so the "HEAD skips the effect" sentence did not apply to them * fix(v2): guard tag field-type changes and publish the 415 every body route can return - `PATCH /knowledge/{id}/tags/{tagId}` accepted a `fieldType` incompatible with the slot the tag already occupies. Slots are enumerated per field type, so a text tag could be relabelled `number` and every later read would interpret its values as the wrong type. Create checked this; update now runs the same two checks - derive `415` from the contract the way `413` already is: the JSON builder answers UNSUPPORTED_MEDIA_TYPE for any body under a content type it cannot read, so all 100 body routes could return a status none of them published - give version activation its own result component instead of publishing it as `RollbackResult`; the shipped rollback keeps that name - correct descriptions that promised behaviour the code does not have: a `processingStatus` field never returned, a `gmail_send` resolution example that short-circuits, bucket widths that overflow the window, a bulk tag save that relocates rather than overwrites, and per-server tool names actually gathered under a page-wide budget - drop 409 from three knowledge and upload reads that cannot emit it * docs(v2): correct the upload transfer contract and 16 other published claims The upload transfer step was documented as Sim's own data plane on every deployment: "success is 204" and "a failure is the v2 error envelope". That holds only when Sim stores objects itself. With object storage configured the URL is the provider's presigned URL, so S3 and GCS answer 200 and Azure 201, and a failure is the provider's XML — a client written to the old text reads a successful cloud upload as a failure. Also states that part ETags do not need retaining: completion takes no body because Sim lists the parts from the provider itself. Other corrections, all to shipped descriptions rather than behaviour: - DELETE table and bulk-delete files archive rather than erase, and neither said so; bulk delete also cannot emit the 409 it declared - complete knowledge upload published a 402 only the create leg can raise - billing status conceals a foreign workspace id as 404, not the 403 its TSDoc and description both claimed - audit entries null a folder's resourceId and strip folder ids from metadata at every level; neither redaction was documented - details=full adds the workflow summary to workflow runs only, never to job runs; GET /logs folderPaths covers a subtree like its two siblings; getLog now carries the retention sentence - list secrets returns description too, and the logs and resources documents described only part of what they serve * improvement(api): consolidate the v2 surface and close seven defects Endpoint consolidation: - Fold POST /logs/query into GET /logs; add sortBy/sortOrder, cap the comma lists, and move the list onto the shared keyset codec - Fold GET /knowledge/archived into GET /knowledge?scope=archived, matching files, tables, and workflows - Re-home chat deployments as a singleton under the workflow they belong to; keep the workspace-scoped discovery list - Move the tag-definition writes off the document path onto /knowledge/{id}/tags, where they already acted - Nest the table export and dispatch reads under their parent table Defects: - Publish isPublicApi on the deployment read; it was write-only, so a workflow could be opened to unauthenticated execution unauditably - Stop publishing raw storage keys and an unusable URL in log files - Classify a chat-identifier unique violation as 409 rather than 500 - Fall back to the root path instead of throwing when a knowledge base's folder is archived - Cap bulk-download at the ceiling it actually enforces - Normalize variables through one helper on both graph write paths - Fix a folder-name log filter that matched workflow names Naming and gaps: - Rename /files/{id}/extract to /unarchive, /rows/find to /rows/search, /rows/batch-update to /rows/bulk-update, /columns/run to /dispatches - Type the last six generic [id] path segments - Add table folder restore and id-addressed dispatch cancel * chore(audits): record the v2 catalog routes in the boundary baseline * fix(api): accept a null chat password and correct three published claims - performChatDeploy validated `password: null` as a password, so the replace-shaped chat PUT answered 400 for every mode that owns no password — public (the default), email, and sso. The declared payload type has always allowed null, and the stored value is cleared by authType regardless, so null needs no validation of its own. The route test could not catch it: it mocks the orchestration module and pinned the exact null the real guard refused. - Redirect the two docs slugs this branch retired that were genuinely published: findTableRows and runTableColumns. - The table folder restore described an idempotent no-op for an already active folder; it answers 404. Say so, and say where the path comes from, since the tables folder list cannot yet report archived folders. - Name the customizations exception to the chat PUT's replace semantics. - A cursor-binding case used status=error, which is a level and not a status, so it failed contract validation and never reached the cursor check. Use an accepted value and pin the reason, not just the status. * chore(cli): classify the new v2 chat operation as non-destructive * feat(cli): expose canonical resource URLs * fix(api): close three caller-reachable failures found by the final probe - GET /files/{fileId}/text called parseBuffer unguarded, and parseBuffer signals every failure as a bare Error that no v2 policy classifies. A zero-byte upload or a mislabelled archive was an unhandled 500. Empty bytes now answer empty text — a zero-length file has no text — and unparseable bytes answer 409, matching the rendered-artifact resolver. - GET /workflows/{workflowId}/state asserted write-side bounds over stored data. workflow_blocks.name and .type are bare text() and the realtime rename op accepts z.string(), so a block renamed past 255 characters made the workflow unreadable, and unrepairable, over v2. The read shape now takes the same input/stored split the variable schema already had. Stored subflow conditions are coerced in the loader beside the existing numeric guards. - GET /knowledge stamped scope into the cursor fingerprint unconditionally. scope defaults to active and is new on that list, so every cursor the deployed build handed out would have been refused with a message saying the caller changed a filter they never sent. Its siblings already carry the guard and the comment. Also: POST /workflow-mcp-servers answers 201 like every other v2 create; GET /logs/stats reuses the log list's entry ceilings; the execute and resume routes install the media-type-aware 415 they publish; and a rationale citing an endpoint that never reached the wire is corrected. * fix(tables): carry the dispatch terminal timestamps through the stale sweep Staging's abandoned-dispatch recovery builds its own `DispatchRow`, and this branch had added `completedAt`/`cancelledAt` to that shape for the id-addressed dispatch read and cancel. The merge was textually clean and left the new mapping short two fields. * fix(workflows): deny workspace API keys on the graph replace `PUT /workflows/{workflowId}/state` stores blocks and their tool wiring wholesale, but the policies deciding which of those a member may add — the EE permission config and block visibility — take a human subject. A workspace API key has none, and both substitutes fail open: the billing owner is a different, typically less-constrained person, and passing no user makes the permission lookup return null, which every caller reads as unrestricted. That made the replace a second graph-write door storing what its sibling `POST /workflows/{workflowId}/operations` refuses, which denies workspace keys for exactly this reason. Both doors now agree. Personal keys keep the capability, so headless authoring is unaffected for a credential that names a human. * chore(api): drop the enrichment catalog endpoint GET /api/v2/enrichments listed the code-defined table enrichments. The per-row enrichment run detail stays; only the catalog read goes. Removes the route, contract, response and query schemas, the semantic operation, the use case, the projection module, its registry-boundary entry, and the CLI command. The "not found" and "blank search" cases it covered are repointed at the connector-type sibling so the shared behaviour stays tested rather than deleted with it. * improvement(api): make the workflow operations endpoint self-describing Two things stood between this endpoint and a caller who has only the published spec. The accepted `params` keys existed only in the edit engine's source. The spec said "the accepted keys depend on the target block type", so a caller reading it could create a nameless empty block and nothing more — while the Copilot tool catalog, over the same engine, has always spelled out the envelope. That guidance now lives in the contract, shared by the add, edit, and insert_into_subflow parameter schemas so the two surfaces cannot describe one engine differently: `inputs` keyed by sub-block id, `retry`/`triggerMode`/`advancedMode` beside it rather than inside it, `connections` keyed by source handle, and `removeEdges` for dropping one edge without restating the rest. A `block_id` that is not already a UUID is replaced with a minted one, and the mapping was computed and then dropped. A caller could not reference the block it had just created except by re-reading the graph and matching on name. The engine now returns it and the response publishes it as `mintedBlockIds`, with the in-batch versus cross-request rule stated in the operation description. * fix(api): close the pre-merge scan's blocking findings Docs and public wire, all of it permanent surface once released. - Two published tag groups, Catalog and Meta, had no sidebar entry in any locale, so six operations shipped unbrowsable. Added to all six, and a test now fails when a published tag has no entry. - deleteWorkflowChatDeployment pointed callers at DELETE on /deployment; the undeploy verb is on /deploy. - PUT /state still promised workspace API keys a degraded lint pass after the operation began rejecting them outright. It also lived in a single-quoted string, so the shared clause would not have interpolated. - POST /tables/move took targetFolderPath as nullable-but-required, which rendered the CLI flag as a required `<json|@file>`: `--to /Archive` failed to parse and omitting it failed outright, while the docs said "omit for root". Now optional and a plain string, matching POST /files/move; the route supplies the null the use case wants. - Table dispatch status and row run state published `cancelled` beside imports, exports, and job state publishing `canceled`, and the note explaining the split was wrong about its own sibling. Both new schemas now publish `canceled`; the stored column is unchanged and mapped at the presenter. The shipped `cancelled` count field is left alone. - applyWorkflowVariables can answer 423 and both workflow-MCP deletes can answer 409; none declared it. Hand-assembled error lists replaced with the shared sets. - /files/{fileId}/unarchive became /unzip. `extract` reads as "extract text" and `unarchive` reads as the inverse of restore on a resource where archived means soft-deleted; unzip collides with neither and is what the implementation calls itself. * improvement(cli,docs): finish the naming and clear the enrichment leftovers - Four single-record GETs derived to `list` while returning one thing: meta, a workflow's chat deployment, log stats, and file text. Renamed to `meta status`, `workflows chat status`, `logs stats`, `files read`, matching the `workflows deployment status` correction already in this branch. None of the old spellings shipped. - Dropped five `renamedFrom` aliases pointing at spellings that never existed, each of which built a hidden command and a permanent deprecation warning for argv nobody could have typed. `tables rows find` keeps its alias — that one really shipped. - Removed the enrichment catalog from three prose sites left behind when the endpoint went: the resources spec description, its Catalog tag, and the contract and pagination-test comments. - Documented the six new command groups in the CLI index table. That page is a guide page, so the docs staleness check cannot flag it. - Published the `customizations` exception to the chat replace semantics. It was in the route TSDoc and invisible to every caller reading the spec, which is where the claim "Replace, not merge" is made. * improvement(cli): use batch- for the tables bulk delete, matching its siblings The CLI renames a bulk form only when it would collide with its singular sibling, and uses AWS's `batch-` prefix when it does — `files batch-delete`, `tables rows batch-delete`, `knowledge chunks batch-update`. `tables delete` exists, so `tables bulk-delete` was that same rename reaching for the other word, and the only `bulk-` command on the surface. No `bulk-` CLI command has ever shipped, so this costs nothing now and would be permanent later. `files bulk-download` keeps its name: there is no `files download` to collide with, and it is one archive rather than N operations. Its config block now says why it exists at all, since the command is never built — the builder skips non-JSON response modes, but the contract sweeps still read the entry and require the folder-path field to be marked. * fix(workflows): allow operations on blockless drafts * feat(workflows): add manual and run-from-block execution * fix(cli): update workflow run description test * fix(tests): align fixtures with current contracts * fix(api): derive chat activity from workflow deployment --------- Co-authored-by: Theodore Li <theo@sim.ai>	2026-08-25 04:19:34 -04:00
3 people	42f6287911	feat(byok): add organization-wide key inheritance (#6834) ... * feat(byok): add organization key management * feat(byok): inherit organization keys at runtime * feat(byok): add organization scope to BYOK settings * fix(byok): refresh org key state after mutations * fix(byok): hide stale inherited status badges * chore(db): drop colliding byok migration ahead of staging merge Staging independently claimed 0293. Remove ours so the merge is clean; it is regenerated at the next free index right after. * chore(db): regenerate byok migration at 0296 Staging claimed 0293-0295 during the merge; the regenerated SQL is byte-identical to the dropped 0293. * docs(byok): document organization scope, precedence, and the full provider list The BYOK section described workspace-scoped keys only. Add the organization scope, its Enterprise requirement, the per-provider precedence rule, what an entitlement lapse does, and the Pi sandbox exposure. Refresh the provider table from the settings page, which had drifted from 14 to 34 entries. * feat(byok): open organization keys to every organization plan Organization BYOK was gated on Enterprise, but an organization is the only thing that can hold the keys, so every plan that can own an organization should qualify — Pro for Teams, Max for Teams, and Enterprise. Add checkOrgPlan/resolveOrganizationPlan beside the Enterprise pair rather than widening checkEnterprisePlan, so the Enterprise-only gates (Access Control, whitelabeling) are untouched, and restore resolveOrganizationEnterprisePlan to module-private now that BYOK no longer needs it. * perf(byok): cache the organization entitlement, not the key material getBYOKKey runs once per agent block and once per hosted-capable tool call, so a loop over N items resolved N times — and each organization-inheriting resolution paid three sequential billing queries on top of the two key reads. Split the two reads by staleness tolerance. Key rows stay fresh, because revocation must be immediate. The entitlement is a billing gate that tolerates bounded staleness in the harmless direction (a lapsed organization keeps using its own key for <=60s), so cache it per organization with an in-flight share so concurrent blocks issue one query set. The management surfaces keep reading it fresh, so an organization that just upgraded is never told otherwise. Also run the block check and subscription read in parallel inside resolveOrganizationPlan, and carry the resolved scope on BYOKKeyResult so a log line can say whether a run used the workspace's key or an inherited one. * feat(byok): let workspaces store the Z.ai and Cohere keys the runtime reads Both ids were already in the BYOK contract enum and both are resolved at execution time — getApiKeyWithBYOK reaches 'zai' (GLM models are in the hosted catalog, so the BYOK branch runs), and 'cohere' backs both the Embeddings block and Knowledge Base reranking — but neither appeared in the settings list, so there was no way to store the key either path looks for. Cohere had no icon; add one from the official multi-color mark so it stays legible on a light and a dark page. Cohere's embed-v4.0 is kbEligible:false, so the description says 'Embeddings and Knowledge Base reranking' rather than claiming KB embeddings. * improvement(byok): shorten the workspace scope chip to 'Workspace' It sits beside 'Organization', so the scope reads from the pair; 'This' only added width. * fix(byok): do not cache a billing outage as an unentitled organization resolveOrganizationPlan maps a failed billing read to false, which is indistinguishable from a real plan lapse. The entitlement cache stored that, so one transient outage held the gate shut for the full TTL and every inheriting run silently fell back to a metered hosted key — and the cache's rejection path, which exists to prevent exactly this, was unreachable. Give the resolver the onError option its neighbours already have and let the cached read ask for 'throw', so a failure stays out of the cache and the next resolution retries. Behavior for the call that saw the error is unchanged: getBYOKKey still fails closed. Reported by Cursor Bugbot. * fix(byok): propagate the subscription read's failure too The previous commit threaded onError through resolveOrganizationPlan's own catch, but getOrganizationSubscriptionUsable soft-fails to null on its own, so a failed subscription read still arrived as an ordinary 'no usable subscription' and returned a successful false — which the entitlement cache then stored for the full TTL. Thread the option into that call as well. Test it at the billing layer rather than the cache layer: the entitlement test mocks resolveOrganizationPlan wholesale, so it could never have caught this. Verified the new test fails against the previous commit. Reported by Cursor Bugbot. * refactor(byok): cache the entitlement with LRUCache, like copilot entitlements The hand-rolled version reinvented three things the codebase already has a canonical answer for. lru-cache is a declared dependency of apps/sim and lib/copilot/entitlements.ts already caches an entitlement with it — by storing the in-flight Promise, which is what makes concurrent callers collapse onto one resolution with no in-flight bookkeeping at all. TTL and the size bound come from the library. That removes the second Map, the manual eviction (and its interaction with an in-flight entry), and the dead value-while-refreshing state: 23 executable lines. The one thing the library does not cover is dropping a rejected promise so a billing outage is not cached for the TTL, which is kept and pinned by a test that fails without it. TTL expiry is no longer re-tested — that is the library's behavior, not ours, and lru-cache reads its clock at module load so faking timers never moved it. * refactor(byok): coalesce the entitlement read with the shared singleflight lib/concurrency/singleflight.ts is the codebase's coalescing primitive and oauth/credential-service.ts already pairs it with a read-through cache. Adopting that shape fixes a case caching the promise directly did not: a *hung* billing read wedged every caller for the full 60s TTL, where coalesceLocally evicts and rejects at its settle deadline. It also removes the hand-rolled rejection eviction — the cache is written only on the success path, so an outage leaves no entry by construction. The cache now holds booleans, which introduces the one trap worth a test: a truthiness check would read a cached false as a miss and re-query billing on every resolution for lapsed organizations. Pinned. * fix(byok): keep an abandoned entitlement producer from writing the cache coalesceLocally does not cancel a producer it timed out — its docstring says so explicitly — so writing the cache from inside the producer let a late billing result overwrite a fresher answer a retry had already cached, and hold it for a full TTL. Move the write onto the value the caller actually received. A caller that timed out throws before reaching it, so an abandoned producer now resolves into nothing. The test reproduces the overwrite and fails against the previous shape. Reported by Cursor Bugbot. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>	2026-08-20 17:45:28 -07:00
Waleed	4f9d5f33b0	improvement(search): search every folder, and document real API error bodies (#6861) ... * improvement(search): search every folder, and document real API error bodies Search on Files, Tables, and Knowledge was ANDed with the open folder, so a query only ever matched that folder's direct children — and the query was not cleared when you entered a folder, filtering the folder you just opened down to the same matches. A non-empty query now searches the whole workspace, a Location column names each result's folder, and opening a folder ends the search. Also gives GET /api/v2/files a `recursive` flag, and replaces the single shared OpenAPI error example — which showed `BAD_REQUEST` under every status tab — with one real body per status. * fix(search): discard the search term on clear instead of masking it `useSearchFilterValue` returned the debounced term whenever the input was non-empty, so clearing only hid the settled needle. The mask lifted on the next keystroke while the debounce still held the pre-clear term — opening a folder and typing within the window searched the whole workspace for the query the user had just abandoned. A clear now resets the settled term rather than hiding it, adjusted during render so the reset is visible to the render that follows the clear. The initial state is seeded from the first value so a deep-linked `?search=` still filters on the first render.	2026-08-19 13:55:53 -07:00
Theodore Li	6006870f02	feat(credentials): add v2 credential lifecycle APIs (#6664) ... * feat(credentials): add v2 OAuth connection APIs * fix(credentials): preserve active OAuth connection links * fix(credentials): bind OAuth links to connection intent * feat(credentials): complete v2 credential lifecycle * fix(credentials): make disconnect idempotent * fix(credentials): stabilize oauth draft retries * fix(credentials): bind oauth callbacks to drafts * fix(credentials): fail closed on oauth completion * fix(credentials): bind shopify completion to oauth state * fix(credentials): align custom oauth reconnects * fix(credentials): centralize application authorization * fix(credentials): keep OAuth draft intent immutable * fix(credentials): allow renamed reconnect targets * fix(credentials): close OAuth draft edge cases * fix(credentials): fail closed without breaking auth * fix(credentials): preserve migrated route behavior * feat(credentials): add provider search * fix(credentials): prevent stale secrets and drafts	2026-08-15 00:47:06 -04:00
Waleed	cf78946529	fix(v2): close seven correctness and honesty gaps found sweeping the API (#6702) ... * fix(v2): close seven correctness and honesty gaps found sweeping the API A ten-slice sweep of the live v2 surface turned up no regression from the recent cancellation work, but did surface a set of pre-existing defects where an endpoint either lost data, hid a failure, or reported something that was not true. Each is fixed at the layer that owns the behavior. Terminal execution logs. The two force-fail boundaries wrote `status: 'failed'` without `ended_at` or `total_duration_ms`, so a force-failed run dropped out of every duration-filtered log query — the same defect class already closed for cancellation, still open on its sibling. The cancellation payload factory is generalized to take the status; the cancellation call sites are untouched and still emit a byte-identical row. Custom tools. One malformed row failed the whole page, and because the list is keyset-paginated that row made every page containing it permanently unreachable. The projection now validates against the same contract schema the route builder applies, repairing only what can be repaired without inventing information — a stringified schema, and a missing `type` whose contract admits exactly one value — and omitting with a warning what cannot. Both rows observed in production are recovered rather than discarded. Table filters. `eq`/`ne`/`in`/`nin` compiled a wrongly-typed operand into a containment test that silently matched nothing, so a filter written against the value the write path had stored returned an empty page instead of its rows. The operand is now read through the same column-type registry the write used, and rejected only where that registry refuses it. Range operators already behaved this way; `null` and the cleared-cell sentinel still pass through untouched. Error messages. A custom `error` on a string schema also replaced the wrong-type wording, so supplying a number for a name reported that the name was missing. Messages now distinguish an omitted field from a mistyped one, `topK` names its own bounds, the knowledge search refine reports against a field rather than the whole body, and a workspace id is bounded before it reaches a lookup. Archived file metadata. A soft-deleted file was listed but unreadable, leaving no way to check share state before restoring it. The read takes the same `scope` selector the list already exposes; the default is unchanged, and the parameter relaxes only the `deleted_at` predicate, never the authorization. Cancellation reporting. Cancelling an already-terminal run reported a durable write that never happened. The service now distinguishes the no-op and names the state it observed, and both surfaces present one vocabulary instead of the internal route deriving its own. No claim predicate or write changed. Protocol. A 401 carries a challenge naming the header the API actually reads, and a body that failed to parse is reported as an unsupported media type only when the caller positively declared a non-JSON one — after the read has already failed, so nothing that succeeds today can begin to fail. * fix(v2): correct three regressions this branch introduced, and harden its tests Adversarial review of the previous commit found that three of its "behavior preserving" claims were wrong. Each is corrected here at the layer that owns it. Table filters no longer coerce a `date` operand, and no longer throw. `date` is the one column type whose registry `coerce` is not idempotent — it drops sub-second precision — and the leaf that compiles a filter also builds the unique-constraint and upsert-conflict probes, so re-reading an already-coerced operand could stop it matching the row it was written from and admit a duplicate inside the write transaction with no error. Throwing was the second mistake: the v2 predicate grammar type-checks structure but not operand values, so a rejected operand no longer failed at submission but inside the delete, update, dispatch and cancel runners, where a filter that cannot compile means the cells it started can no longer be cancelled. Coercion is now total — it rewrites what the registry accepts and passes everything else through unchanged, exactly as before. Reviving a force-failed run no longer inherits its terminal duration. Writing `ended_at` and `total_duration_ms` on the force-fail boundary was correct in isolation, but a partial resume flips that row back to `pending` and those columns survived. The preserved value is meant to be the pause checkpoint — the run's active time — and it had become wall clock measured at the failed resume, which the checkpoint rule then faithfully carried into the next terminal write. The revival clears them only for a row that was terminal, so an ordinary paused row keeps the checkpoint it is supposed to keep. Cancelling reports the terminal state it actually observed. Reclassification now requires that nothing else went wrong, so a genuine paused-reconciliation failure survives instead of being rewritten as an already-terminal no-op, and the claim's own row count — not a snapshot read before it — decides whether this cancel terminalized the run or lost a race to something else. The status the snapshot needed rides along on the ownership query that already reads the row, rather than the second read that query's own contract warns against. A custom tool that cannot be projected now answers the same way everywhere: the list omits it, and reading or patching it by id reports it as absent rather than as a server fault. Analytics stops reporting a cancellation for a request that cancelled nothing. The tests around all of this were audited by mutating each fix and checking the suite noticed. Where it did not, the assertion is stronger now: the absent content-type branch is genuinely exercised rather than relying on a header the client library supplies, the duration encoder is pinned to the column it must measure from, execution ownership is pinned to both ids it must match, and the archived-file concealment test proves it conceals the archived read specifically. Two tests that asserted a paused branch they could not observe are gone; the rendered-SQL test that can decide it already covers them. * fix(execution): report a workflow-group cancellation as the write it performed Cancelling a workflow-group run whose log had already been cancelled, but whose cell sidecar still needed reconciliation, durably cancelled that sidecar and then reported `already_cancelled` with `durablyRecorded: false` — because the terminal-status shortcut answered from the entry snapshot alone and never asked what this request had written. The analytics event, which now gates on that field, stopped firing for a cancellation that really happened. The outcome a cancel reports is the same question whichever path answers it, so there is now one vocabulary for it rather than one the direct claim tracked and one the group transition did not. Every group result maps to that outcome through a total map, so a new group result cannot compile without deciding what it wrote, and the reclassification leads with whether this request wrote at all. A group transition that reports itself already cancelled is deliberately mapped as unknown rather than as a no-op: it leaves the sidecar alone but still terminalizes a log that was active, and the result does not say which happened. That costs nothing today, because the only snapshot that would reclassify proves the log was already terminal. * fix(execution): have a workflow-group cancellation report the writes it made Three review findings landed on the same reporting logic, each a different face of one cause: the caller could not see what the group transaction had written, so it inferred. It inferred from an entry snapshot, then from the returned kind, and the remaining blind spot was the kind that covers two different transactions — a repair that terminalizes an active log, and a genuine no-op — which left a cancel that wrote nothing still claiming a durable write when it lost a race. The transaction now reports both writes it can make, each read from that statement's own returning row and recorded immediately before the throw that already depended on it, so the report cannot drift from the write. The caller derives its outcome from those rather than from the kind, and the kind is back to naming the situation instead of standing in for the work. The group path can now always answer whether it wrote. The only remaining unknown is the direct claim when its update throws or is never attempted, which genuinely has no row count to report.	2026-08-14 14:14:30 -07:00
Waleed	ab8a64fcae	fix(v2): close the defects live probing found (#6681) ... * fix(v2): close the defects live probing found Staging finally deployed the merged release, so the surface could be exercised for real. Every fix already shipped held up. These are the defects only live traffic surfaced, plus the ones a static sweep had found and left. A cursor named a position in a sequence without naming the sequence. `cursorScopeKey` hashed only the caller's filters, so any two lists filtering on nothing but `workspaceId` produced one fingerprint and accepted each other's tokens: a tables cursor replayed against the knowledge list answered 200 and silently skipped a row. Table rows never reached that check at all, so a cursor from one table paged another. Identity now comes from the route's own contract — method plus resolved path — because a hand-written name is the step an author forgets, and forgetting it is invisible. An unresolved path placeholder throws rather than fingerprinting the template, so a misconfigured route fails on every request instead of an unlucky one. Every token minted before this is refused with an accurate message; they are single-walk and unpersisted. Knowledge search and the document list answered different questions. Search grouped same-tag filters by slot and joined them with OR while the list conjoined every filter, so `gte 9` and `lte 2` on one tag returned nothing from the list and a full billed page from search. Search now conjoins. The OR grouping replaced an explicit `|OR|` mechanism that was deleted outright, was never documented in any contract, and cost the ability to express a range on a single tag; the union it gave is still reachable as separate searches. The search body also accepted an unbounded query that was billed and then silently truncated to the embedding model's window, and ignored the tag-filter cap the list enforces. A body over ten mebibytes was reported as malformed JSON. Next's proxy truncates there, well under this app's fifty-megabyte ceiling, so the parse failed on a body the caller sent whole and the size branch was unreachable. The ceiling is now clamped to what the proxy will pass. Also: a group's output columns accepted a `workflowGroupId` and discarded it; an enrichment group could never gain an output, because a new output coordinate demanded workflow metadata a group with no workflow cannot have; `newOutputColumns` alone reported success and created nothing; a saved view stored layout references to columns that do not exist while refusing the same name in a filter; an MCP server stored `retries: 0` as three and overrode an explicit auth type; a disabled server answered tool discovery with an unclassified fault; rotating a header server's headers left it reading connected; and a run whose workflow was deleted reported the root folder path while also reporting the workflow deleted. Where the honest fix was out of reach, the contract was corrected instead of half-fixing the code: the polled run resource rebuilds `error.code` by matching the persisted message, so it can never report the two codes that need block attribution, and now says so. `OUTPUT_TOO_LARGE` is removed — no path ever emitted it. `triggers` was left alone deliberately. It reads as a closed enum but production holds 43 distinct values, because a webhook run stores its provider id; pinning the enum would refuse legitimate history a log search exists to find. The description now says the vocabulary is open. * fix(v2): clamp explicit body caps to the proxy ceiling too The previous commit clamped the default JSON body cap but left explicit per-route overrides alone, so a route declaring a larger `maxBodyBytes` still fell into the truncation it was meant to report: the four inline workspace-file routes at 70 MB and the deployed-chat route at 220 MB. Next attaches `proxyClientMaxBodySize` to every request and clones the body unconditionally for any non-GET method on a matched path, pushing EOF at ten mebibytes with only a warning, so the handler reads a truncated prefix. Those routes therefore already fail above that size — as a malformed-JSON 400. Clamping the effective limit inside the two body readers makes the same request fail as payload-too-large, quoting the limit actually in force. One existing test asserted the unreachable case, allowing a sixty-mebibyte base64 body; it now asserts what the proxy will forward intact. The inline-file path still advertises fifty mebibytes and cannot exceed the proxy ceiling until that ceiling is raised, which changes buffering for every route and belongs in its own change. * fix(v2): close the two holes the first review round found Both are places where a fix in this branch shut one door and left a smaller one open in the same wall. Letting an enrichment group gain an output meant skipping workflow resolution — but that resolution was the only thing validating a new output, so a PATCH began storing coordinates the runner can never fill. It fills a cell from `result[out.outputId]` and skips an output with no `outputId` at all, while the writer diffs on that same id and the sidebar reads and writes by it; the contract leaves it optional. The regression test added with that fix was itself asserting such a dead coordinate. Create's registry checks are now two shared helpers both paths call, and on update an output is exempt only when an identical binding already existed, so renaming a group whose enrichment has since changed still works while anything added or repointed must name a real output. `mappingUpdates` on an enrichment group now says it is inexpressible rather than resolving an empty workflow id into a missing workflow. The layout-reference check was handed the tolerant column set, so a placeholder minted to keep a dangling filter writable also whitelisted a brand-new layout reference — storing an entry the next read discards, which is the inconsistency the check was added to remove. Layout now resolves against the live columns, which is exactly what pruning keeps, while filters and sorts keep the exemption they need.	2026-08-13 16:12:40 -07:00
Waleed	6de8ba2504	fix(v2): close the correctness gaps an end-to-end audit found (#6655) ... * fix(v2): stop a third-party tool description from 500ing MCP discovery `v2McpToolInputSchema` declared `description: z.string().optional()` inside a `.catchall(z.unknown())` object, and a declared key beats the catchall. The MCP SDK's own `ToolSchema.inputSchema` does not declare `description` at all, so any value — including the JSON `null` a Python server emits for an absent one — passes its validation and reaches Sim unchecked. The builder's outbound `.parse()` then threw, and the discovery error policy correctly declines to classify a Sim-side schema defect, so the endpoint that completes MCP onboarding answered a bare 500. The key is dropped and left to the catchall; `type`, `properties`, and `required` stay pinned because the SDK enforces those at least as tightly. Also in the v2 resources family: - The single-resource query schemas for MCP servers, skills, custom tools, and secrets are now `.strict()`, matching every list in the same family. A mistyped flag was silently ignored behind a 200. - `openapi/resources.ts` re-derived `RESOURCE_ERRORS` and `RESOURCE_CONFLICT_ERRORS` inline in 21 of 22 operations. They now import the shared constants; the generated spec is unchanged, which is the point. - The internal MCP refresh route stamped `updatedAt` alongside `lastToolsRefresh`. `updatedAt` means "configuration last changed" and is a public keyset sort, so a refresh moved rows out from under an in-flight page. `updateServerStatus` already held that invariant; the route now matches it. - The discovery cooldown is a typed `McpServerCooldownError` rather than a substring search for `cooldown`. `McpConnectionError` interpolates the server's display name into its message, so a server named after the word was reported as a transient cooldown when its connection had genuinely failed. * fix(v2): close correctness gaps in the workflows deployment surface Deploy and rollback bodies were plain objects, so a misspelled key was stripped rather than rejected. On rollback that is silent misbehavior: an omitted `version` legitimately means "reactivate the preceding version", so `{"versoin": 5}` rolled back somewhere else and answered 200. Both v2 bodies, the run-read query, and the versions cursor are now strict. Deployment versions are an `integer` column, but the path param, the versions cursor, and the v1 body each bounded it differently or not at all — an out-of-range value overflowed the comparison into an unclassifiable 500. One exported bound now covers all three. Resume admission raised bare `Error`s for a stale contextId or an already-resumed run, which the resume surfaces could not classify and reported as 500. They now use the sibling `ResumeAdmissionError` already in that file, carrying 404/409/400 and whether an automatic retry can clear the refusal. Docs corrections: rollback publishes the 409 its webhook-path conflict already produces; deploy/undeploy/rollback reject a workspace key with 403, not the concealed 404 they documented; the workflows OpenAPI module imports the shared error sets instead of re-deriving them; import and the folder ops explain their folder-tree 413. The export route is marked `headSafe: false` so a HEAD probe stops filing a WORKFLOW_EXPORTED audit event for an export that never happened. `runId` is one bounded schema across the run and log resources. * fix(v2): conceal knowledge upload existence, tighten knowledge/files bounds Security: the four knowledge document-upload routes rendered a bare upload error policy with no resource concealment, while every sibling knowledge route uses one. Because the use case resolves the knowledge-base context before workspace authorization, the unconcealed 403 told any valid API-key holder that a knowledge base exists in a workspace it cannot reach — the exact signal GET /api/v2/knowledge/{id} withholds by answering 404 either way. All four now use the composed concealing policy, which also renders the 415/402/413 the route-local renderer already handled; that duplicate renderer is deleted. Contracts: - POST /knowledge/search is strict. It was the only non-strict v2 request body, so a mis-cased rerankerEnabled or topK returned 200 with the key stripped, changing what the caller was billed and silently disabling reranking. - The document list takes limit, cursor, and search from the shared v2 schemas. search was an unbounded, empty-accepting v1 string, so ?search= answered 200 with a full page here and 400 on GET /knowledge, and the term reached an unindexed filename LIKE scan with no ceiling. - The 16 non-strict single-field workspace query slices across both families are strict, matching GET /knowledge/{id}/tags. - GET /audit-logs takes workspaceIdSchema instead of a bare string (?workspaceId= was forwarded as a filter and returned zero rows) and the shared run-window bounds for startDate/endDate. Documentation: - listAuditLogs drops the 404 it has no code path to emit. - upsertFileShare describes its workspace-key refusal as the 403 it renders; the operation denies the key by principal kind, which the concealment policy does not rewrite. - The 12 body-reading knowledge and files operations publish the 413 their pre-validation body read raises, and the file list publishes the folder-tree 413 its now-capped path index raises. Correctness: queryWorkspaceFilePage loads its folder path index under MAX_FOLDERS_PER_WORKSPACE like the workflow, table, and knowledge lists. An uncapped index does not fail on truncation, so a real folder outside the read rows resolved to undefined and answered "Folder not found". * fix(v2): publish the reachable 413 on body-carrying resources ops `parseRequest` buffers a JSON body through `parseJsonBody` under `DEFAULT_MAX_JSON_BODY_BYTES` before any schema runs, and the v2 builders supply `V2_PARSE_DEFAULTS.payloadTooLargeResponse`, so every operation whose contract declares a body already answers 413 above the cap. The resources family published it on none of them. A status a caller cannot see in the spec is a status they will not handle. Adds `RESOURCE_BODY_ERRORS` and `RESOURCE_CONFLICT_BODY_ERRORS` to the shared sets and applies them to the seven affected operations: createMcpServer, updateMcpServer, createSkill, updateSkill, createCustomTool, updateCustomTool, and setSecret. All seven are `defineV2JsonRoute` handlers on non-GET methods with no `parseOptions` override, so the 413 is genuinely reachable on each. The new sets are opt-in rather than folded into the base sets precisely because reachability is not automatic — an operation with no body, or one whose payload reaches it through an uncapped path, would be publishing a response that can never arrive. A sweep test pins the invariant across the resources, billing, and logs documents. It is one-directional by construction: several bodyless operations publish 413 for their own folder-tree and render ceilings, so the converse would flag correct documentation. Also completes the shared-constant consolidation started in cd3efefab9: `openapi/billing.ts` and `openapi/logs.ts` each re-derived `RESOURCE_ERRORS` inline in two operations. Both now import it, and both regenerate byte-identical. * fix(v2): head-safe binary downloads, coded 403s, and truthful surface docs Adds `headSafe` to `defineV2BinaryRoute`, mirroring the JSON builder: a HEAD on a route that declares itself unsafe is authenticated and rate-limited, then answered bodiless before parsing or executing. `GET /api/v2/files/{fileId}` is the one binary v2 route and it records a `FILE_DOWNLOADED` audit event, so a HEAD probe used to fabricate a download that never happened. Names the cause of five refusals that reached the wire as codeless 403s (billing principal-kind, personal-keys-disabled and role, secret admin and write, the workspace table quota, and public sharing), adding three members to the closed `FORBIDDEN_DETAIL_CODES` set. The billing cross-tenant refusal is concealed as a 404 instead of coded, and the credential-list and knowledge file-ownership refusals stay codeless deliberately, documented at the site. Makes `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` reachable: an operation that denies workspace keys also omits them from `principalKinds`, so the kind guard always fired first and callers got `PRINCIPAL_KIND_NOT_PERMITTED` instead of the published code. Drops the unused 410 response, shares one `order` schema between the two run reads so both specs spell the enum the same way, and corrects the false statements about 403 codes, 413 causes, cursor schemes, and full-set lists in the conventions skill and the contract TSDoc. * fix(tables): close the v2 tables correctness and contract gaps - updateColumnOptions was the only column mutator with no lock assert: an options-only PATCH applied on a schema-locked table, and an option REMOVAL cleared cells on a delete-locked one. Assert schema always, escalate to the destructive gate only when options are dropped. - GET/DELETE /tables/imports/{id} 500'd on a first-party import job (null payload) or an unrepresentable status. Both now read as absent, so the answer is the 404 it always was. - Offset cursors stamped the sort but not the filters, so a page-2 cursor replayed under a different predicate paged an unrelated sequence silently. Offsets now carry a filter fingerprint and refuse a mismatch. - Publish 413 on every tables operation that accepts a request body: the v2 JSON builder reads the body under a byte ceiling before validation, so the status is reachable on all of them. Derived at document assembly so a new route cannot regress it. - Enforce MAX_VIEWS_PER_TABLE on view create, making the list contract's "small bounded set" claim true. - Accept the upload control token on the import read, so an upload-backed import is readable during the phase its own 201 reported; drop the `queued` status the reads can never return. - Declare the Find search-term cap, the Find match cap, and the run row-id ceiling the domain already enforces. - Uniform 201 on the row and column creates. * docs(v2): record why the two migrate-on-read GETs stay head-safe An enumeration of side-effecting v2 GETs flagged these two for issuing a workflow_blocks update. The write is convergent and would be issued by the next ordinary read, and headSafe: false answers 200 unconditionally, so declaring it would cost HEAD its existence check to prevent nothing. * fix(api): classify the caller input that reached the driver unvalidated Four families of caller-reachable 500s share one shape: a value the contract admits, the application forwards, and the database rejects. An unclassified driver throw renders as INTERNAL_ERROR, so a bad request came back as a server fault — on pure reads as well as writes. NUL bytes are rejected at the contract boundary, in parseRequest, not per field. A shared string primitive only protects the fields somebody remembers to build on it, and it cannot protect the values that have no string schema at all: a table cell and a predicate value are z.unknown() because their type belongs to the column, not the wire, and those are exactly the values found reaching the driver. One scan over the already validated params/query/body covers every field including the ones nobody has enumerated. Only U+0000 is rejected; every other control character is ordinary content that Postgres stores verbatim. Date bounds on a filter are now parsed, not merely type-checked, with the same normalizer the date column type uses to store cells — so the filter grammar and the storage grammar agree, and gt/gte/lt/lte on both JSONB date columns and the createdAt/updatedAt system columns answer an unparseable bound with 400 instead of an invalid-input-syntax 500. An afterRowId/beforeRowId anchor that does not exist is a classified not-found rather than a bare Error, and a zero-byte knowledge document is refused at admission: every parser rejects an empty buffer outright, so the upload could only ever consume storage and quota on its way to processingStatus failed. * fix(v2): stop six endpoints from returning a confident untruth Six defects that share a shape: a 200 that misrepresents what happened, which is the one class a caller cannot detect from the response. Knowledge search silently degraded. Reranking is implemented and does run, but a deployment with no Cohere credential, a provider error, or a timeout was swallowed into a warning log and answered 200 with plain vector ordering and no `rerankerScore` anywhere — indistinguishable from a reranker that ran and agreed with the vector order. The fallback stays (an outage should not take search down) and is now reported: `rerankerStatus` is required on every search response. v2 also omitted the `rerankerModel` default the internal contract supplies, so `rerankerEnabled: true` alone failed the use case's model guard and returned unreranked results after paying for the widened candidate retrieval; it now defaults like its sibling. `GET /billing/logs` accepted `startDate`/`endDate` with any relative period and dropped them, answering over the default 30-day window — a caller reconciling charges got real rows that were not the rows it asked for. Both bounds are now rejected outside `period=custom`, take the same strict UTC form as `GET /logs` via the shared `v2RunWindowBoundSchema`, and reject an inverted window instead of returning an empty page. MCP registration stamped `connectionStatus: 'connected'` and `lastConnected: now` at insert without contacting the endpoint, and did the same on any non-OAuth re-registration while leaving `lastError` stale. `tool-validation` gates tool availability on that column, so an unreachable server read as healthy. Both paths now leave the columns at their honest defaults for `mcpService.updateServerStatus` to move after a real discovery; the client-side optimistic copy matches. `skills.create` allowed a workspace API key while every other skill write denies one, so a key could only ever accumulate skills it could never remove — and the row it left was attributed to the workspace's billing owner, minting an editor grant for a human who did not act. Creation now denies a workspace key, making the lifecycle symmetric on the per-skill editor model that authorizes the rest of it. `runCount` counts successful non-paused runs and is never decremented by retention, so it disagrees with the runs list in both directions; the description now says so rather than claiming "total recorded runs". Run retention itself was undocumented — free-plan runs are hard-deleted after 30 days, which is why a workflow reports runs beside an empty list — and is now stated on both reads over the execution-log table. * fix(tables): refuse the writes v2 was silently discarding - Uncoercible cell values were stored as null under a 200 on any optional column: "abc"/true/[1] into number, "yes"/1/{} into boolean, "not-a-date" into date, an undeclared option into select, an object into string. The read side already 400s on the same mismatch in a predicate, so the two halves of the API disagreed about the same value. `coerceRowValues` / `coerceRowToSchema` now take an explicit policy and default to `reject`; `null` is passed only where a machine produced the value for a cell no caller typed — a computed (workflow/enrichment) write and a CSV import, neither of which has anyone to answer with a 400. - A multi-select coerced `["green"]` to `[]` — the drop was inside the registry, so no policy above it could see it. It now refuses any part that matches no option, which is what the single branch and the bulk retype gate already did. - A bare number in a date cell was read as epoch milliseconds, so the far more common Unix-seconds shape stored a timestamp 50 years early. The unit is not recoverable from the value and both readings are in range, so a bare number is refused in both directions and the retype gate no longer needs an override to be stricter than the write path. - Unknown column names were dropped by the name→id remap: an insert of {"nosuchcol":"x"} created an empty row under a 201, and a patch of {"zzz":"x"} answered updatedCount:0, indistinguishable from an empty match. The v2 row boundary now names them and refuses. - The table ceiling was enforced only inside createTable, which for an upload-backed import does not run until the CSV has crossed the wire: a full workspace got a 201 and a presigned PUT for up to 5 GiB, then a 403 at complete with an orphaned object left behind. The advisory check now runs when the session is created; the authoritative one stays in the transaction because the quota can move mid-upload. - Cap workflow groups per table. GET /tables/{id}/groups is published as a full-set list, and the group count had no bound of its own — the indirect one does not survive an update path that adds no columns. - Present a group's outputs/dependencies/inputMappings by column NAME. They are created by name, stored by id, and were read back as ids on a surface that is otherwise name-keyed, so a group could not be round-tripped. - Publish the predicate grammar: the operator set, the per-type restrictions, and that `*` — not `%` — is the wildcard. It was true only in the SQL builder's own comments, so the natural guess matched zero rows under a 200. - Stop advertising a `workflowId` default of "" on group create; a manual group that omits it has always been refused. * fix(v2): bind every paged list's cursor to its filters, not just its sort A v2 cursor names a position in one sequence, and a list decides that sequence from its sort AND its filters. Only the sort was stamped on the shared keyset codec, so a cursor from an unfiltered walk was accepted under a changed `search`, `scope`, `deployedOnly`, or folder and answered from a sequence the caller never asked for. The two offset lists already stamped both; nothing else did. The failure differs by scheme but is silent in both. An offset lands at an unrelated ordinal. A keyset stays internally coherent — correctly ordered, duplicate-free — and drops every match sorting before its position, which a caller holding an opaque token reads as "almost nothing matched". One mechanism, shared with the table-row codec: canonical JSON plus a SHA-256 fingerprint (`lib/api/cursor-binding.ts`), stamped by `cursorFilterScope` alongside `cursorSortKey`. The two stamps stay separate so the 400 names which half changed. `limit` is never bound — it selects how much of the sequence to return, not what it is. The three lists whose token is minted by a domain codec (`/logs`, `/audit-logs`, `/billing/logs`) get the same binding by wrapping that token in a query-stamped envelope; the domain cursor is untouched. `present` now also receives the parsed request, so a presenter reads the filters it stamps straight from the query instead of the use case carrying an HTTP cursor concern back out — the `cursorSort`/`cursorScope` round-trips through three application services are removed. `list-pagination.test.ts` now declares each paged list's binding and checks it against the contract in both directions, so a new list, or a new filter on an existing one, fails until its binding is decided. * fix(v2): authorize HEAD probes and declare every v2 query schema Two ways the v2 surface answered a request it had not checked. `headSafe: false` exists so a HEAD cannot fire the side effect its GET performs — an outbound MCP discovery, a FILE_DOWNLOADED audit event, a WORKFLOW_EXPORTED audit event. The short-circuit sat between admission and parsing, so it returned a bodiless 200 before resource authorization ran at all: authorization lives inside the use case, and the use case was exactly what the short-circuit skipped. Any valid API key drew 200 for a denied principal kind, a nonexistent id, another tenant's workspace, and a request missing a required param, while the GET beside it answered 403 or 404. That is an existence oracle over MCP server ids, file ids, and workflow ids. `OperationUseCase` gains an optional `authorize()` that runs the phase before the business transaction — allowed-principal check, canonical load, asserted-scope comparison, current access check — and stops. `defineAuthorizedWorkspaceUseCase` shares one implementation between it and `execute`, so the two cannot answer differently. A HEAD on a not-head-safe route is now admitted, parsed, and authorized like the GET, rendering refusals through the route's own error policy, then answered bodiless. The builders refuse at definition time to pair `headSafe: false` with a use case that has no `authorize`, so the next such route is a boot failure rather than a silent 200. Separately, `parseRequest` validates the query slice only when the contract declares one, so an omitted `query` means "never look at the query string" rather than "takes no query params". 69 v2 contracts omitted it and accepted anything: `?bogus=1` was a 200 on `GET /workflows/{id}` and a 400 on every list. They now declare `noInputSchema`, and 8 more contracts that declared a query without `.strict()` are tightened. A sweep over the contracts tree is the enforcement — a compile-time gate on `defineRouteContract` was tried and reverted because the required intersection collapses inference of the sibling generics. Four route tests appended `?workspaceId=` to a PATCH/PUT that reads it from the body; that copy was being silently dropped and is now a 400. The generated specs are byte-identical: the OpenAPI generator learns that a slice declaring no keys publishes no parameters. * test(tables): pin the multiselect paste on the refusal, not the silent empty cleanCellValue runs the same registry coercion the server does, so tightening multiselect on the server changed this helper too. The case asserting an empty array was pinning the silent-drop the tightening removed. * docs(v2): make the API-key security description render as plain prose The description was already published on every spec but did not appear in the rendered Authorization block. It carried a raw > and backticks, which the markdown pass in the docs renderer does not survive; the operation description on the same page renders fine. Reworded to plain prose with the same substance. * fix(v2): bind the query cursor to its filter on every shape Two agents each fixed half of this: the shared list codecs gained filter binding, and the table codec gained a fingerprint, but the pure-keyset shape stamped it on neither encode nor decode. A keyset position is absolute in (order_key, id), which is why it was left unbound — but absolute ordering is not completeness. Replaying the cursor under a wider filter silently omits every match sorting before it, so paging predicate A then B returned rows 7,9 where the full B sequence is 1,3,5,7,9. Also answers a lost create race with the conflict it already documents, and shortens three descriptions that dwarfed their siblings — the forbidden-code catalogue now lives on the error envelope's details field, published once per document instead of on all 135 operations. * fix(tables): make a saved view's column references survive the write A view config stores every column reference as a stable column id, but two things wrote it in different vocabularies and nothing translated between them. `config.sort` was pruned on read against the live column ID set while the contract defines `sort[].field` as a column NAME, so every name-keyed sort — the only kind the v2 surface can express — pruned to nothing and the view came back with `sort: null`, on both create and PATCH, with no warning. The same prune dropped a sort on `createdAt`/`updatedAt`/`id`, which are sortable row columns that simply are not in `schema.columns`. `config.filter` had the opposite failure: it was stored verbatim, so a predicate naming a column that does not exist saved happily and then 400'd on every `/query`, `/query/count`, and `/rows/find` that tried to use it. The write path now canonicalizes a config before storing it: every column reference (layout keys, `sort[].field`, each `filter` leaf `field`) is resolved to the column's stable id, and `filter`/`sort` are validated against the live schema so a reference that can never resolve is refused instead of saved. The v2 read presents the config back keyed by column name, matching `presentV2WorkflowGroup` and every other v2 row/data surface — a caller never sees a `col_…` id, and what it wrote is what it reads. Resolution is a lookup with pass-through, so the id-keyed first-party UI is unaffected. Column LAYOUT stays unvalidated on write and pruned on read: it auto-saves as the user drags, so racing a column delete must self-heal, not fail the drag. The read path still never prunes a predicate, for the reason already documented there — a pruned condition silently widens the view's row set. * fix(storage): validate at the decode and multipart boundaries, bound derived keys Four caller-reachable 500s shared one shape: input passed boundary validation, then failed in the storage/key layer. Each is fixed at the boundary that owns the transformation, not at the call sites. Percent-encoded NUL in a canonical folder path. `parseRequest`'s NUL scan sees `%00` as three ordinary characters; the NUL only exists after `parseFolderPath` decodes it. Reads survived as 404s, writers carried the decoded name into an INSERT and the driver threw. The rejection now lives in `encodeFolderPathSegment`, the single chokepoint both building and parsing funnel through, so it covers every escape a caller can spell. NUL in a multipart field. A multipart route declares no body contract, so its fields never reach contract validation at all — the knowledge-document key was sanitized while `original_name` was not, and the object landed in storage before the insert threw. `readFormDataWithLimit` is the shared multipart reader every such route already funnels through, so the scan goes there and runs before a caller holds a File to upload, which removes the orphan rather than cleaning it up. Storage-key overflow at 225 characters. Every generator embedded the file name in a path component it also prefixed with a timestamp and a uniquifier, so the effective limit was 255 minus that prefix while the contract advertised 255 — a 225-character name produced a 256-byte component and ENAMETOOLONG from local storage, and the upload session handed out a transfer URL that could never succeed. `buildStorageKeySegment` reserves the prefix out of the component's budget, making the key independent of name length and the declared limit honest. The NUL predicate is now shared from `@sim/utils/string` by all three boundaries instead of being restated at each. * docs(v2): make the published spec describe the API it has Three descriptions asserted behavior the code no longer has, and three rules the code enforces were published as unconstrained strings. `downloadFile` and `listMcpServerTools` still told callers a `HEAD` on a not-head-safe route "is answered with an empty 200 ... reports only that the endpoint exists and the caller is authorized". That was true of the old short-circuit, which sat between admission and parsing and therefore returned 200 for an id the same caller's `GET` refused. The builders now authorize a HEAD exactly as the GET, so the spec said the opposite of a security fix. One `HEAD_MIRRORS_GET` constant replaces both sentences and is added to `exportWorkflow`, whose `headSafe: false` was never documented at all. A test walks the `app/api/v2` tree for the declaration and fails on any operation that carries it without the sentence, or that resurrects the old claim. `createMcpServer` promised that re-registering an existing URL "rewrites the configuration and returns the server to the same unverified state"; it is a 409 pointing at PATCH. `authType` claimed Sim "detects it from the server when omitted" — registration deliberately never contacts the server, and the column defaults to `headers`. The default stays: `headers` and `none` are behaviourally identical (only `oauth` branches), so changing it is a migration with no caller-visible payoff, while the sentence was simply false. `predicate` was the API's most consequential gap: a `pipe` over `z.unknown()` documents from its input, so the leaf keys `field`/`op`/`value` appeared nowhere in the contract and `{column, operator, value}` was a 400 a caller could not correct against. Both predicate schemas now publish a real recursive JSON Schema through `.meta()`, self-referencing so the recursion resolves from one `$defs` entry, with every bound read from the constant that enforces it. Also published: the canonical folder-path rule and its 4096-byte cap on the four path components (the `superRefine` contributed nothing to JSON Schema); the closed 12-value `recursive` vocabulary on a destructive delete; and the null-matching behaviour of the negating operators. The clamping `limit` branch drops `minimum`/`maximum`, which in JSON Schema mean "rejected outside" and made SDKs refuse locally what the server clamps. `deleteFile` stops publishing a 409 nothing in its path can raise. `restoreFile` and `abortFileUpload` keep theirs — the report called them unemittable, but restore raises `FileConflictError` after exhausting its rename retries and abort refuses a completed session. Description tail, across the seven specs: p99 733 to 465, max operation 1643 to 1114, over 700 chars 31 to 13, over 400 70 to 61. Constraints moved from operation prose onto the fields they constrain rather than being deleted. * fix(v2): make upload completion, blank query values, search, and folder filters answer correctly Four defects on the v2 surface, each reproduced before it was fixed. Upload completion dispatched document indexing from inside the completion transaction, so a queue or processing failure returned 500 after the object was stored, the document row was created, and the session was marked completed — and the only recovery, replaying the request, answered 200. The dispatch is now a follow-on step that runs after the session is durably completed and is logged rather than raised. Its outcome stays visible on the document itself (`failed` with an error, or `pending` when it was never picked up), and the recovery path re-queues a `pending` registration instead of keying off a message left on the session. A query parameter sent with no value was read as `0`, `false`, or the parameter default: `?limit=` became `LIMIT 1` on the three lists that clamp, and `?minCost=` on `/logs` became a live `cost >= 0` filter. `search` and `cursor` already rejected a blank and documented "omit the parameter instead"; that rule now applies to every v2 parameter, enforced on the raw query before coercion so a parameter added later inherits it. The document list matched `_` and `%` in `search` as live LIKE wildcards while every sibling list escaped them through `searchFilter`, so the documented substring match returned everything for `a_itest`. It now uses the same helper. A `folderPath`/`folderPaths` naming no folder answered 404 on `/logs`, `/files`, `/workflows`, `/tables`, and `/knowledge`, while every other filter answers an empty page and the sibling folder lists already do. All five now return an empty page. Mutations keep their 404. * chore(v2): regenerate the specs from the merged sources The four spec conflicts in the wave-3 merge were resolved by taking one side, which left them describing neither branch. Regenerated so the published documents match the contracts they are built from. * docs(v2): give a built-in skill's id its real form The contract said a built-in skill uses its name as the id. The ids are `builtin-` plus the name, so a client following the description asks for /skills/research and gets a 404 where the spec promises the skill. * fix(uploads): keep local upload artifacts inside NAME_MAX `POST /api/v2/files/uploads` accepted a name of up to 255 characters, returned 201, and handed back a transfer URL that could never succeed: the PUT against it 500'd and `complete` then reported the object missing. The local provider named its staged object after the destination — `{key}.{uploadId}-{uuid}.tmp` plus a `.upload-metadata.json` sidecar — so the staged component was the key's length plus ~99 bytes of fixed overhead. Past roughly 125 characters of name that crossed POSIX `NAME_MAX`, and `ENAMETOOLONG` is not a `LocalUploadBodyError`, so it escaped as a 500. Multipart `complete` built the same name and failed the same way. Only local storage is affected; S3, Azure, and GCS have no per-component limit. `buildStorageKeySegment` already budgeted the key to 255, one layer above where the overflow happened. Two changes close it at the layers that own each suffix: - Staged artifacts move to a `.staging` root and are named from the upload id alone. A name derived from the destination inherits its length and then adds to it; a fixed-width one removes the arithmetic instead of re-budgeting it, so no suffix added here later can depend on the caller's file name. The staging root is a cleanup sweep root, which also reclaims artifacts that used to be orphaned beside the destination. - The durable sidecar is reserved out of the key budget centrally. `LOCAL_UPLOAD_METADATA_SUFFIX` moves next to the budget that must account for it, and the budget is derived from a list of sidecar suffixes, so adding one shrinks every key builder at once. The declared `maxLength: 255` stays honest: a 255-character name now completes PUT and `complete` end to end. * fix(uploads): budget every key built from a caller-supplied name Auditing the rest of the codebase for the shape that broke the upload-session PUT found five more key builders that put an unbounded name into a path component local storage writes directly. Three are on the same route as the original bug: `table_import`, `profile_picture`, and `workspace_logo` built their key inline with `sanitizeFileName`, which maps characters and never truncates, while their sibling purposes went through `buildStorageKeySegment`. A 255-character name broke `table_import` at the metadata sidecar and the other two at the object write itself. The other two are local-storage writers reached from elsewhere: knowledge-base connector sync capped the document title at 200 and then appended a timestamp, a uuid and `.txt` on top of the cap, landing at exactly 255 with no room for the sidecar; the Mistral-OCR staging and chunk keys inlined the sanitizer with no bound at all; and inbound email attachments went into a key with neither sanitizer nor bound, on a file name an outside sender chooses. All now derive their component through `buildStorageKeySegment`, so the reservation is stated once. The upload-session test asserts it for every purpose the contract admits, which is what keeps a newly added purpose from reintroducing the hand-built form. * fix(v2): stop the logs and billing reads answering 500 or a silent restart Four caller-reachable failures on `GET /logs`, `GET /logs/{runId}`, and `GET /billing/logs`, each fixed at the layer that owns the guarantee. `minDurationMs`/`maxDurationMs` were published as `number` against an `integer` column, so `1.5`, `-0.5`, `2147483648`, and `1e30` all reached Postgres as bind parameters it refuses to parse. They are now whole milliseconds bounded to int4, and the generated spec says so. `0000-01-01T00:00:00Z` satisfies the published `date-time` pattern but names no instant Postgres can store, since the proleptic Gregorian calendar has no year zero. `v2RunWindowBoundSchema` now rejects it, which covers both log families and the files-audit read that share the schema. A scoped cursor whose inner token was the empty string passed the `typeof === 'string'` envelope check and then read as falsy in every domain reader, so both lists silently served page one again with a `nextCursor` inviting another lap — the exact failure `UNKNOWN_CURSOR_MESSAGE` exists to make visible. An empty inner is now unreadable, and the sibling `decodePublicLogCursor` gets the same treatment for its `id` half. The rejection message no longer names `sortBy`/`sortOrder`, which neither operation accepts. `GET /logs/{runId}` reported `folderPath: null` for both a workflow at the workspace root and a folder it could not resolve, so a caller could distinguish neither, and `null` is not a value `folderPaths` takes back as a filter. The root is now `/`, matching the workflow resources. Also, from the same audit: comma lists reject an empty entry the way `folderPaths` already did instead of dropping it; a query param sent twice is named as duplicated rather than reported absent; and the `triggers=all` sentinel, the detail-level promotion by `includeTraceSpans`/`includeFinalOutput`, and the 403/404 split against the billing family are documented where each is decided. * fix(v2): pin naive timestamps to UTC and close six contract divergences Application-written timestamps reached the wire as a local wall clock labelled `Z`. Every column in `schema.ts` is `timestamp without time zone`, so the instant a value denotes was decided by whoever wrote it and whoever read it, and the writers disagreed: `now()` renders in the session's TimeZone, drizzle's `mapToDriverValue` is `toISOString()`, and a raw `Date` bound through postgres.js is cast down in the session's TimeZone. The read side disagreed the same way — postgres.js parses oid 1114 with `new Date(x)`, which is the process's local zone, while a value it hands back as a string is read as UTC by drizzle. The result passes every `date-time` check, so it silently corrupts sorts and range predicates and can place `updatedAt` before its own `createdAt`. `packages/db/timestamps.ts` removes the ambiguity at the driver boundary rather than at the call sites: the session TimeZone is pinned to UTC so all three write paths store the same wall clock, and oid 1114 is parsed as UTC so every read path recovers that instant. `withUtcTimestamps` merges both into a client's options, because `connection` is nested and a pool setting its own `application_name` would otherwise drop the TimeZone. Production already runs both in UTC, so nothing changes there; every other environment now behaves the way production does. Alongside it, six places where the published contract and the code disagreed: - Multi-select `ncontains` was documented as "the exception" that excludes nulls. It never did, and no test claimed it did — `data` is never NULL, so containment is false for an absent key and the negation is true, exactly like every other negation. The sentence was wrong. - `recursive` published twelve lowercase spellings while `z.stringbool()` folded case, so the server honoured `recursive=True` as a destructive recursive delete that a generated client would have refused to send. Narrowed to case-sensitive: accept exactly what is published. - The upload data plane answered with a bare `{ error: string }`. Being absent from the OpenAPI documents is a statement about addressability, not about behaviour; both PUTs now use the canonical envelope, and what the transfer step promises is published on `transfer.url`. - Full-set lists told callers to "send it back as `cursor`" on a `.strict()` query that rejects `cursor`. `v2CursorListResponse` now takes `paged`. - A `HEAD` on a download skips the read that produces `Content-Length`, so it cannot size a download; the description says so. - The upsert conflict-target rejection echoed the storage id a name-keyed surface had already translated to, and the scoped-cursor 400 named `sortBy`/`sortOrder` params `/audit-logs` does not accept. * improvement(v2): cut the extraneous half out of the published descriptions The v2 spec's description median was already healthy at 42 characters; the tail was not. 174 descriptions ran past 200 characters and 13 past 700, almost all of it rationale, cross-references, and constraints restated on the wrong object. Trim the shared error, folder-path, retention, pagination, and workspace-key constants first, since each is published on between two and twenty-seven operations. `FOLDER_TREE_TOO_LARGE` dropped the clause explaining why the tree has to load, `FULL_SET_LIST` dropped a second sentence restating its first, `RUN_RETENTION` dropped the `runCount` caveat that already lives on `runCount`, and the 503 and 499 descriptions dropped the paragraphs narrating why they are documented at all. That reasoning belongs in the TSDoc beside each constant, which is where it now is. Then the operations. Execute Workflow and List Runs each restated a rule their own parameters already carry — the `X-Run-Id` uniqueness claim and the `order` sort deviation — so both moved to the parameter that owns them. The run-status enum sent a caller to `paused.automaticResumeWaitingReason` and then explained that field in place of describing it; the explanation moved onto the field, which previously said only that it was "the reason automatic resume is waiting". Align the parameter vocabulary a caller meets in every family. One `cursor` description had forked on the table row query, one `sortBy` on knowledge documents, and the table row `limit` published neither its bounds nor its default. `nameSortCollation` is now a function of the column it names, so the knowledge document list can state the caveat about `filename` without claiming a `name` field it does not have. `scripts/openapi/documents.test.ts` pins `cursor` and `sortOrder` to one string each, and the retention window to both reads that publish it. Distribution over the seven documents: mean 71 to 67, p95 223 to 199, p99 453 to 370. Over 200 characters 174 to 147, over 300 94 to 59, over 400 54 to 20, over 700 13 to 9. The median is unchanged at 42. * fix(v2): keep one unreadable-cursor message Two branches each added the constant, in cursor-binding and list-query. It belongs beside its sibling REFILTERED_CURSOR_MESSAGE, so the list-query copy and its importers move there. * fix(v2): bind a cursor to what a set filter means, not how it was spelled workflowIds, triggers and folderPaths are comma lists the query treats as unordered sets, and tagFilters is an object whose key order carries no meaning. Fingerprinting the raw spelling bound the cursor to the spelling, so a caller who reordered an equivalent filter mid-walk got a 400 for a page that was genuinely the next one. * fix(v2, db): make two unfalsifiable tests observable and document strict query Three follow-ups on the w5 policy work: one decision recorded, two tests that could not fail. The `query: noInputSchema` sweep is kept. It is a real tightening — 69 v2 operations that ignored an unknown query param now answer 400 — so it was weighed rather than assumed. The v2 body slice on those same endpoints was already `.strict()`, and every v2 list already rejected `?bogus=1`, so the split was arbitrary rather than a promise: the same typo was a 400 on `GET /workflows` and a silent 200 on `GET /workflows/{id}`. A parameter the server drops without saying so is the bug class the lists' rule already exists to prevent. No first-party caller is affected — the two SDKs send only `includeOutput`/`selectedOutputs`, both declared; the UI and the desktop app make no v2 calls at all; `requestJson` appends nothing implicitly and no v2 cache buster exists; every docs example uses a declared param. A third-party caller appending a tracking tag does break, which is why the behavior is now documented in the API reference with the exact 400 body rather than left to be discovered, and why the reasoning sits in the v2 conventions skill next to the rule instead of only in a commit message. `packages/db/timestamps.test.ts` asserted that `withUtcTimestamps` registers a UTC parser on oid 1114 by reading it off a bare postgres.js client. Every real client is then handed to `drizzle()`, which overwrites that entry with a transparent parser, so the assertion held whether or not the parser had any effect. The mechanism is fine and stays: drizzle's own `PgTimestamp` mapper appends `+0000`, so the read is UTC-correct either way and the session `TimeZone` pin — the write-side fix — is untouched by `drizzle()`. The test now resolves the parser both before and after `drizzle()`, pins the clobbering it depends on, and asserts the instant recovered through the full composition, so a regression in either layer is red. `timestamps.ts` records why the inert entry is kept. `nul-byte-boundary.test.ts` embedded a raw U+0000, so git classified it binary and rendered it as `Bin 0 -> 4102 bytes` — the test proving the NUL hardening works was the one file a reviewer could not read. The escape is byte-for-byte equivalent at runtime. Two older files had the same defect and are fixed the same way. `check:source-text` now fails the build on a raw NUL in any tracked source file, and `.gitattributes` forces source files to diff as text so the next one is visible in review rather than hidden by it. * fix(w5): narrow three fixes that reached past the harm they were fixing The workflow-create `23505` handler answered for the whole transaction, which also runs `saveWorkflowToNormalizedTables`. `workflow_blocks.id` is a global primary key, so a block-id collision — an integrity fault already seen in production — surfaced as `A workflow named "X" already exists in this folder`. Match on the constraint name; any other unique violation propagates unchanged. Moving the knowledge dispatch out of the completion transaction was right, but a dispatch failure then committed the session as `completed` and left the document at `pending`, which nothing sweeps and `retryProcessing` refuses. Record the failure on the document instead, so it lands on the existing failed-document path, and describe what the code does rather than a recovery branch that cannot fire for this state. The MCP re-registration reset stopped a registration claiming a connection it never made, but reset for any re-registration. `isServerEligibleForDiscovery` skips an OAuth row that is not `connected`, so a rename removed every tool the server published with no path back. Scope the reset to url, transport, headers, auth type, OAuth credentials, and revival. * fix(tables): confine the write-policy tightening to what the caller sent The null-policy work made `reject` the default for caller-supplied writes, which is right, but it landed on the wrong values. - A partial update coerces the MERGED row, so an untouched legacy cell failed an unrelated column's update — and failed a paged bulk job after its earlier pages had committed. The merged-row callers now name the patch's keys; every other key follows the `null` policy, in the in-memory copy only (the write sends the patched keys alone). - A multiselect whose members do not all resolve returned `{ok:false}`, which on the machine paths that pass `'null'` — CSV import, computed writes, the cell-write snapshot — erased the whole cell. Those paths now consult a new `salvage` hook and keep the members that do resolve; a caller-supplied write still 400s on an unknown option. - Refusing a bare number in `date.coerce` reached the executor, v1, copilot and the grid. The refusal stays where there is a caller to tell, and `salvage` restores the milliseconds reading where the only other answer is a blank cell. Also: the cursor docblocks claimed pure-keyset cursors were left unbound while the code and its tests bind them; a saved-view create took the table's SCHEMA advisory lock, so it queued behind column rewrites whose statement timeouts run past its 3s lock_timeout, and now takes a views-scoped lock instead; and a view whose column was deleted could not be saved at all, because the Save chip always resends the filter — references the stored config already carries are now exempt while a newly introduced one is still refused. The cursor version is deliberately not bumped: the stamp is additive, unfiltered in-flight tokens keep working, and a filtered one fails with the accurate "restart paging without the cursor" rather than a generic unreadable-cursor 400. * test(db): narrow the mapped timestamp to Date mapFromDriverValue is typed unknown, so the composition assertions did not type-check outside the test's own runner. * fix(v2): correct four stale contracts and clear the merge debris behind them Five of the reported defects were real and four of them were documentation that had stopped describing its own code. `cleanCellValue` said only "coerce a raw input value"; it also answers `null` for anything the column type refuses, and since the multiselect write path started refusing partial matches that is the difference between a paste storing one option and blanking the cell. It deliberately does not consult `salvage`, which would read the same paste as the option that did resolve — that reading is for writes with no caller to answer, and a typed cell has one. The pairing is now asserted, so a future helper that "improves" the paste by salvaging it fails. `EXECUTE_OPTION_CONSTRAINTS` carried two stacked TSDoc blocks, the second explaining that the enumeration had moved onto the fields; the body schema still told a reader the six combinations were enumerated in the constant. The deployment route's second block orphaned the endpoint documentation above it, and `list-query.ts` kept the TSDoc for a cursor message that now lives, with its own rewritten doc, in `cursor-binding.ts`. Two agents left near-identical essays arguing the same 400-vs-403-vs-409 question about the table ceilings and concluding that neither status changes; the decision is recorded once, in `billing.ts`, and `service.ts` points at it. The credentials use case echoed `sortBy`/`sortOrder` back with a TSDoc explaining that the presenter needs them, which it no longer does — it reads `query.*`. The local upload roots move from the data-plane provider to `core/storage-key.ts`, beside the sidecar suffix, so the cleanup sweep can name what it reclaims without importing the transport that writes it. `documents.test.ts` justified sweeping only knowledge and files for the 413 by saying the same sweep over the other five documents still reported gaps. It does not: widened to all seven, every body-carrying operation publishes it. Three reports did not survive checking, and the evidence is recorded where the next reader will look. An empty rerank result is not the reranker matching nothing — `rerank` asks for `top_n` over a non-empty document list, so an empty array means the response carried nothing usable, which is what `unavailable` already promises. The zero-byte knowledge document is refused on the upload-session path too, by `validateFile`, under both boundary contracts; that parity is now pinned, and it fails if the guard is removed. The MCP re-registration reports exactly the connection fields its SET clause writes, and the create mutation already drops both caches — what lags is the status badge, not the tools, because discovery is gated on `connected` for OAuth rows only. * fix(v2): de-duplicate a set filter before fingerprinting it The filters compile to inArray, which is set membership, so workflowIds=A,A,B selects exactly what A,B does. Sorting alone still bound them to different pages, so an equivalent filter with a repeated member 400d mid-walk. * fix(w6): close a head-authorization hole, a TZ leak, and five tests that could not fail Six risks an adversarial read of this week's diff raised, verified one at a time. Two of the six were already correct and are reported as such rather than changed. `v2HeadAuthorizationResponse` optional-called the use case's authorization phase, so a use case without one would have answered the bodiless 200 that `headSafe: false` exists to prevent. The definition-time guard does cover both builders that reach it — they are its only callers — but an optional call turns a missing phase into that leak silently, so the responder now refuses instead of skipping. `packages/db/timestamps.test.ts` assigned `process.env.TZ` at module scope and never restored it. `TZ` is process state: a worker running files back to back carried Asia/Tokyo into every file that followed, and only when the ordering put it after this one. The zone is now set and restored around the file, with both properties the suite depends on intact. Upload publication moved its staging area out of the destination's own directory into a shared `.staging` root, which makes the publishing `link` a cross-subtree one. A volume mounted under part of the uploads tree puts the two on different devices and `link` answers `EXDEV`, which the same-directory link could not. Publication now copies onto the destination's device and links from there, keeping the create-or-fail step that stops a replay from overwriting a stored object. Five tests that passed regardless of the code: - `resolveFolderPathFilter` was only ever exercised through hand-written reimplementations in the suites that mock it out, so widening a miss to unfiltered — every filtered list answering with the whole workspace — left them all green. The real helper is now tested where it lives. - The only measurement of `generateWorkspaceFileKey` asserted the key's last component against `NAME_MAX` rather than the component plus the sidecar written beside it, so it passed with the sidecar reservation removed. - `GET /logs` asserted only that a rejected cursor does NOT name `sortBy`, which almost any wording satisfies, including one saying nothing at all. - The skills lifecycle test asserted that the four writes agree on a workspace-key policy, which a lifecycle uniformly allowing one also satisfies; it now pins the policy they agree on and the kinds they admit. - The v2 skills create test lost `expect(capture).not.toHaveBeenCalled()` when the create path moved to a personal key. The behaviour it pinned is gone — the workspace-key create is refused now — so it is re-homed as the refusal reaching the caller as a 403 with no analytics behind it. Two claims did not hold. `CURSOR_VERSION` is correctly left at 1: the filter stamp is additive, a pre-stamp token still decodes, an unfiltered read still resumes, and only a filtered replay fails — with a conflict that names the filter, where a version bump would answer a generic unreadable-cursor 400 to every in-flight token. Tests pin all three, plus the minted version itself. And the upload-session key-budget cases do exercise the real shared budget through the real segment builder; only the workspace-key prefix is the stub's, which is now stated where the stub is declared. * refactor(v2): collapse two names for the cursor scope key onto one helper `cursorFilterScope` in the v2 response module was a one-line pass-through to `cursorScopeKey` in `lib/api/cursor-binding`, so the same function was reachable under two names from two modules. Routes now call `cursorScopeKey` directly, the way they already import `unorderedScopePart` and the cursor messages from that module, and the wrapper plus its duplicated doc comment are gone. Also folds the `id -> name` column map in the v2 tables presenter onto `buildColumnNameById`, which the same file already imports and calls thirteen lines above; restores two doc comments that had drifted onto the wrong declaration; and replaces three `as Date` casts in the timestamp test with `toEqual(new Date(...))`, which needs no cast and additionally fails when the mapped value is not a Date at all. * refactor: delete three pieces of surface this branch added with no consumer `v2CursorSchema` had one caller, `v2PaginationFields`, in the same file, and its only parameter was a default nobody overrode — so the export and the parameter were both unreachable. Inlined into the pair it belongs to; the emitted schema and its description are byte-identical, so the generated OpenAPI does not move. `PatchedKeys` was declared `ReadonlySet<string> | readonly string[]`, but all four callers pass `Object.keys(...)` and no test passes a set, which left the `instanceof Set` arm of `policyResolver` unreachable. Narrowed to the array form the callers actually use. `NUL_CHARACTER` was exported from `@sim/utils/string` and imported by nobody — every boundary imports `containsNulCharacter` instead. Kept as the module-local constant the predicate reads, dropped from the package surface. * docs(v2): state why the local upload data-plane routes bypass the builders Both local-storage PUT routes use raw `withRouteHandler`. The global rule allows that only for documented protocol or lifecycle exceptions, and their TSDoc explained the OpenAPI exemption and the error envelope but never the builder bypass itself. Record the actual reason: a signed `upload-token` is the credential, so there is no API key, `Principal`, or semantic operation for a builder to authenticate and authorize against, and the body streams straight to storage rather than being parsed. * test(v2): pin cursor-to-filter binding on the tables and runs lists The branch binds every paged cursor to the filters it was minted under, but the binding was enforced end-to-end on only 4 of 16 paged lists. The contract-level CURSOR_BINDINGS sweep looks like the safety net and is not: it checks each contract against a hand-maintained map of param names, never against what a route actually stamps into cursorScopeKey, so it stays green for a route that dropped the stamp entirely. Confirmed by deletion. Removing tableCursorFilters from both call sites on GET /v2/tables left all 8 tests passing, and the runs route was worse — its one relevant assertion was weakened from toEqual to toMatchObject in this same branch, leaving the new filter field unpinned. Adds a mint-then-replay test to each: a cursor minted under one filter set and replayed under another is a 400 that never reaches the use case, with a same-filter resume case as the control so the 400 cannot be satisfied by blanket rejection. Restores toEqual on the runs cursor payload, pinning that a filter is stamped without hardcoding the fingerprint. Both new guards were verified to fail: removing the binding reddens the refiltered test on tables, and both the refiltered and the re-armed toEqual test on runs. * fix(tables): keep the v2 write strictness inside v2 The write-path tightening on this branch changed shared code that every first-party surface reaches, so the workspace grid, the internal `/api/table` routes, `/api/v1`, the Copilot table tools, and the executor's Table block all inherited a contract only `/api/v2` publishes. Each of them now behaves exactly as it does on staging again, and v2 keeps the strictness by opting into it. - `coerceRowValues`/`coerceRowToSchema` default to the `null` policy again — an uncoercible optional cell is blanked and the row is written. `reject` is reached through `RowWriteOptions.uncoercibleValues`, which the v2 row routes set via `strictWrite` on the application input. - The same `strictWrite` scopes the unknown-column refusal to v2. Copilot feeds the model's raw arguments in unfiltered, so a hallucinated key, an echoed `id`, or a name left over from a rename had begun refusing the whole write. - Multiselect and bare-epoch values land again for first-party callers through the registry's existing `salvage` hook, which the `null` policy already consults; the grid's `cleanCellValue` consults it too, so a paste naming one live option and one deleted one keeps the live one instead of erasing the cell. - The saved-view name→id remap no longer rewrites a ref that already means something else, so a user column named `id`/`createdAt`/`updatedAt` cannot hijack a view's system-column sort or filter. - `createTableView` tolerates the refs its own config carries unless the caller is strict, so "Save as view" stops 400ing on a dangling filter the Save chip accepts. - The bulk update runner is byte-identical to staging again. The 100-view cap stays: the list read is unpaginated, so the promise it makes only holds if the write side enforces it, and it refuses a new view rather than an existing config. * test(v2): pin cursor-to-filter binding on seven more paged lists Extends the mint-then-replay guard from tables and workflow runs to the remaining paged v2 lists the audit found with no route-level coverage: credentials, audit-logs, custom-tools, mcp-servers, secrets, knowledge bases, and knowledge documents. Each gets a cursor minted by driving GET under one filter and replayed under another, asserting a 400 carrying REFILTERED_CURSOR_MESSAGE that never reaches the use case, plus a same-filter resume control so the 400 cannot be satisfied by blanket rejection. The three cursor schemes are all covered: keyset (readSortedCursor), the scoped wrapper audit-logs uses for its domain token, and the offset cursor on knowledge documents. The documents suite had no GET coverage at all, so its list use case gains a real mock and the route's GET export a describe block. All fourteen were verified to fail: dropping the cursor-filter argument from both call sites on each route reddens exactly that route's refiltered test and leaves every other assertion in the file green, which is the failure mode the contract-level CURSOR_BINDINGS sweep cannot see. * test: cover four untested behaviors and drop five tests that cannot fail Adds coverage that goes red when the behavior is reverted: - `rejectDuplicateQueryValues` through `parseRequest`, not just the pure helper — the existing blank-query tests stay green even when parseRequest ignores the flag entirely. - `failUndispatchedDocumentProcessing`'s pending + not-deleted WHERE guard, asserted on the condition tree so removing it fails. - The widened `present(result, request)` signature, so dropping the second argument stops being a silent no-op. - The NUL scan on `readFormDataWithLimit`'s content-length branch — the branch every ordinary browser and curl upload takes, and the one the existing multipart tests never reached. Removes tests verified incapable of failing: the credentials projection row (the outbound `.parse()` strips unknown keys either way), the per-document 413 sweep (vacuous on two of three documents, subsumed by the sweep in scripts/openapi/documents.test.ts), the two upload-session rows that assert their own `generateWorkspaceFileKey` stub, the storage-key row whose 20-byte name never reaches the budget, and the views-lock assertion against a function `views/service.ts` does not import. * fix(v2): parse a bound list filter once, so the scope matches the query The logs list fingerprinted `workflowIds`, `triggers`, and `folderPaths` through unorderedScopePart, which trims each member, then split the same raw values itself with `.split(',').filter(Boolean)`, which does not. So `?workflowIds=A,B` and `?workflowIds=A, B` produced one fingerprint and two different result sets: the second selects on a member with a leading space that matches no row. A cursor minted under one was accepted under the other, which is the exact failure the filter binding exists to refuse. Extracts parseUnorderedList as the single parse. unorderedScopePart now derives from it, and the route passes the array to the query and the joined form to the scope, so the members fingerprinted are by construction the members filtered on. Also drops three inline splits. Reported by Greptile. * fix(v2): bind an AND-conjoined filter array as a set, not a sequence The knowledge documents list fingerprinted tagFilters through canonicalJson, which sorts object keys but preserves array order. Each filter compiles to a condition in and(...whereConditions), and AND is commutative, so the same clauses written in a different order select the same documents — and got a different fingerprint, refusing a cursor for a page that was genuinely the next one. Adds unorderedJsonScopePart beside parseUnorderedList: members are canonicalized, de-duplicated, and sorted, so `A AND A` binds like `A` and clause order stops mattering. A non-array or unparseable value still binds by its raw spelling, since that request fails validation anyway. Replaces the route-local canonicalTagFilters, and corrects the claim on canonicalJson that array order only ever costs a restart — for a set-valued filter it costs a spurious 400. Reported by Greptile. * fix(v2): bind list filters by the value the query acts on, not its spelling Third report of one root cause, so this fixes the cause rather than the case. A cursor scope must fingerprint what the query filters on; every place it fingerprinted the caller's raw text instead, two spellings of one filter got two scopes and a valid next page got a 400. Knowledge documents: tagFilters bound the raw query text while the route already parsed it two lines below for the use case. The schema defaults operator to 'eq', so {tagName,value} and {tagName,value,operator:'eq'} are one filter to the query and were two scopes to the cursor. The scope now binds the parser's output, which also subsumes the clause-order fix — both route tests go red against the raw-text form. Logs and workflow runs: startDate/endDate bound the raw text, but z.string().datetime() admits every sub-second spelling of one instant, so `…00Z` and `…00.000Z` name one window and got two scopes. New instantScopePart binds the parsed instant. Replaces unorderedJsonScopePart, which took raw text and could not see a schema default, with unorderedScopeOf over the parsed value. Swept all fourteen routes that build a cursor scope for the same divergence; these were the only ones where a scope part is derived differently from the value reaching the use case. Reported by Greptile. * fix(v2): bind the audit and billing window bounds by instant The previous sweep for this defect looked for a transform in mapInput, so it missed the two routes that pass their raw bounds to a use case that parses them deeper. Both fingerprinted startDate/endDate as text while their predicates convert to a Date, so `…00Z` and `…00.000Z` name one window and got two scopes, refusing the genuine next page. Billing keeps stamping the raw params rather than resolveDateRange's output, for the reason already recorded there: a relative `period` resolves against the clock, so hashing the resolved window would reject every next page. Normalizing the explicit bounds is compatible — instantScopePart is a pure function of the caller's own text and resolves nothing. Re-swept all fourteen cursor-scope routes by scope part rather than by transform site. Every temporal and structured part now binds canonically; the rest are enums and identifiers with one spelling per value. Reported by Greptile. * fix(v2): drop an inert field from the document tag-filter scope resolveKnowledgeTagFilters builds every structured filter with the stored definition's fieldType and never reads the caller's — not for resolution, not for validation, not in its output. Fingerprinting it made a field the query ignores decide whether a cursor resumes, so adding or removing a matching fieldType refused a page that had not moved. Swept the other twelve cursor-scope routes for the same shape. No scope part is absent from its mapInput, this was the only scope carrying a structure resolved against stored state, and knowledge/search has no cursor at all. Reported by Greptile. * refactor(v2): derive the body 413 from the contract in every document Two mechanisms encoded one rule. `withRequestBodyErrors` derived the 413 from `route.contract.body` for the tables document, while the resources document hand-picked RESOURCE_BODY_ERRORS / RESOURCE_CONFLICT_BODY_ERRORS at nine sites. The cross-document sweep caught drift, but only after the fact: a new body operation that forgot the _BODY_ variant published a reachable 413 nowhere until a test failed. Hoists the mapper to openapi/shared.ts and applies it in both documents, so the rule is derived rather than remembered. The two hand-picked sets and their shared TSDoc are gone. Regenerating all seven specs produces zero drift, which is the proof the two mechanisms were computing the same thing. * refactor(v2): collapse duplicated cursor and validation mechanisms, drop dead exports One rule, one implementation: - `parseRequest` hand-inlined the "caller envelope or default" validation-error projection four times. Extract `projectValidationError` and route all four through it. - Nine keyset lists hand-rolled the `present` half of the cursor pair that `readSortedCursor` already owns the read half of. Add the symmetric `writeSortedCursor` and use it everywhere. - `GET /workflows/{id}/runs` re-derived `readSortedCursor`'s invalid/refiltered ladder from `decodeSortedCursor`; it now calls the shared reader and keeps only the key-arity check that is genuinely its own. Files and exports that no longer earn their place: - Inline `credentials/utils.ts` into its single consumer. - Delete symbols with zero references repo-wide: `v2CustomToolWriteError`, `secretCredentialTypes`, `v2CursorList`, `v2WorkspaceAccessError`, `resolveFolderPathIdentity`, `folderPathForId`, `v2FolderPathMutationError`, and seven of twelve `tables/utils.ts` exports. - Drop `export` from symbols used only inside their own module. No behavior change; every response body and error message is byte-identical. * docs(v2): cut duplicated and non-load-bearing comment prose Five rationales were written three to five times each by parallel agents that could not see one another. Each now has one home and the rest point at it: - HEAD existence oracle -> the headSafe option on defineV2JsonRoute - cursor query binding -> cursorScopeKey in lib/api/cursor-binding.ts - storage-key prefix budget -> buildStorageKeySegment - NUL / U+0000 -> the containsNulCharacter predicate - blank and duplicate query values -> their own implementations Also drops changelog-in-source (prose narrating what the code used to do), anchorless module headers attached to no declaration, rejected- alternative essays, and @param tags that only restate the signature. Comments only: the diff contains no executable-code change. * fix(v2): name the undecodable-cursor failure on the two sortless lists GET /workflows/{id}/versions and GET /workspaces/{id}/members threw a bare 'Invalid cursor' literal where every other v2 list uses a shared constant. The right one is UNREADABLE_CURSOR_MESSAGE, not INVALID_CURSOR_MESSAGE: both lists take only limit and cursor, so naming sortBy/sortOrder would answer one 400 with advice that earns a second. Their missing filter scope is correct and stays. Neither contract accepts a filter — v2PaginationFields is the whole query — so there is nothing to bind, and limit is excluded from a scope by design. Pins the message on the versions route, verified to fail against the literal. * test(openapi): give the determinism check a chosen timeout `serializes all documents deterministically` serializes all seven published documents twice — roughly 2MB of JSON — under vitest's 5s default, which is not a budget anyone picked for it. The published specs grew 3.3% on this branch (961KB -> 993KB) from richer descriptions, which is far too small to move a comfortable test and is enough to tip one already sitting just under the cap. Measured at 5.1s in isolation with nothing else running. Raises it to 30s for the openapi suite rather than trimming a real assertion. * fix(v2): make the NUL path scan linear, and force a write surface to choose Two findings from a simplify pass, both in code this branch added. findNulBytePath copied `[...path, key]` per child, which is O(nodes x depth). A caller controls that depth directly: v2 row cell values are `z.unknown()`, so nesting passes Zod untouched and reaches the scan. Measured on Node 22 -- JSON.parse accepts a 200KB body nested 100k deep in 9.8ms, and the scan then blocked the event loop for 27.7s. Frames now carry a parent link and the path is materialized once, for the node actually reported: 27.7s -> 5ms, with byte-identical paths across nested arrays, records, NUL keys and clean input. The always-run first pass drops Object.entries for Object.keys, which halves its cost on large bodies by not allocating a pair array per object. `strictWrite` was optional with the lenient default, so a v2 write route added tomorrow would silently inherit first-party behavior -- unknown column dropped under a 201, uncoercible cell stored as null -- defended by nothing but five copies of a literal. It is now required on the five write-shaped inputs, so omission is a compile error. The type-checker named every caller: the five v2 routes already passed true, and the three Copilot sites now say false explicitly, which is the behavior they already had. * refactor(v2): apply the body-413 mapper to every OpenAPI document The earlier unification wired withRequestBodyErrors into two of the five content documents and left files-audit, knowledge and workflows hand-writing the entry, so the helper's own claim that "a new body route cannot forget it" held on 40% of the surface while reading as global. Regenerating all seven specs produces zero drift, which is the useful proof: the mapper agrees with every hand-written entry today, so the gap was never a missing 413 — it was a missing guarantee for the next body route added to those three documents. The existing hand-written entries stay. The mapper is one-directional and several bodyless folder reads publish 413 for the folder-tree ceiling, so stripping them by hand would risk removing one the mapper cannot restore. * refactor(v2): fold the v2 validation renderer into the shared parse defaults V2_PARSE_DEFAULTS calls itself "the parse failures every v2 route renders the same way", but the option deciding how a v2 validation failure renders sat outside it and was re-stated at seven sites. A raw route that spread the defaults and stopped emitted a non-v2 error envelope. Removes the redundant line from the five sites that only restated it. The two builders keep theirs: theirs sits after `...options.parseOptions`, so it is a deliberate override that stops a caller swapping the v2 renderer, not a copy. Also adopts the mandated `filterUndefined` in cursorScopeKey in place of the Object.fromEntries/Object.entries form CLAUDE.md forbids, and collapses a one-element `as const` array plus a Math.max over it to the single `.length` they computed. * test(persistence): keep the wire round trip without tripping the utils audit check:utils forbids `JSON.parse(JSON.stringify(...))` and points at structuredClone, which is right for a deep clone and wrong here: this test exists to prove the schema accepts a `deployedAt` that arrived over HTTP as a string as well as an in-process `Date`. structuredClone preserves the `Date`, so adopting it would leave the test asserting nothing about the wire form. Splits the serialize and the parse into two statements. The round trip stays lossy — verified `JSON.parse(JSON.stringify(...))` yields a string where structuredClone yields a Date — and the pattern the audit matches is gone. Arrived from staging in #6660, so `check:audits` is red on origin/staging too, not only here.	2026-08-13 10:52:20 -07:00
Waleed	29853fbbcf	improvement(docs): make the API reference read as code and unify its type token (#6653) ... * improvement(docs): make the API reference read as code, and unify the type token The API-page font override matched every span/div/p inside the page, which outranks the .font-mono class on specificity, so every parameter name, type, and identifier silently rendered in the body sans face. Exclude .font-mono so code tokens stay monospace. Consolidate the three divergent type-slot treatments — plain scalar, union, and schema reference each carried their own chip definition, differing in size, weight, face, and box height — onto one code token that reuses the docs inline-code recipe and the platform's 20px chip height. Demote the row metadata: 'required' and 'header' were filled pills, 'required' on the error token, making a constraint the loudest element on the page and a page of required parameters read as a page of alarms. Both are now uncontained text, leaving the type token as the only box on the row. Pin the two 'application/json' labels to one treatment; the Request Body and Response headers rendered the same string at different weights and faces. * improvement(docs): mono status-code tabs, and match fumadocs' lucide icons to emcn Status codes in the example panel are numeric literals and render as code everywhere else on the page, including the Response header's own trigger, but fumadocs rendered the strip in the body sans face. Language tabs sit in a separate container and stay sans — those are product names, not code. fumadocs draws a few lucide glyphs on API pages that its client-component overrides do not expose (the heading anchor and the code-block copy button). emcn strokes at 1.55 and lucide at 2, so those icons read heavier than every icon around them; match the weight. * fix(docs): align the auth type chip with every other property row, and wrap example code The auth row collapses its real `<token>` type and renders the chip through ::after, so the span is only a wrapper — but it still matched the type-token rule and kept that rule's border, height, and gap. The border drew a second empty box around the real chip and the gap opened in front of it, because the collapsed text remains an anonymous flex item; together they pushed the chip right by roughly 8px that no other row had. Example-panel code overflowed sideways instead of wrapping: fumadocs sizes the block with `w-max`, so it grew to its longest line inside a 400px scroller and the existing pre-wrap never applied. Cap the width, switch break-all to overflow-wrap anywhere so only unfittable tokens split, and reserve room for the copy button fumadocs floats over the first line. * revert(docs): let example code overflow instead of wrapping Wrapping restarts every continuation line at column zero, and in a JSON body indentation is what carries nesting depth — so a wrapped response misreports its own structure. A hanging indent keeps the depth but needs the shiki lines forced from flex rows to blocks, which breaks the line rhythm. Removes the pre-wrap rules rather than repointing them: fumadocs sizes the block with w-max, so the previous rule never took effect and overflow was already the behaviour on the page. * fix(docs): tighten array type tokens and keep the union separator legible An `array<T>` slot holds its angle brackets as bare text nodes, which become anonymous flex items, so the slot's gap prised `array<` and `>` away from the type they wrap. Drop the gap and let the union separator carry its own margin; this also makes the auth row's gap override redundant. The separator was dimmed twice, by a muted token and again by opacity, which on the dark chip fill left `string | null` reading as `string null`. * fix(docs): restore the hidden API key description, and drop dead API-reference CSS The rule hiding the trailing `In: header` line matched `p:has(> code)`, which is a shape, not a target — every scheme description in our specs cites a status code, so the whole explanation of personal vs workspace-scoped keys was display:none on every API reference page. Match the last child instead, and shorten the description to one line now that it renders. The dropdown trigger's hover rule had been left below a new id-qualified base rule that outranked it, so the trigger could no longer change colour on hover. Removes what does not run: the four `::-webkit-scrollbar` rules (specifying a non-auto scrollbar-width makes Chromium ignore them, and Firefox never had them) and an `order: 2` block whose selectors and declaration the type-token rule above it already carried. Names the two values the API reference repeats — the monospace stack, written out eleven times, and the 12.5px code size, written nine — as --font-mono-stack and --text-code. Also drops four !important declarations that already won on specificity, a --text-muted fallback that can never fire, and a lucide selector subsumed by the one beside it. * refactor(docs): define the API-reference chrome once, and cut the commentary The metadata face — size, leading, weight, mono stack — was written out in seven rules that a comment asked future readers to keep in sync by hand; it is now one rule those seven consume, each adding only its own colour, content, and order. The auth row's chip likewise re-derived all eleven declarations of the type token and now joins that rule, keeping only its label. Comments were running longer than the rules they documented — 88 added comment lines against 73 declarations. Trimmed to the load-bearing facts: cascade traps, browser behaviour, and the bugs a rule prevents. Dropped the block narrating why the wrap rules were reverted, which duplicated its own commit message. Also retires a scrollbar token left unreferenced by the webkit removal, moves the last two fumadocs colours in our own components onto platform tokens, and brings the callout icon to 1.55 so the docs really do have one icon weight. * fix(docs): keep the union separator in the type token's own face The `|` between union members is a classless span, so the page-wide `span:not(.font-mono)` rule assigned it the body sans face while the members beside it stayed mono — one chip rendering in two faces. Applies the inherit reset to every descendant of a type token rather than just its links, so anything fumadocs nests there later is covered too.	2026-08-12 19:27:32 -07:00
Waleed	1fa40b8118	feat(v2): complete and align the v2 API surface (#6643) ... * fix(v2): close four validation holes in the logs and billing surfaces Each of these answered a caller-supplied value with a 500 or a silently wrong result instead of a 400. - `GET /api/v2/logs` accepted any string as `startDate`/`endDate`. The route constructs a `Date` from it, so `?startDate=abc` reached the driver's timestamp mapper as an `Invalid Date` and 500'd. Both bounds now carry `.datetime()`, matching the sibling run list so one timestamp works on both collections. This narrows the accepted set: a date without a time and an offset-bearing timestamp are now rejected, and the field descriptions say "UTC ISO 8601" rather than overpromising "ISO 8601". - `v2BillingStatusQuerySchema` was the only non-strict query schema in its family, so a mis-cased `workspaceID` was stripped and the caller got account-scope billing in place of the workspace scope it asked for — a wrong answer about money, served as a 200. - An unresolvable `cursor` on `/api/v2/billing/logs` applied no cursor condition and restarted the sequence at page 1 while still reporting `hasMore`, so a pager holding a cursor across a deploy loops over the first page and counts the same credits on every lap. It is now a 400. The message does not reuse `INVALID_CURSOR_MESSAGE`, which names `sortBy`/`sortOrder` params this collection does not accept. - The logs `status` field disagrees with the run resources for the same run: the run projection overlays `paused` from `paused_executions`, so an ordinary human-in-the-loop pause reads `paused` there and `pending` here. Reconciling would mean joining `paused_executions` in this read and silently moving live runs between two buckets of a shipped field, so the divergence is documented on the contract instead. * feat(v2): expose the MCP tool plane and page the MCP server list Registering an MCP server through v2 dead-ended: nothing on the public surface ever ran tool discovery, so connectionStatus, toolCount, lastError, and lastToolsRefresh stayed at their registration defaults and there was no way to read a server's tools without opening the UI. Adds GET /api/v2/mcp-servers/{id}/tools over a thin use case composed from the existing mcp_servers.tools.discover operation, resolveServerContext, and mcpService.discoverServerTools. It is personal-API-key-only — discovery resolves the acting user's own OAuth credentials, which a workspace key cannot supply — and the contract says so rather than letting callers meet an unexplained 403. Discovery failures are classified instead of collapsing into a 500: an unreachable or cooling-down server is a retryable 503, a stale OAuth grant is a 401. Also pages GET /api/v2/mcp-servers. It was the one unbounded list on the v2 surface, classified full-set on a bounded-by-construction rationale that only holds for folder lists; nothing caps how many servers a workspace registers. * feat(v2/tables): strict row bodies, a filtered row count, and round-trippable required columns Three tables gaps from the v2 capability evaluation. Strictness. Every v2 tables request body is now `.strict()`. The row family was the whole hole: `POST /query` sent v1's `filter` key answered 200 with a fully unfiltered page, because Zod strips unknown keys unless told not to. The same laxity covered the row create/update/delete/upsert/find bodies, the run and cancel-runs bodies, the enrichment body, and — outside the row family but the same class — the column delete, view create/update, and export bodies. A contract sweep now walks every body-bearing tables contract and fails if one of them stops rejecting an unrecognized key. Filtered row count. `POST /api/v2/tables/{tableId}/query/count` answers the question v1's `includeTotal`/`totalCount` answered and the `{data, nextCursor}` envelope has nowhere to put: how many rows a predicate matches. It binds the existing `queryTableRows` use case with `includeTotal: true, limit: 1` — no new domain logic and the same `tables.rows.query` read policy. The use case types `totalCount` as nullable because paged callers can decline it; this route always asks for it, so a null is treated as a broken invariant rather than presented as a fabricated zero. Required columns. `required` is accepted on create-table, add-column, and update-column, matching v1. v2 emitted the flag on every read while stripping it from every write, so a column could not round-trip. Enforcement was already complete: turning it on over rows with null, missing, or empty cells is rejected by the domain. * test(skills): pin the workspace-API-key split as structural, not accidental A workspace API key can create a skill it can then never update or delete, which no sibling resource does — so the asymmetry reads like an oversight worth widening. It is not. Skill edits are authorized by the per-skill editor row belonging to the acting user, which is why update/upsert/delete declare a 'read' floor rather than 'write': workspace role is not the authority. A workspace key carries no user subject, so allowing one replaces a 403 with an unclassified PrincipalSubjectUserRequiredError that the v2 surface renders as a caller-reachable 500. Records the reason on the registry and pins it, so the next reader finds the argument instead of flipping the flag. * feat(v2): read deployment state, and undo a file delete Two v2 reads that existed only as a side effect of a mutation. `GET /api/v2/workflows/{id}/deployment` publishes the state the deploy, undeploy, and rollback responses carry, plus `needsRedeployment` — which those responses structurally cannot carry, because they answer at the moment the draft and the live version are equal. A caller that lost the mutation response, or that polls from another process, had no way to ask. Reuses `readWorkflowDeploymentStatus` behind `workflows.read`, the same use case the internal status and deploy GETs already adapt. `DELETE /api/v2/files/{fileId}` was a soft delete with no way to see what it archived and no way to reverse it. `GET /api/v2/files?scope=archived` pages the archived set and `deletedAt` on the file resource dates each one; `POST /api/v2/files/{fileId}/restore` reverses the delete through the existing `files.restore` operation. Restore is not a pure undo — it returns the file to the root and renames it on a collision — so the use case now reads the file back and both the response and the OpenAPI description say what actually came back rather than what was deleted. `scope=all` is rejected on the list for the reason the internal contract already gives: it drops the `deleted_at` predicate and cannot use the partial index. `scope=archived` combined with `folderPath` 404s when the containing folder was archived too, which the contract documents. * fix(v2): keep the unresolvable-cursor rejection a 400 on every surface The cursor rejection lived in shared billing core but was an OrchestrationError only, which the session-only GET /api/users/me/usage-logs cannot project: that route is raw withRouteHandler and readTypedError matches instanceof HttpError, so any signed-in caller typing ?cursor=x got a 500. UnknownUsageCursorError is an HttpError carrying the OrchestrationError as its cause, so the v2 route still renders BAD_REQUEST off the cause chain and the internal route answers 400. Also closes the other half of the run-list parity: an inverted window on GET /api/v2/logs is now a 400 instead of a silently empty page. * fix(v2/tables): sweep union bodies per member and name the shapes on a rows 400 Review follow-ups on the strictness work. The sweep was vacuous on the one union body it covers. Parsing `{ notAContractField: true }` against `v2CreateTableRowsBodySchema` and looking for `unrecognized_keys` anywhere in the issue tree is satisfied by either member alone, so dropping `.strict()` from the single-row branch shipped green — reproduced, 36/36 passing with the regression in place. The sweep now flattens a union body into its members and asserts each one separately; removing `.strict()` from either branch now fails a case that names it. `POST /rows` answered an unknown key with `Invalid input`, the exact message the v2 conventions name as failing the actionable-error rule, because a union surfaces `invalid_union` first. The union now carries a message naming both accepted shapes; the per-member failures still ride along in `details`. Two TSDoc corrections. The `required` docstring claimed the domain rejects turning the flag on over rows with empty cells — true of the update path, false of add-column, which applies the flag as given (the same shape `unique` already had here). And `.strict()` binds the top level only, so the view `config` object and the shared sort-spec elements still strip unknown keys; both docstrings now say so instead of implying full coverage. * fix(v2): classify MCP discovery failures by type, not by substring The tool-discovery error policy consumed categorizeError's status, whose fallback is a substring match on the upstream message. Three consequences, all caller-visible: - A ZodError from the builder's own response `.parse` contains `invalid_type`, so a Sim-side response-schema defect answered 400 "Invalid request parameters" and suppressed the builder's 500 and its unhandled-error log. - An upstream `Invalid params` or `not found` became the caller's 400/404 on a request the contract had already validated. - A stale OAuth grant to the third-party server answered 401, the status this surface reserves for a missing or invalid Sim API key, so a client would rotate a credential that was never the problem. The policy now dispatches on the MCP error families and returns null for anything else. Reauthorization is a 409 carrying `details.code: MCP_SERVER_REAUTHORIZATION_REQUIRED`; an unreachable, slow, or cooling-down server is a 503 with a constant message. Also: widen the shared server path-param description now that it covers tool listing, map the list query explicitly so no undeclared `cursor` reaches the use-case input, and document the endpoint's write side effects. * merge: bring in the MCP tool plane workstream * feat(v2): make knowledge tags usable and let documents be updated v2 accepted tag slots on upload and filtered search by tag display name, but no response ever returned a tag value and nothing listed the vocabulary, so a shipped feature dead-ended in the public API. A document that failed processing could only be deleted and re-uploaded, and retiring 500 documents cost 500 requests. - GET /api/v2/knowledge/{id}/tags returns the vocabulary (display name, slot, field type) as a full-set list. - Document list and detail responses carry `tags`, keyed by display name exactly as search keys its result metadata. Writes stay slot-keyed; the tags endpoint is the mapping and the contract documents the split. - PATCH /api/v2/knowledge/{id}/documents/{documentId} renames, enables, disables, retags, or requeues processing. Derived indexing state is not writable: asserting `processingStatus` on an unindexed document would corrupt search. A retry may not ride along with field updates. - PATCH /api/v2/knowledge/{id}/documents bulk-enables or bulk-disables. Bulk delete is deliberately absent — that operation records no semantic audit, and a public bulk delete would empty a knowledge base leaving no DOCUMENT_DELETED entries. - The document list accepts the same name-based `tagFilters` as search; the name-to-slot resolver moves out of search into a shared helper, and the filters are stamped into the offset cursor scope so a replayed cursor cannot cross a filter change. - Search accepts `rerankerEnabled`, `rerankerModel`, `rerankerInputCount` and returns `rerankerScore`; `rerankerApiKey` and `skipUsageBilling` stay unexposed. Every result now names its `knowledgeBaseId`. knowledge.tags.list flips from workspaceApiKey 'deny' to 'allow' (and gains the workspace_api_key principal kind) so it matches the sibling reads knowledge.documents.list / read / search. The vocabulary is required input for two operations a workspace key can already perform. Every tag write stays human-delegated. * fix(v2): name every 403 cause, unfork boolean params, close nested strictness holes Four cross-cutting consistency gaps on the v2 public surface. **403s now carry a machine-readable cause.** The conventions skill mandated `error.details.code` on 403 and nothing emitted one, so a client had to string-match prose to tell "raise this member's role" from "this workspace refuses personal keys" from "buy an enterprise plan" — four different remedies behind one status, and every message reword a silent break. The vocabulary is a closed set, `FORBIDDEN_DETAIL_CODES`, with a `Record` of descriptions beside it that the generated OpenAPI 403 description is built from, so a code cannot reach the wire unpublished. Refusals throw `ForbiddenOperationError` in the domain and `v2CaughtOrchestrationError` — the function every v2 error policy falls through to — attaches the code, so a route cannot forget it. The audit-log resolver distinguished four causes and collapsed them into one; it now names each. Cross-tenant refusals deliberately get no code: they are concealed as 404 and naming their cause would hand back the existence signal the concealment withholds. **Two boolean query params rejoin the majority.** `?includeDeparted` and `?includeOutput` were `'true'`/`'false'` string enums inherited from the internal shapes they reused, while four sibling params were real booleans. Both move to `booleanQueryFlagSchema`, which still coerces both strings — a strict widening, so an existing caller is unaffected, and the spec stops telling callers to send a string. **Two nested strictness holes close.** `.strict()` binds the top level only, so `sort: [{ field, direction, nulls: 'last' }]` was answered 200 with the null-ordering request dropped, and an unknown key inside a saved view's `config` was accepted and discarded — the headline `filter` bug one level down. `sortSpecSchema`'s element and both view-config schemas are now strict. Safe on the read side because `normalizeStoredViewConfig` projects the schemaless stored blob onto the declared keys first, so a legacy row cannot turn into a 500. The two sort dialects stay as they are. `/logs` and `/workflows/{id}/runs` have one sortable column, so there is no `sortBy` to pair with; renaming `order` breaks every caller and an alias is a second spelling of one thing with undefined precedence. Both contracts and the skill now state the rule. * style: format the files the workspace-scoped lint gate does not reach `turbo run lint:check` runs `biome check .` per workspace, so `scripts/` at the repo root is outside the graph and four changed files were unformatted — one of them a merge artifact from reconciling the route baseline across branches. * fix(v2): collapse the four knowledge document projections onto one null-tolerant summary Extracts toV2DocumentSummary in app/api/v2/knowledge/utils.ts and composes the list, upload-acknowledgement and detail presenters from it. toV2TaggedDocument serialized uploadedAt with a bare .toISOString(), so a document with no upload timestamp threw where every sibling returned null and the contract declares the field nullable. Also consolidates the two Zod strictness walkers onto one shared introspection helper that unwraps wrappers and expands unions, closing the hole where a union-shaped schema answered null and was skipped by the pagination sweep. * fix(v2): stop HEAD driving MCP discovery, and unbreak the updatedAt keyset page B1: Next aliases HEAD onto GET, which RFC 9110 permits only because GET is safe. The MCP tool-discovery GET is not: it opens a live connection to the registered endpoint and writes the outcome onto the server row. The v2 JSON builder gains a headSafe option, default true, and the discovery route declares itself unsafe — a HEAD is authenticated and rate-limited, then answered bodiless. B2: a discovery status write stamped updatedAt, which this branch added as a keyset sort, so any concurrent discovery duplicated and skipped servers across a caller's pages. Discovery liveness already has lastConnected, lastToolsRefresh, lastError and statusConfig. B4: a public refresh now skips the positive cache but keeps the failure cooldown, so it cannot be used to drive a connection attempt per request at a failing endpoint. An explicit user action on their own server keeps the full bypass. B6: the consecutive-failure counter is incremented SQL-side rather than read, incremented and written back, and the success branch carries the same workspace, liveness and staleness guard the failure branch already had. * fix(v2): bound the bulk update echo, close the search leak, and make the docs true B3: a selectAll bulk document update echoed every changed identifier, which the request does not bound — a 100k-document knowledge base produced a multi-megabyte array, materialized and then element-wise validated. The use case now reports whether the selection was unbounded and the presenter omits the echo. A1: the knowledge search presenter spread the whole use-case result, which also carries userId, workspaceId, a cost breakdown and a live secret-trace registry. Only Zod's default key-stripping kept them off the wire. Projected explicitly. P1-a: GET /knowledge/{id}/tags advertised all 17 slots while the document PATCH accepted only the seven text ones. The writer already coerces every slot type, so the PATCH now takes all 17 in their declared types, with a 400 where a malformed value used to silently clear the tag. P1-b: both new PATCHes deny workspace API keys and now say so. P1-c: the two table query reads declare maxBodyBytes and now document the 413. P1-d: getWorkflowDeploymentV2 loses its legacy suffix. C3: deletes two orchestration error mappers with no callers that mapped 'forbidden' with no details. D2: a stored null in table_views.config survived the pick and failed the response schema. Also folds the six 'bounded set' paraphrases onto one FULL_SET_LIST constant, shares the run-window date bound between the logs and runs lists so their documented parity is enforced rather than asserted, adds the missing barrel export for FORBIDDEN_DETAIL_CODE_DESCRIPTIONS, and strictens two response schemas whose peers were already strict. Migrates 40 v2 route tests onto the shared @sim/testing harness: 26 asserted a rateLimitSubjectIds shape v2 auth never returns, 26 asserted the wrong refillRate, 33 could not exercise their 401 path at all, and 6 hard-wired the rollout gate to null. * fix(mcp): bound the connect handshake, and stop the 403 description over-claiming B5: the connect clamp was getMaxExecutionTimeout(), the workflow ceiling of seven days, so the real bound became the server row's own timeout — which the registration contract permits up to 300s — times the connect retries. A slow server could hold a Node request for roughly twenty minutes. Connecting is not a workflow run, so the handshake now shares the one-minute ceiling tools/list already applies to itself. C2: the generated 403 description asserted that error.details.code names the cause on every 403. Nine domain refusals still throw a bare forbidden OrchestrationError and reach the wire codeless, so the wording now says 'where the cause is one a caller can act on'. Reparenting those throws is left as a deliberate change: one of them is a cross-tenant refusal that belongs in the codeless class and would change its status. * chore: reconcile the route ratchet with staging * style: sort imports and format the three files biome flagged * fix(openapi): import the forbidden-code constants from their module, not the application barrel The barrel also re-exports the authorized use-case layer, which loads @sim/db at import time. That pulled a database connection into the OpenAPI spec check, so check:audits failed wherever DATABASE_URL is absent, including CI.	2026-08-12 15:04:24 -07:00
Waleed	6541a22aa6	fix(v2): tell a caller when to come back on every failure meant to be retried (#6625) ... * fix(v2): tell a caller when to come back on every failure meant to be retried Three related gaps in retry signalling, found auditing the v2 surface against RFC 9110/6585 and against how Stripe, GitHub and Google's AIPs handle the same problems. **No 503 carried `Retry-After`.** Every one of them — the three route builders' `unhandledErrorResponse`, the execute and resume routes, and `serviceFailureResponse` — funnels through `v2Error`, so the default lands there, keyed on the response *status*: `Retry-After` is defined against the status, and the status is the only half of the code/status pair a client sees. A caller that supplies its own value still wins. RFC 9110 §15.6.4 makes this a `MAY` rather than a `SHOULD`, so it is a deliberate improvement, not a conformance fix: without it a client's only defensible policy on a 503 is an immediate retry, and Sim raises 503 exactly when a dependency is too degraded to absorb one. **A 429 that already knew its wait threw it away.** The admission descriptors declare `retryAfterSeconds` per denial, but mapping a descriptor onto a preprocess error copied only `statusCode`, `code` and `retryable`. A concurrency denial therefore reached the client as a bare 429 with no `Retry-After` despite the policy layer having named the wait five seconds earlier. The value now travels `descriptor.retryAfterSeconds` → `PreprocessExecutionError.retryAfterMs` → `ExecuteWorkflowServiceFailure.retryAfterMs` → `serviceFailureResponse`, so the transport reads a number the policy owns instead of re-guessing one. The 503 default is now only the floor for paths with no policy signal. **One failure must not advise a retry at all.** `ASYNC_ENQUEUE_AMBIGUOUS` is a 503 whose enqueue may have succeeded — it deliberately retains its execution-ID claim because a job may already exist. Telling that caller to come back in five seconds invites a client with no `X-Run-Id` to start, and bill, a second run of the same workflow. It opts out via `omitRetryAfter` and returns the run id so the caller reconciles instead. `ADMISSION_RETRY_AFTER_SECONDS` is reused rather than restated, so the execute route's capacity 429 and every other surface's 503 cannot drift apart. Also records the audit in `.agents/skills/v2-api-conventions/SKILL.md`: the retry rule, the cursor-tampering invariants, and reasoned rejections of RFC 9457 problem+json, the `RateLimit-*` draft fields, renaming `X-RateLimit-*` under RFC 6648, 422-for-semantic-validation, `Location` on 201, ETag/`If-Match`, and `merge-patch+json` — each with the spec text and the industry evidence, so they are not re-litigated. `Deprecation`/`Sunset` on v1 is left open pending a retirement date, which is a product decision. * docs(v2): name the one 503 that omits Retry-After in the shared contract The shared ServiceUnavailable description claimed every 503 carries the header, which the ASYNC_ENQUEUE_AMBIGUOUS response deliberately does not. It now says the header is normally present and names that exception, so the published contract matches the runtime behaviour for all 128 operations.	2026-08-12 11:09:23 -07:00
Waleed	5cf60f6a41	fix(v2): give every collection one pagination contract and close four envelope holes (#6620) ... * fix(v2): give every collection one pagination contract and close four envelope holes A fractional `limit` reached Postgres as `LIMIT 2.5` and answered 500 on both `GET /workflows` and `GET /audit-logs`: each list re-declared the param inline, and these two copies lost their `.int()`. The same divergence left `limit` validated five different ways and five collections emitting `nextCursor` while accepting no `limit` at all, or accepting one and silently discarding it. Adds `v2PaginationFields()` in `contracts/v2/shared.ts` — a bounded integer `limit` and an opaque `cursor` — and adopts it across all 17 paged lists, so the family cannot drift again. `/files`, `/logs` and `/tables` keep the truncate-and- clamp leniency they published, now as an explicit named mode rather than three hand-rolled copies. Gives `/skills`, `/custom-tools`, `/secrets`, `/credentials` and `/knowledge` real pagination using the existing cursor codecs: a keyset for the four whose page comes from one ordered SQL read, and the offset cursor for `/skills`, whose merge of the static builtin registry into DB rows cannot be expressed as a SQL keyset. Each keyset sort now ends in a unique `id`; knowledge tie-broke on `createdAt`, which cannot separate rows sharing a millisecond. Two correctness fixes pagination forced: the secrets visibility filter moved from a post-query JS pass into SQL, because trimming rows after the page is cut returns fewer than `limit` while `nextCursor` claims more; and the skills list stopped selecting the 50k-char `content` column only to discard it. Also restores the canonical error envelope where it had holes: a malformed JSON body returned a bare `{"error":string}` because the envelope was a per-route opt-in only 8 of 77 routes remembered, and an unknown `/api/v2` path returned an HTML 404. Both are now defaults — `V2_PARSE_DEFAULTS` on the builders and the two raw routes, and a catch-all whose body is byte-identical to the rollout gate's so an unknown path stays indistinguishable from an ungated one. Consolidates the keyset paging block (`resumeKeyset`/`keysetPage` in `list-query.ts`) that had been open-coded in six modules, and folds the bespoke `InvalidWorkflowListCursorError` into the `OrchestrationError` every other list already used. Prevention: the contract sweep in `list-pagination.test.ts` now also asserts that every paged list rejects a fractional `limit`, that every list query is `.strict()`, and that the three clamping lists still truncate. The fractional- limit assertion is what caught `/audit-logs`. Documented in `.agents/skills/v2-api-conventions/SKILL.md`. 405 responses still carry no `Allow` header — Next.js generates those before any handler runs. Recorded as a known gap. * fix(v2): bind the offset cursor to the query state it counts positions in An offset names a position in one exact sequence. `GET /skills` accepted a bare `{offset}` cursor and applied it to whatever sequence the next request asked for, so following `nextCursor` with a different `search`, `sortBy` or `sortOrder` silently skipped rows, repeated them, or landed past the end and returned an empty page while the cursor implied more. Fixed in the codec rather than the route so the sibling could not keep the gap: `decodeOffsetCursor` now takes a scope stamp and rejects a cursor minted under a different one, which is what `decodeSortedCursor` has always done for keysets. `offsetCursorScope()` builds the stamp from every param that filters or orders the sequence; `limit` is excluded because it selects how much of the sequence to return, not what the sequence is, so paging with a different page size still works. `GET /knowledge/{id}/documents` had the identical latent gap and gets the same treatment — the compiler surfaced it as soon as the signature changed.	2026-08-12 09:08:00 -07:00
Waleed	366829b6b0	fix(v2): derive the log and run status enums from the persisted status list (#6612) ... * fix(v2): derive the log and run status enums from the persisted status list `GET /api/v2/logs` and `GET /api/v2/logs/{runId}` parse the raw `workflow_execution_logs.status` column against a six-value enum that omits `paused`, so a run holding that value returns 500. The list response is validated whole-page, so one such row 500s every page it lands on, and the row is durable until the run is resumed, cancelled, or failed. `paused` is not written by an ordinary human-in-the-loop pause — that path persists `pending` (logging-session.ts:1180). It is written by `PauseResumeManager.markResumeAttemptFailed`, which fires on any `ResumeAdmissionError`: a workspace over its usage limit, an archived or undeployed workflow, or a concurrent resume losing the claim race. That is a routine business path. The enum was supposed to be protected by an `AssertNever` exhaustiveness gate, but the gate was vacuous: it compared against `PersistedWorkflowExecutionStatus`, a hand-written union that was itself missing `paused`, because the write goes through a raw `sql` CASE fragment Drizzle cannot type-check. Adding `paused` to both lists would leave the same vacuous gate in place for the next status. Instead, `PERSISTED_WORKFLOW_EXECUTION_STATUSES` becomes the single runtime source of truth, `PersistedWorkflowExecutionStatus` is derived from it, and both v2 contracts derive their enums from the const rather than re-declaring them. Both surfaces pass the column through verbatim, so their reported set is the persisted set by definition — there is no editorial choice for a gate to force, only the question of whether a newly persisted status should be public, which the option-list tests now pin. The `[...V2_PERSISTED_RUN_STATUSES, 'paused']` append on the runs contract is deleted rather than adjusted; it would otherwise be a duplicate. Alternatives rejected: - A `.catch()` or `safeParse` in the presenters is dead code: `v2-json-route.ts:271` re-parses the whole body with the same schema. - Normalizing `markResumeAttemptFailed` to write `pending` would remove the distinction the resume claim query at human-in-the-loop-manager.ts:973 relies on, and leaves the contract wrong for any other future status. - Typing the Drizzle column does not help: the offending write is a raw `sql` fragment, and `packages/db` cannot import the app's status list. The v2 workflows spec changes are reordering and description only — the value set there already contained `paused`. The v2 logs spec gains `paused`, which is additive and safe while the whole `/api/v2` surface is behind the off-by-default `v2-api` flag; it must land before v2 GA, after which it would be breaking. * fix(v2): document both provenances of a reported paused run status * fix(v2): stop promising a paused discriminator the response cannot always provide * fix(v2): describe the paused discriminator as the code actually records it	2026-08-12 01:51:55 -07:00
Theodore Li	23318a13a0	fix(security): redact workflow snapshot secrets (#6581) ... * fix(security): redact opaque workflow snapshot inputs * fix(security): document fail-closed tool redaction * fix(security): redact malformed tool params * fix(security): redact nested credential references * fix(security): isolate opaque tool schemas	2026-08-11 23:14:15 -04:00
Waleed	7c05e36049	fix(api): close five defects found auditing the v2 migration against main (#6575) ... * fix(tables): stop a column retype from nulling empty-string cells A type conversion rewrote every cell holding '' to null. Main only nulled a blank the target type could not read; '' is a real stored value that both string and json columns accept, so string->json and json->string silently destroyed those cells. Worse on a required target: countEmptyCells matches only a missing key, SQL NULL, or '[]', so '' passes the required guard and the rewrite then wrote null behind a constraint that had just succeeded. The per-cell decision is now the pure retypeCellRewrite, restoring main's rule: null a blank only when the target cannot read it, otherwise coerce. * fix(execution): release the concurrency slot when a group cancel is refused The stop-the-work effects (durable Redis abort record, queue-job cancel, in-process abort) all fire before the workflow-group sidecar is consulted, and none can be undone. When the sidecar refuses the claim we throw a conflict, which skipped releaseExecutionSlot and stranded the plan concurrency reservation until it expired. Every conflict return is a terminal-or-absent state - a missing log row, a log already completed or errored, or a terminal cell - so a refusal never means the run is still executing. The slot is released before the throw, keeping the exact success && !isPausedCancellationPath predicate rather than a blanket finally that would free reservations for live runs. * fix(uploads): recover an ambiguous PUT instead of discarding the object Main recovered an upload whose bytes committed but whose response was lost, via a verify endpoint. The session client retries the PUT instead, but every provider now signs a create-only precondition, so the retry returns 409/412, is classified non-retryable, and the session aborts - deleting the object that had already landed. A transient blip on the final ack cost the whole upload. A conflict on a retry attempt is now treated as our own earlier PUT having committed, and completion proceeds. That is safe because completeUploadSession independently verifies the object through assertObjectIdentity, which rejects on uploadId mismatch before anything durable is registered. A first-attempt conflict still fails loudly. * fix(folders): enforce the workspace folder ceiling on the create path Readers bound the active path index at MAX_FOLDERS_PER_WORKSPACE and throw once a workspace exceeds it, but POST /api/folders reached createFolder, which has no maxFolderRows field and never counts. A workspace could therefore be driven past the ceiling, after which the 27 capped read sites failed on a state the product had allowed. createFolder now asserts room inside its transaction, right after the mutation lock, so the count cannot be raced. The refusal is a typed conflict rendering 409 with an actionable message rather than a 500. The check counts rows directly instead of loading the path index, so an already-over-cap workspace gets a clean refusal rather than a read error, and no reader gained a cap. folderMutationStatus also gained the payload_too_large mapping it was missing, which had been rendering a delete-cascade cap breach as an unexplained 500. * fix(skills): route internal skill writes through the shared use cases The internal route made the workspace authorization decision itself, never consulting the skills operation policy, never loading canonical workspace context, and recording an audit entry with no operation id or actor projection. v2 and Copilot already went through the use cases; only this surface did not. GET/POST/DELETE now authenticate, parse, call the shared use case, and present. Request and response shapes are unchanged. Two behavior changes fall out: a write against a deleted workspace is now refused with 404 rather than accepted, and permission-denial text matches the rest of the platform. Legacy internal-JWT auth is dropped because no principal kind expresses that caller and nothing calls it: the whole repo references /api/skills only in two comments, no tool declares an internalRoute to it, and the executor reads skills through a direct listSkills call rather than over HTTP. * fix(folders): enforce the workspace ceiling on the remaining create paths Folder duplication, admin workspace import, and workspace forking all inserted folders without consulting the ceiling that 27 read sites enforce, so any of them could leave a workspace whose reads then fail. Each now asserts room for the rows it is about to add rather than one at a time: duplication measures the whole subtree up front, forking counts its bulk insert, and import counts per segment because that is genuinely one row. assertFolderCollectionHasRoom gained an additionalRows notion for the bulk case, and short-circuits when nothing is being added so an over-cap workspace still reads and still syncs. Duplication deliberately does not take the folder mutation lock. Holding it across the copy would block folder creation workspace-wide for an unbounded time - there is no cap on workflows per subtree and duplicateWorkflow runs sequentially - and narrowing it is impossible because an advisory transaction lock cannot be released early; splitting the transaction would leave a half-copied tree on failure. A rare few-row overshoot near the ceiling is the better trade, and it matches what forking already does. A test asserts the lock is absent so re-adding it is a visible decision. Admin import gained the transaction and lock it never had. Its folder-full refusal escapes the per-workflow result list, because a full tree is a property of the workspace and would otherwise be buried as N failures behind a 200. The fork and promote routes had no catch at all, and withRouteHandler only classifies HttpError, so a refusal rendered as an opaque 500 - twice over, since drizzle wraps the throw. Both now project a classified conflict as 409 and rethrow anything unclassified. * fix(uploads): bound the signed PUT lifetime and advertise its real expiry A single-PUT transfer was signed for the whole 24h upload-session TTL, because expiresAt was reused as both the session lifetime and the signing lifetime. Multipart part URLs in the same file kept 1h, and the pre-migration presigned route signed every PUT for 1h, so the widening was unintended rather than a policy change. No provider clamps below 24h. The PUT presign is now clamped at the provider boundary by a shared UPLOAD_URL_TTL_MS, which the part-URL path also uses so the two cannot drift. An expired PUT URL is deliberately not recoverable: unlike multipart, which re-signs per part call because its progress is durable, a PUT is not resumable, so an expired URL and an interrupted PUT have identical recovery. Nothing leaks, since every provider signs a create-only precondition. Clamping alone would have made the contract lie: the URL would die an hour before the session's advertised expiresAt, with nothing telling an integrator why the 403 happened. The PUT transfer now carries its own expiresAt, mirroring the multipart part-URL field. It is provider-dependent on purpose - cloud transfers report the clamped signature expiry, while the local data plane has no signature and admits against the session, so reporting an hour there would have been a new inaccuracy in the other direction. * chore: delete the dead presigned-upload and skill-adapter paths The presigned upload routes and the internal skills adapters were both replaced during the v2 migration, leaving their implementations behind with no callers. Removed generatePresignedUploadUrl and verifyPresignedUploadReceipt with their three provider helpers, QUOTA_EXEMPT_STORAGE_CONTEXTS and the types it orphaned, and the performCreateSkill/performUpdateSkill/performDeleteSkill adapters with recordSkillEvent and statusForSkillOrchestrationError. Each was verified unreachable across apps, packages, scripts and ee, including barrel re-exports and string access, not just direct imports. recordSkillEvent needed the closest look, since deleting an audit writer can silently drop coverage. The use cases declare the same action, resource, and description, and the framework adds the operation and actor the old helper lacked; recordAudit back-fills actorName and actorEmail from the user table when both are omitted, so the one field the helper passed is not lost. The self-hosting architecture doc described a directUploadSupported flag on an endpoint that no longer exists, and now describes the upload-session flow that replaced it. * refactor(folders): keep the cheap resource facts out of the schema graph Reading a folder resource type's label or its lock support meant importing folderResourceConfig, which imports the db schema for every table it serves and from there reaches lib/table/service, the executor, and the tool registry. That mattered as soon as lib/folders/queries needed a label: queries is reached from workspace-file-manager, which is reached from the files and chat pages, so one import edge put roughly 4,700 modules into those page graphs and broke the tool-registry boundary audit. Labels and lock support now live in a leaf module that imports only a type, and config composes them so there is still one source of truth. The three folder routes that pulled the whole config in for a single boolean read the leaf instead. * fix(skills): apply an upsert batch in one transaction The internal skills route looped the batch, calling an independently committing use case per item. A rejection on a later item left the earlier ones written and audited while the request reported failure - the compound-mutation rule in CLAUDE.md exists for exactly this. No new transaction plumbing was needed: upsertSkills already wraps its whole item loop in one db.transaction, so the partial commit came from calling it N times rather than once. upsertSkillBatch now validates and per-skill authorizes every item before issuing a single write, and createSkill and updateSkill became thin wrappers over it so v2 and Copilot keep one authority for the rules. The compound operation declares the read floor that skills.update already used, and the use case additionally authorizes skills.create when any item lacks an id, still ahead of every write. A read-only member who is a skill editor keeps their edit, and creates are not authorized more loosely than before. Audit projects one entry per committed skill, and analytics moved after the commit so nothing is reported for a rolled-back item. Note metadata.operation for these writes is now skills.upsert rather than skills.create/update; the action field still carries the distinction. * fix(security): close two disclosure gaps and finish the slot-leak fix The payer-pool gate only covered the workspace branch. A personal API key that omits workspaceId takes the account branch, where getHighestPrioritySubscription resolves an organization subscription from any member row regardless of role - so a plain member read the organization-wide credit and storage pool by dropping one query parameter. The account branch is now gated by the same authority, and the storage pool is not queried when it may not be disclosed. Forcing that branch self-scoped instead would have downgraded plan, period, and status, which is what a member needs to see whether the org is blocked. GET /api/v1/logs/executions/[executionId] emitted the workflow snapshot raw, carrying password sub-block values and oauth-input credential ids. It now shares the sanitizer the v2 read already used, extracted so there is one implementation rather than two. Env-var references are still preserved. cancelWorkflowGroupExecution itself was unguarded, so an unexpected throw from its transaction escaped ahead of every release site - the same reservation leak this branch set out to close, still open on the adjacent path. It now releases through the shared predicate and rethrows, because a failed transition means the cell state is unknown and a success-shaped answer would be a lie. The comment claiming the abort record cannot be taken back was false and now states the real reason: a refusal is always a terminal-or-absent state. * fix(v2-api): cover every persisted run status and every capped body The workflow-runs endpoints carried the same omission the logs contract had: the execution logger persists redacting, the run schemas did not list it, and because validation is whole-response one such row returned 500 for an entire page. Both schemas now derive from PersistedWorkflowExecutionStatus behind the same AssertNever gate, so a future status is a type error rather than a production 500. The single-run read keeps its extra queued value, which only it can observe. That schema was also serving as the run-list status filter. Widening it would have accepted a filter value the application input cannot express, so the reported set and the accepted filter are now separate schemas. Routes declaring maxBodyBytes without payloadTooLargeResponse fell back to a bare string with no error code and no private cache header. Rather than patch the four, the default moved into the builders - all three had the hole - which covers 58 body-bearing handlers, and a route override still wins. The five per-route overrides that merely restated the default are gone. Also documents the 413 on the one knowledge route that has a real body cap, adds the rollout gate's 404 to the last v2 operation missing it, and rewords the nextCursor description, which read as though every list were a full-set list. * fix(api): make the shared traits and lifetimes single-sourced The folder resource-traits leaf composed labels into config but restated lock support independently, so the routes reading the trait and the orchestration reading the config could disagree about which resources lock. Config now composes both, and supportsLocking is required rather than optional so a new resource type cannot silently omit it. The upload commit claimed the PUT clamp and the multipart part URLs shared a constant and could not drift. That was true only of the advertised expiry - the three provider signers each hardcoded an hour, so changing the constant would have moved what we advertise while leaving what we sign, recreating exactly the mismatch the clamp removed. Each provider now receives the lifetime in its own unit from the one constant. Table restore hand-rolled its status map and returned the driver message verbatim at 500, leaking the failed statement and its bound parameters - the same defect this branch closed at nine other sites. It and import-csv now use the shared projection; import-csv's result type also had to carry the lock the classifier already set, so a 423 can name it. Deletes v2RowWriteError, which had no callers and would have rendered a locked table as 400 by discarding the 423 it was handed.	2026-08-11 19:37:40 -07:00
Siddharth Ganesan	7f39678264	improvement(desktop, executions): fix desktop update script and throw error when thinking enabled on stream false (#6578) ... * Fix update check * fix include tool calls error * fix comments	2026-08-11 18:48:16 -07:00
Theodore Li	092311ea68	fix(api): restore migrated endpoint and SDK compatibility (#6564) ... * fix(api): restore migrated endpoint compatibility * fix(api): close remaining migration regressions * fix(files): validate ensured folder paths * fix(files): project folder path validation * fix(files): align archive regression fixture * fix(tables): avoid partial bulk update failures * fix(auth): project legacy knowledge audits	2026-08-11 20:36:37 -04:00
Waleed	9b9f4ee596	fix(v2-api): close three secret disclosures, make the surface consistent, and align docs with signatures (#6560) ... * fix(v2-api): close two secret disclosures and align docs with signatures Two P0 disclosures, five correctness bugs, and the standardization and guard work that came out of auditing them. **Secret disclosure — workflow version state.** `GET /api/v2/workflows/{id}/ versions/{version}` served the deployed graph unsanitized, so a read-role workspace API key received plaintext block-password values and OAuth credential ids. The sibling export route has always sanitized. Every other v2 response is protected structurally because the builder re-parses it, but this field is `z.custom<WorkflowState>()` — a predicate that validates nothing — which is why it survived earlier audits. Sanitization now lives in the use case, secure by default, with a named `includeCredentialValues` opt-in that only the session-authed deploy-preview route sets. **Secret disclosure — MCP headers.** The internal list and update routes returned custom `Authorization` headers verbatim to any read-role member; headers are stored unencrypted. Values are now gated on write permission and projected through one shared helper. The settings UI genuinely prefills from them, so blanking outright would wipe headers on unrelated edits — write-only headers plus encryption at rest are the follow-up. Correctness: - v2 execute ignored `X-Sim-Via`, resetting the call chain on every hop and defeating the recursion guard. Wired on both the keyed and anonymous paths. - v2 knowledge search accepted `searchMode` and dropped it, silently serving vector-only results for a hybrid request, and allowed 50MB bodies where internal caps at 2MiB. - v2 run cancel never released the plan concurrency slot and half-cancelled group runs; a group conflict now returns 409 instead of reporting success. - v2 table row writes stamped no secret provenance, so the next internal read reported the whole page incomplete. `secretProvenance` is now required on the primitives, making the next omission a compile error. - Folder conflicts and malformed paths returned 500; they are 409/404/400 now. `FolderPathError` splits from `FolderHierarchyError` so a corrupt stored tree stays a 500 and stays in 5xx alerting. Standardization and documentation: - `PUT /files/{id}/share` -> PATCH. The resource is not round-trippable (`hasPassword`, never the password), so merge-on-omission is the only implementable semantics. - ~40 spec truthfulness fixes: a 410 the API cannot emit, eight 423s with no lock guard, ~30 reachable-but-undocumented 404/400/413s, and six inverted field claims. Eleven operations that always reject a workspace key now say so — four of them answer 404, so a workspace key was told the resource did not exist. - `NAME_PATTERN` lost its `/i` through `z.toJSONSchema`, publishing 15 patterns that reject names the runtime accepts. Every generated client rejected any capitalized table or column name, and two of the spec's own examples failed the spec's own schema. Guards, so these classes cannot recur: - `check:route-verbs` (new) cross-checks all 212 builder routes' exported verb and path against their contract. The builders only compare at runtime, so a half-done rename previously passed CI and 500'd in production. - Example validation now runs against the published JSON Schema with formats on, covering 225 nodes instead of 100 — this is what caught the regex bug. - The list-pagination sweep is union-aware and fails loudly on a schema it cannot introspect, rather than counting it compliant. * refactor(v2-api)!: flatten the single-resource response envelope BREAKING: 31 endpoints that returned `{ data: { <resource>: T } }` now return `{ data: T }`. This corrects drift, not a design decision. PR #5273 added skills, custom tools, MCP servers, secrets, and knowledge nested while adding workflows, files, and logs flat — and in the same commit wrote the `v2/shared.ts` docblock declaring `single resource: { data: T }` is the standard. The nested half appears to have been modelled on the v2 tables surface (#6067), which landed twelve days earlier. Lists were already `{ data: T[], nextCursor }`, so flat single-resource is what actually matches them; nesting made every client destructure a layer that carries nothing. Doing it now because the cost only grows: `v2-api` is still dark-launched, so today this breaks no one. After GA it needs a deprecation window. Payloads that carry real information were deliberately left alone — this was a classification exercise, not a mechanical sweep. Unchanged: delete acknowledgements (`{ id, deleted }`, `{ path, deleted, deletedItems }`), the knowledge search envelope (which echoes query, knowledgeBaseIds, topK and totalResults alongside hits), upload payloads carrying signed tokens and transfer instructions, bulk-operation counts, `{ row, operation }` upserts, named acknowledgement scalars (`{ dispatchId }`, `{ cancelled }`), and `{ columns: [...] }` — a collection, where a bare `{ data: T[] }` would be indistinguishable from the list envelope but without `nextCursor`. Also flattened the two file-share responses, which were not in the original survey: leaving them would have put one resource in two shapes on one path. `GET /files/{id}/share` now returns `{ "data": null }` when a file has never been shared. No consumer is affected. Both SDKs touch exactly two v2 endpoints — execute and run status — and both were already flat. No docs MDX, client hook, or internal caller reads a changed response; Copilot table tools call the application use cases directly rather than the HTTP surface. The shared `v2FolderSchema` is untouched: every folder flatten was achievable at the response site, which is itself evidence flat was the intended shape. * fix(v2-api): close a third secret disclosure and make concealment coherent **Secret disclosure — run snapshot.** `GET /api/v2/logs/{runId}` returned `workflowState` straight from `workflowExecutionSnapshots.stateData`, which is the workflow graph: `blocks[].subBlocks[].value` holds `password: true` field values and `oauth-input` credential ids. Nothing on that path sanitized it, and the field was typed `z.unknown()`, so the builder's response parse stripped nothing. A read-role workspace API key could read plaintext credentials. This is the third instance of one pattern, and the pattern is the finding: the builder protects every response by re-parsing it, so the only fields that can leak are the ones typed `z.unknown()` or `z.custom()`. Both prior disclosures sat behind exactly such a field. The snapshot is now sanitized in the use case and the field is typed object-or-null. An inventory of every remaining `z.unknown()` in the v2 contracts is in the PR description; two carry data with no projection behind them and are named there as follow-ups. **Concealment was bypassable.** `createV2ResourceConcealmentPolicy` rewrites resource-authorization failures to 404 so a caller cannot probe for existence. Workflows and files applied it on every verb; tables and knowledge applied it only on reads. A caller could therefore probe with PATCH, read the 403, and learn the resource exists — the read-side concealment bought nothing. Nine mutation sites now conceal, plus the three table-column verbs, which were inconsistent with their own sibling sub-resources. `lib/logs/api/route-policies.ts` was a second, divergent implementation that sniffed `response.status === 403` and so also swallowed workspace-policy denials the canonical helper deliberately preserves. It now uses the helper. A third such sniff survives in the upload-control helper and is noted as a follow-up. Also: - `DELETE /tables/{tableId}/rows/{rowId}` returned the bulk `{deletedCount, deletedRowIds}` shape while nine sibling single-resource deletes return `{id, deleted}`. It now matches them. - Nine operations can 404 on an unknown folder path and did not document it; `createWorkflow` could 413 on an oversized folder tree and did not; getting a run can 409 when trace data was truncated and did not. - `queryTableRows` documented a 413 it cannot emit and `resumeWorkflowRun` a 423 with no lock guard anywhere in its path — the same un-producible-status class already cleared for 410 elsewhere. - Execute's 409 description covered only the run-id case after the recursion-guard fix added a second cause, and named a code the route does not emit: the wire carries `error.code: CONFLICT` with the specific cause in `error.details.code`. `x-sim-via` is now a declared request header. - Deploy and rollback published examples that were impossible: `isDeployed: true` beside `activeDeployment: null`, where the route computes the former from the latter. - `afterRowId`/`beforeRowId` were published on row insert and silently dropped by the route, so a positional insert became a tail append. - A generated document whose script fails permanently answered "still being generated, try again" forever; the underlying cause is now preserved. * docs(v2-api): correct eleven false or misleading spec claims Structural parity between contracts and specs is CI-enforced; semantic truth is not. These are claims the spec made that the code does not honour. Outright false: - `DELETE /files/{fileId}` said it deletes "the stored bytes". It archives: the row is retained with a deletion timestamp and the bytes are never removed. Restore exists, but only on the internal API, so the description now says so rather than implying v2 offers it. - Execute documented `409 EXECUTION_ID_CONFLICT` in three places. The wire carries `error.code: CONFLICT` with `error.details.code: RUN_ID_CONFLICT`; only v1 ever emitted the documented string. - The files spec claimed every endpoint uses the canonical envelopes while `GET /files/{fileId}` returns octet-stream. - The shared timestamp rule justified itself with a rendering claim that is false — 29 bare-form sites publish `format: date-time` identically. The real difference is runtime validation, so the rule now says that. It was softened rather than enforced: responses are re-parsed, so adding `.datetime()` to a field whose producer can emit a non-ISO string turns a working read into a 500, and that could not be proven for all 29 without a much larger audit. Misleading: - The billing ledger silently defaults to a 30-day window, so a client paginating to `nextCursor: null` believes it has the whole ledger. - Deleting a connector-backed knowledge document does not delete its chunks — the row survives as excluded and the embeddings remain. - `listTables` said "all tables"; it is keyset-paged with a default limit. - `GET /files/{id}/share` omitted the `data: null` never-shared case its own schema and example already declare. - The share PATCH matrix omitted two hard 400s, so following it literally against a never-shared file fails. - Five knowledge operations render a canonical folder path back and can 413 on an oversized tree without carrying the sentence that says so. Also: the upload-control helper was a third implementation of concealment by sniffing `response.status === 403`, which masks workspace-policy denials the canonical helper deliberately preserves. It now uses the shared policy, so those denials keep their 403. And the shared docblock's search-field enumeration was presented as exhaustive while omitting two lists, and its error-envelope claim omitted the two upload data-plane routes that emit a bare `{error: string}` — both now carry the carve-out the CI allowlist already had. * test(v2-api): align upload concealment test with cross-tenant-only semantics #6557 narrowed `createV2ResourceConcealmentPolicy` to conceal only the three cross-tenant authorization classes, deliberately letting a same-workspace policy denial keep its 403 so the caller learns why. My test predated that and asserted a workspace-key denial was concealed as 404. Split into two cases that pin the distinction rather than paper over it: a cross-tenant reach conceals, a workspace-key policy denial does not. * fix(v2-api): accept the redacting log status and envelope the knowledge-search 413 The v2 log presenters parsed status against a five-value enum, but the execution logger persists a sixth, redacting, while a finished run's output is scrubbed. Any such row failed the response parse; on the list route one row 500'd the whole page. The enum is now derived from PersistedWorkflowExecutionStatus with a compile-time exhaustiveness assertion, so a future status is a type error rather than a production 500. POST /api/v2/knowledge/search declared maxBodyBytes without payloadTooLargeResponse, so its 413 returned a bare string instead of the v2 error envelope. It now matches the sibling deploy/rollback routes. * fix(uploads): restore archive extraction folder parity Archive extraction into workspace files/ was rewritten onto the authorized application-operation boundary, and three behavioral regressions came with that move. Together they broke every archive containing a subdirectory, and 100% of copilot extract() calls (materialize-file always passes rootFolderSegments: [baseName], and its catch only handles ArchiveError). 1. Non-canonical folder path. The extractor joined the folder segments with "/" and passed the result as `path` to createWorkspaceFileFolderOperation. That path reaches requireNonRootFolderPath -> parseFolderPath, which requires a leading "/" and byte-for-byte canonical per-segment encoding, so "bundle/data" threw FolderPathError before anything was written — and a folder name containing a space or a reserved character would still have thrown after merely prefixing a slash. 2. exactName: true. createWorkspaceFileFromBuffer was told to demand the exact leaf name, which sets maxAttempts = 1 and raises FileConflictError when the name already exists. The extractor's rollback then deleted every file written so far, so one colliding name destroyed the whole extraction. Reachable today for flat archives through the unzip action of POST /api/tools/file/manage. Restored to auto-suffixing via allocateUniqueWorkspaceFileName. 3. Wrong folder primitive. createWorkspaceFileFolderAtPath creates exactly one leaf, conflicts on an existing path, and requires the parent to exist already. The extractor never creates intermediates and caches by full path, so the first nested entry asked for a folder whose parent was never created. The correct semantics are ensureWorkspaceFileFolderPath: walk every segment, reuse what exists, create only what is missing. Rather than bypass the operation boundary by calling the manager primitive directly, this adds ensureWorkspaceFileFolderPathOperation — an authorized application use case under files.folders.create that expresses "ensure this whole chain exists" — and routes the extractor through it with raw decoded segments, so no path string is built and no encoding can be malformed. archive.test.ts previously mocked the folder operation and asserted the broken shape (path: 'bundle'), which is why this shipped. The suite now fakes the workspace-file store in memory while enforcing the real rules: folder paths run through the production parseFolderPath family, the create-one-leaf operation conflicts and requires a parent, and exactName governs conflict vs auto-suffix. Nested, reuse, encoded-name, and collision cases are covered and each fails against the pre-fix code. * chore(files): tidy archive extraction cleanup * fix(uploads): roll back folders archive extraction created Extraction now materializes folders before uploading files, but the failure path only deleted the extracted files — every folder the call created was left behind. That is not cosmetic: `materialize_file` guards re-extraction by looking up the root folder path and refusing when it has any child, so a half-extracted nested archive turned every retry into "already extracted — delete that folder first" until a human cleaned up the tree by hand. The rollback must delete only folders this call actually inserted, never one it reused: extracting into an existing path is normal (a sibling entry, an earlier successful extraction), and deleting a pre-existing folder would destroy unrelated user data. `ensureWorkspaceFileFolderPath` already distinguishes the two while walking the segment chain, so it (and its application operation) now reports `createdFolderIds` alongside the leaf id. The extractor accumulates those ids in creation order and, on failure, deletes them in reverse — parents are recorded before their children, so reverse order is deepest-first and a parent is never removed out from under a child. Folder cleanup is best-effort like the existing file cleanup, so a cleanup failure never masks the original error. * fix(billing): withhold the payer credit pool from v2 status readers `GET /api/v2/billing/status` resolved the workspace's payer and projected that payer's pooled allowances — credits used, credit limit, credits remaining, and the payer entity's storage usage and quota — to any caller holding only `read` on the workspace, including a personal API key. The payer pool is shared across every workspace that payer funds, and the platform already treats it as privileged: the workspace credit-availability surface computes `canViewPayerPool` from `canManageWorkspaceBilling` and substitutes member-scoped or null figures for everyone else. The new versioned endpoint had no equivalent gate. `credits` and `storage` are now projected only to a caller who may manage the resolved payer's billing: the billed account holder of a personally hosted workspace, an admin of the hosting organization, or a workspace API key, which only a workspace admin can provision. The endpoint stays at `read` so a plain member keeps the plan, period, and standing the workspace UI already shows them, and an exceeded pooled limit still reports as `limit_exceeded` without disclosing the numbers behind it. Both fields are nullable on the wire and in the regenerated OpenAPI spec. The decision lives in the application use case, resolved from canonical workspace state, not in the route: billing authority is payer identity and organization role, which the workspace permission ladder cannot express — a plain workspace `admin` is deliberately not enough. * chore(api): remove the unused public API route builder and dead endpoint labels `withPublicApiRouteHandler` and 27 `ApiEndpoint` union members landed together in #5273, but the v2 surface shipped on `defineV2JsonRoute` + `v2RateLimits` instead. The builder had no production caller — only its own test — and the v2 rate limiter never reads an `ApiEndpoint` label, so those members were never emitted to telemetry by symbol or by string literal. Remaining members are exactly the labels a v1 route passes to `checkRateLimit` or `authenticateRequest`. Drops the now-unreachable `hasZodUsage` branch from the API validation audit; no ratchet metric moves (route total stays 1093). * fix(billing): deny the payer pool to actor-less workspace API keys The first pass gated `credits` and `storage` on billing authority for personal API keys but let a `workspace_api_key` principal through unconditionally, which left the excluded role a way back in. Any workspace `admin` may mint a workspace API key, and a workspace `admin` is deliberately not a billing manager, so an admin who reads `null` as themselves could mint a key and read the full pool with it. On an organization-hosted workspace that pool is the organization's, spanning workspaces the admin has no standing in. Billing authority is payer identity or an organization admin role — a property of a person. A workspace API key is deliberately actor-less, so it can never satisfy it and now reads both fields as `null`. Attributing the key to its creator was rejected: it would launder the same workspace-admin role, it breaks when the creator's authority is revoked while the key lives on, and substituting a key's owner for the acting principal is what the application operation boundary forbids. The reasoning sits in TSDoc at the decision point. The key keeps the plan, period, and standing it needs to monitor a workspace, including `limit_exceeded` and `billing_blocked`. No in-repo caller reads `credits` or `storage` from this endpoint. The payer storage pool is now read only once disclosure is authorized, so a caller who may not see it no longer triggers the query at all. * fix(folders): bound the workflow folderId-branch path index reads `createWorkflow` and `updateWorkflow` each resolve a folder two ways inside one function. The folderPath branch goes through `resolveWorkflowFolderPath`, which loads the path index with `maxRows: MAX_FOLDERS_PER_WORKSPACE`; the folderId branch loaded it with no bound at all, issuing a `SELECT` over every active folder row in the workspace. In `updateWorkflow` the unbounded read and the bounded fallback sit thirty lines apart in the same function. Passes the cap at both sites, matching the read sites that already opt in. Exceeding it throws `FolderCollectionLimitExceededError` rather than truncating, because a partial path index resolves real folder paths to `undefined` and re-roots resources at the workspace root. `maxRows` deliberately stays opt-in rather than becoming the default. Folder creation does not refuse at the same ceiling on every path — `POST /api/folders` goes through the `createFolder` name/parentId variant, which passes no `maxFolderRows`, so the count guard in `executeCreateFolderAtPath` never runs and a workspace can already hold more than `MAX_FOLDERS_PER_WORKSPACE` folders. Defaulting the bound would make every path-index consumer throw for a state the product allows to exist. Reconciling reader and writer is a separate change with a user-facing limit, not a chore. * chore(billing): tidy payer-pool concealment cleanup * fix(api): reject an undecodable offset cursor on v2 table rows GET /api/v2/tables/{tableId}/rows coerced an undecodable pagination cursor to offset 0 and re-served page one. A client paging forward reads that as a fresh first page and can loop over it forever. Every sibling v2 cursor list — logs, files, workflows, workflow runs, workflow versions, workspace members, tables, knowledge documents — already rejects with a validation error instead. Extracts the offset-cursor decode both offset-paginated v2 routes had inlined into `decodeOffsetCursor`, next to the existing `decodeSortedCursor`, so the reject-don't-restart rule has one home. * fix(api): restore v1 table error-response parity and stop internal message leak The v1 table routes were rewritten to consume `lib/table/orchestration` results, and two response behaviors drifted from what the live API returned. Information disclosure: an unclassified failure's `outcome.error` carries whatever text the fault happened to have. Drizzle wraps a throw raised inside a transaction in an error whose own message is the failed statement and its bound parameters, so `DELETE /api/v1/tables/{tableId}` and `DELETE /api/v1/tables/{tableId}/rows/{rowId}` returned that verbatim in the 500 body to any API-key holder. Previously these returned a fixed generic string. Lost `lock` field: the 423 body used to be `{ error, lock }`. The delete, row-delete, and column-update routes (v1 and internal) dropped the lock kind the orchestration result already computes, leaving clients unable to tell which lock to clear. Both are fixed at one altitude: `orchestrationOutcomeErrorResponse` in `app/api/table/utils.ts` is now the only way a table route projects an orchestration failure onto the wire. It renders the route's fallback for an unclassified failure and the real message for a classified one (validation, not-found, conflict, locked keep their specific text), and carries `lock` on a 423. A future route cannot reintroduce either bug by hand-spelling the body. Duplicate table names on `POST /api/v1/tables` keep answering 409 rather than reverting to the previous 400. 409 is the correct semantic, and every other v1 duplicate-name surface (knowledge, files, workflow import) already answers 409; the tables 400 was the outlier. v1 tables appears in no published OpenAPI document and no in-repo client branches on the status, so the compatibility cost is limited to a caller matching 400 specifically for a name collision. * fix(skills): only reject a built-in name collision on an actual rename The built-in-name guard ran on every update that carried a `name`, without comparing it to the skill's current persisted name. Skills created before the guard existed can legitimately carry a built-in's name (they simply shadowed the built-in at read time), and the skill modal always submits the full object including the unchanged name — so every save of such a skill returned 400 with "The skill name ... is reserved by a built-in skill", with no way to fix it short of renaming. Move the guard in `updateSkill` to after the canonical row is loaded and run it only when the submitted name differs from the current one. Creating a skill with a built-in name, and renaming an existing skill into one, are still rejected. The check stays in the shared orchestration primitive because that is the only layer both the internal `/api/skills` adapter (via `performUpdateSkill`) and `updateSkillUseCase` (v2 + Copilot) pass through, and it is where the current name is in hand. * chore(tables): tidy v1 error projection cleanup * chore(skills): tidy collision guard cleanup	2026-08-11 16:41:27 -07:00
6 people	263e3ca67e	improvement(external-endpoints): v2 versions with clean signatures + updated docs based on openapi spec (#5273) ... * v0.6.29: login improvements, posthog telemetry (#4026) * feat(posthog): Add tracking on mothership abort (#4023) Co-authored-by: Theodore Li <theo@sim.ai> * fix(login): fix captcha headers for manual login (#4025) * fix(signup): fix turnstile key loading * fix(login): fix captcha header passing * Catch user already exists, remove login form captcha * improvement(external-endpoints): v2 versions with clean signatures + updated docs * feat(usage): accept X-API-Key on usage-logs list + export /api/users/me/usage-logs and /export now use checkHybridAuth — the same auth /api/users/me/usage-limits already accepts — so external monitors can read summary.bySourceCredits (the source breakdown of usage-limits' aggregate currentPeriodCost) instead of estimating Copilot spend by subtraction. Workspace-scoped keys are pinned to their own workspace's slice of the ledger: the filter defaults to the key's workspace and an explicit mismatch 403s. Both endpoints documented in openapi-core.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(billing): dedicated v2 usage endpoints; keep internal usage routes session-only Replaces the earlier X-API-Key enablement on /api/users/me/usage-logs with a dedicated public surface, so the internal Billing-settings endpoints can evolve with the UI while external monitors get a stable versioned contract: - GET /api/v2/billing/usage — current-billing-period summary with bySourceCredits (the source breakdown external monitors need to watch e.g. Copilot consumption without estimating by subtraction), plus limitCredits and plan - GET /api/v2/billing/usage/logs — cursor-paged credit ledger in the v2 envelope - workspace-scoped keys are pinned to their own workspace's slice; personal keys read the account ledger The public wire is credits-only: usage-logs rows now carry a hasCost boolean instead of dollarCost (the Billing UI only needed the >0 signal), and the rateLimit block is removed from the usage-limits response and docs (deploy-modal tab relabeled accordingly). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(docs): validate OpenAPI specs against the Zod contracts in CI The specs in apps/docs are hand-authored because they carry what Zod never defines — error envelopes, status codes, prose, examples — so they can't be generated; check:openapi validates them instead: - spec integrity: $refs resolve, operationIds unique, 2xx documented, no orphaned component schemas - v2 conventions: every /api/v2 operation documents 401 + 429 and every 4xx/5xx resolves to the canonical { error: { code, message } } envelope - contract cross-check: contracts are auto-discovered from lib/api/contracts/v2 (each carries its method + path); doc<->contract coverage both ways, query/body/response field diffs via z.toJSONSchema - examples: documented request/response examples must parse with the matching contract's actual Zod schemas First run caught real drift, fixed here: 16 stale orphaned schemas in the core spec, the v2 billing ops referencing v1-shaped error components, deploy/rollback examples missing the required nullable lifecycle keys, CreateTableBody missing folderId, a legacy-grammar delete-rows example, and four knowledge document ops missing their required workspaceId query param. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(docs): recursive field diff in check:openapi + the deep drift it found A mutation test showed the doc<->contract field diff only compared top-level properties, so a typo inside the { data } envelope passed. The diff now descends through matching object properties and array items (both sides must expose a property set — passthrough contracts and prose-only docs end the descent instead of false-positive), with the Zod JSON-schema root doubling as the $defs context. Deep drift it immediately caught, fixed here: select-column config (options/multiple) missing from every tables column schema, AddColumnBody hand-rolling a third column shape (now composed from ColumnInput, with position/workflowGroupId as the per-op extensions the contracts actually admit), chunking strategyOptions undocumented, and the deployment lifecycle fields (activeDeployment/latestDeploymentAttempt) missing from DeploymentState. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(security): close the triggerType rate-limit bypass on workflow execute Caller-supplied triggerType flowed unchecked into preprocessExecution, whose checkRateLimit default turns OFF for 'manual'/'chat' — so any API-key caller, and any anonymous public-API caller billed to the workspace owner, could execute unthrottled by sending {"triggerType":"manual"} (async runs also skipped the worker-side check via admissionCompleted). External callers may now only send the redundant 'api' value; internal JWT callers ('workflow'/'mcp') are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * refactor(execution): extract enqueue/status/cancel into shared libs Prepares the v2 execution surface: handleAsyncExecution's queue logic moves to lib/workflows/executor/enqueue-execution.ts (slot/claim semantics encoded in a discriminated outcome, not HTTP statuses), the execution-status read to execution-status.ts, and the order-sensitive cancel machinery to lib/execution/cancel-workflow-execution.ts. The v1 routes re-render identically — their suites pass unmodified. Also: preprocessExecution gains rateLimitCounter ('sync'|'async') and its 429 now carries code RATE_LIMIT_EXCEEDED + retryAfterMs (previously indistinguishable from the concurrency 429 and Retry-After was discarded); and the duplicate cancel contract in contracts/logs.ts is unified on the full 5-value reason enum — its narrower copy made requestJson throw a client ZodError when cancelling a paused HITL run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(execution): callable execution service + structured error classifier executeWorkflowService composes the same libs the v1 route holds inline (call-chain guard, execution-id claim, LoggingSession, preprocessing, deployed-state load + file-field processing, timeout-bound executeWorkflowCore, output hydration/compaction) for the deployed-state caller class — the seam the v2 execute route and in-process internal callers share, making the HTTP endpoint syntactic sugar. classifyExecutionError stops discarding the block context that buildBlockExecutionError already attaches at throw sites: failed runs now yield {message, code, blockId, blockName, blockType} with a stable append-only code enum (TIMEOUT/CANCELLED/USAGE_LIMIT_EXCEEDED/ INVALID_INPUT/BLOCK_EXECUTION_FAILED/CHILD_WORKFLOW_FAILED/ OUTPUT_TOO_LARGE/EXECUTION_FAILED), so callers route on error class instead of substring-matching messages — the single place raw errors are interpreted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): POST /api/v2/workflows/[id]/execute Thin route over executeWorkflowService: X-API-Key or anonymous public-API auth (sync/stream only for anonymous), strict body with body-flag async (no mode headers on v2), SSE passthrough for stream, and the execution resource response — executionId always present, in-band run failures are status:'failed' with the structured {message, code, blockId, blockName, blockType} error, sync timeout is status:'failed' + TIMEOUT instead of v1's 408, and a Response block's payload stays inside output (authors never control response status/headers on this origin). Async debits the async bucket and the 202 statusUrl points at the v2 executions resource. Adds CLIENT_CLOSED_REQUEST/SERVICE_UNAVAILABLE to the v2 error codes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): v2 executions status + cancel with queued backfill GET /api/v2/workflows/[id]/executions/[executionId] is the single status URL for sync and async runs: before the async worker writes the durable log row, status is backfilled from the job queue (deterministic job id) as 'queued'/'running' — closing v1's 202-to-pickup 404 window — and failed runs carry the structured error object. POST .../cancel renders the shared cancellation lib in the v2 envelope with the tightened 5-value reason enum. Both authenticate via the shared resolveV2WorkflowAccess (X-API-Key, authz masked as 404, allowPersonalApiKeys honored). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(execution): workflow tool + MCP bridge run in-process workflow_executor (workflow-as-agent-tool) short-circuits in executeTool through WorkflowBlockHandler — the same invocation boundary canvas child workflows use — mirroring the deployed_block_executor precedent. The MCP serve bridge calls executeWorkflowService directly instead of fetching its own execute endpoint; deployment-version pinning, MCP response-size rejection, and the actor override become typed options instead of header sniffing. Both callers drop the double admission slot and duplicate top-level log row the HTTP hop cost, and failed child runs now surface the structured error + child executionId so parents and MCP clients can route on error class and hand providers a reproducible handle. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(infra): CORS + CSP coverage for the v2 execute path /api/v2/workflows/:id/execute gets the same wildcard-origin, credential-free CORS policy as v1 (the default credentialed policy would block browser API-key calls and open a cookie CSRF surface) with X-Sim-Stream-Protocol allowed and no X-Execution-Mode (async is body-selected on v2), plus the COEP/COOP/CSP header block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(ui): deploy modal + copilot advertise the v2 execute surface All 20 API-tab snippets move to POST /api/v2/workflows/{id}/execute with the nested {"input": ...} body, async as the "async": true body flag (X-Execution-Mode gone), status polling against the v2 executions resource, the third tab renamed Usage and pointed at /api/v2/billing/usage, and {data} envelope unwraps in the printed responses. Fixes the latent baseUrl derivation (endpoint.split('/api/workflows/')) that would have silently built garbage URLs under a v2 endpoint, and deletes dead code (exampleCommand across 3 sites, getAsyncExampleTitle). Copilot deploy/manage/serializer endpoint builders and the api_trigger bestPractices example follow (the latter also drops its hardcoded staging host). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * docs(api): document the v2 execution surface Adds execute, execution status, and cancel to openapi-v2-workflows.json with the structured ExecutionError schema (append-only code enum + block attribution) and the ExecutionResource contract, documenting the rules that differ from v1: modes are body-selected, a failed run is HTTP 200 with status 'failed', an executionId always means data (never the error envelope), queued status is visible immediately, and Response-block payloads stay inside output. Registers the three pages in the generated workflows meta.json and bumps the route-count baseline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): gate the whole /api/v2 surface behind one flag; UI stays on v1 Every v2 route now runs exactly one check immediately after auth — v2ApiGateError — and answers 404 when the `v2-api` flag is off, so the surface is invisible until it is deliberately rolled out. The gate is keyed on userId only: a workspace/org-keyed check would have to read membership for a caller-supplied id before authorization runs, and its 404-vs-403 split would leak cohort membership (the trap the per-domain table gate worked around by running late). The two executions routes inherit it from the shared access resolver; the tables-specific gate is removed so no route checks twice. `tables-v2-api` stays, now gating only the internal predicate-grammar route /api/table/[tableId]/query — note v2 tables routes move to the unified flag, so enabling them is a `v2-api` decision now. Reverts the deploy modal, copilot handlers, and api_trigger example to the v1 execute endpoint: v1 works unchanged, and the UI must not advertise a surface most users would get a 404 from. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(executor): restore child-cost aggregation dropped by the staging merge Staging's custom-block rewrite deleted `aggregateChildCost` from workflow-handler.ts, and git merged that file cleanly — but this branch's workflow-tool-runner.ts, added for the v2 execute migration, still imports it. A silent semantic conflict: no marker, broken build. Taking staging's rewrite is correct, so the helper is defined locally in its one remaining consumer rather than resurrected in the file staging just rewrote. Same four lines over the still-exported `calculateCostSummary`, so a failed child workflow keeps billing the hosted-key spend it consumed instead of reporting $0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): make lib/table/orchestration the single implementation (#6134) * refactor(orchestration): move the shared error contract out of lib/workflows OrchestrationErrorCode and statusForOrchestrationError are the contract every lib/[resource]/orchestration module returns against, but they lived inside the workflows module, so resource-neutral code (lib/folders) already had to import from a workflow path. Moved to lib/core/orchestration/types. Adds a 'locked' class mapping to 423. Both tables and workflows have a lock that forbids a mutation, and each caller was translating that to a status itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): make lib/table/orchestration the single implementation Column update was implemented four times — the UI route, v1, v2, and the copilot table tool — each calling the same column services but owning its own guards, error mapping, and audit. The copies had drifted, and the drift was the bug: v2 was missing both guards, only the copilot copy minted stable option ids, and only v1/v2 audited. performUpdateTableColumn, performDeleteTable, and performDeleteTableRow now own that logic; all ten call sites reduce to auth, parse, call, render. The guards are asserted once in lib/table/orchestration rather than four times against four routes. Behavior this consolidates, previously true on only some paths: - The typeChanging guard. updateColumnType early-returns on an unchanged type and drops any options sent with it, so restating the current type alongside new options silently discarded them. v2 had no guard at all and, since its contract shares v1's body schema, accepted options and ignored them. - The select-unique guard. Each write is its own locked transaction, so a rename or type change paired with a constraint write that is going to fail commits first and then throws, half-applying the schema change. - Stable select-option ids. Cells reference the option id, so an edit that re-sends an option by name has to reuse it or every cell holding it is orphaned. Only the copilot path did this; normalizeSelectOptionsInput moves to lib/table/select-options and now covers every caller. It preserves a supplied id, so it is a no-op for the fully-formed options the HTTP contracts accept. - required forwarded into the type and options writes, so a conversion validates against the constraint the same request is setting. - An audit on every successful update. The UI route and the copilot tool emitted none. - Single-row delete through the row service. v2 did a raw db.delete, skipping assertRowDelete and deleteOrderedRow, so a delete-locked table returned 200 and the row-count bookkeeping never ran. - The delete actor handed to deleteTable, which audits only when a row was actually archived. v1 and v2 omitted it and audited themselves outside that check, emitting TABLE_DELETED for a no-op delete of an archived table. Failure classes come back as OrchestrationErrorCode; v2 renders them through a new v2ErrorForOrchestration, mirroring statusForOrchestrationError on the v1 and UI surfaces, so a given failure maps to the same status everywhere. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(tables): bind the column-update tests to the orchestration function The base's route tests assert which column service each payload reaches — the behavior that now lives in performUpdateTableColumn. They mocked the `@/lib/table` barrel; the orchestration module imports the service directly, so they mock that too and keep asserting the same thing through the extracted implementation. The orchestration tests move onto the base's semantics: writes address the stable column id, a rename rides inside the write it accompanies rather than running first, and the currency guards replace the non-select options guard the service now owns. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(copilot): drop the column-type import the delegation made dead Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): move the audit log out of the table service `lib/table/service.ts` wrote its own audit rows, so whether an operation was audited depended on which function a caller reached for rather than on a user having performed it. That is what let v1 and v2 audit a no-op delete, and what made `deleteTable`'s optional `actingUserId` double as an audit opt-out flag. Worse, most sites fell back to `actingUserId ?? createdBy`, so an unattributed call was logged against the table's *creator*. The copilot `mv` path passed no actor at all: renaming someone else's table recorded them as the renamer. Audit now lives in the orchestration functions — performDeleteTable, performRenameTable, performMoveTableToFolder, performUpdateTableLocks — and the services just write. Internal callers (folder cascade, import rollback) keep calling the service and are silent by construction rather than by remembering to omit an argument. Two services now return what the audit needs: `deleteTable` reports whether it actually archived a row, so a repeat delete logs nothing; `updateTableLocks` returns the before/after locks, since only the locked write can observe the transition its description names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(tables): restore audit provenance and conflict status in orchestration Moving the audits into the orchestration functions dropped three things the routes had been carrying, and added one the orchestration now owns twice. - The v1 and v2 column-update routes passed `request` to `recordAudit`, so their audit rows recorded the caller's IP and user-agent. The orchestration function had no way to receive it. Every table orchestration function now takes an optional `OrchestrationRequestContext` and every HTTP route forwards it; the copilot and VFS callers, which have no request, omit it. - `classifyTableMutation` matched `TableConflictError` on "already exists" appearing in the message and reported it as `validation`, turning the UI route's 409 on a duplicate table rename into a 400. It now matches the type, the way `performRestoreTable` already did. - `captureServerEvent` ran on every delete while the audit was gated on a row actually being archived, so a repeat delete of an archived table still reported `table_deleted`. Both now hang off the same evidence. - The copilot delete path kept its own `captureServerEvent` from when the service did not emit one, double-counting every copilot table delete. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a * fix(tables): say which type a no-op column update restated A copilot `update_column` payload whose only content was the column's current type used to return success with the live schema, while the v1, v2, and UI routes rejected the same payload with "No updates specified". Delegating to `performUpdateTableColumn` unified them onto the routes' rejection — correct, but the message tells the caller its request was empty when it named a type. The orchestration function now reports the same thing `updateColumnType` reports when it loses this race concurrently: the column is already that type, re-issue without the type change. An empty payload still reads "No updates specified". Drops the copilot's `outcome.table ?? tableForUpdate` fallback with it — the comment described the no-op that can no longer reach that line, and a success always carries a table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a * refactor(tables): classify failures by type instead of by message text The table module decided HTTP statuses by searching error messages for phrases. `VALIDATION_MESSAGE_FRAGMENTS` and `ROW_WRITE_ERROR_PATTERNS` held 32 substrings between them, and fifteen more lists were inlined in routes — 83 matchers over 17 files, each its own copy of the guesswork and already drifted apart. It made message wording load-bearing: `TableRowLimitError`'s own doc comment noted that its text had to contain "row limit" for a route to answer 400, and adding "already exists" to a rename message silently demoted a 409 to a 400 (the bug fixed one commit ago, by adding another special case). Services now throw `OrchestrationError`, which carries the transport-neutral `OrchestrationErrorCode` the layers above already speak. Classification is one `instanceof` in `orchestrationErrorResponse` (UI + v1) and `v2CaughtOrchestrationError` (v2). Every pattern list is gone. Wording is free to change; an unclassified error still becomes a generic 500, which is what an unexpected fault should be. `asOrchestrationError` walks the `cause` chain rather than testing the caught value directly: drizzle wraps a throw raised inside a transaction callback in a `DrizzleQueryError` whose own message is the failed SQL, so a bare `instanceof` would drop every failure raised inside `withLockedTable`. That is the same reason `rootErrorMessage` had to dig for a root cause before. Three throws stay bare `Error` deliberately — `Table ID mismatch`, `Workspace ID mismatch`, and `Failed to build upsert conflict predicate` are internal invariants no consumer classified, and they keep falling through to a 500. `Insufficient capacity` was in the pattern list with no producer anywhere in the codebase. Status changes, all deliberate: - `'forbidden'` joins the code union so the table-row-limit ceiling keeps its 403; without it this refactor would have flattened it to 400. - import-async's table-limit rejection: 400 -> 403, matching the two other create routes it had drifted from. - Renaming a table to an invalid name: 500 -> 400. `validateTableName` messages don't contain "Invalid", so no matcher ever caught them. - Restoring a table that isn't archived, or into an archived workspace: 500 -> 400. - A duplicate *column* name stays `validation`/400 rather than becoming a 409 like a duplicate table name. Both v1 and the orchestration have always answered 400 for it; changing a published status is not this refactor's job. The twelve tests that changed were asserting the substring mechanism itself, constructing plain `Error`s with magic strings. They now assert the real contract, plus new cases pinning that identical wording carrying no classification stays internal and keeps its message off the wire. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials (#6150) * feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials * fix(api): correct credential role, skill permission bar, MCP url identity, and custom-tool conflict mapping * fix(api): align credential mutation gating, provider-outage status, and unique-violation conflicts * fix(api): close unique-violation, revival, orphan-write, and env-rename gaps * fix(api): treat every provider-outage code as unavailable on create and update * fix(credentials): use the shared outage predicate on the session update path * fix(contracts): anchor the predicate double-cast annotation to the cast `check:api-validation:strict` counted 9 unannotated double-casts against a baseline of 8, failing CI. The predicate leaf schema was annotated, but the annotation sat above the declaration while the checker anchors on the line carrying the cast — five lines below, at the close of the object literal. The scanner walks back at most three lines and stops at the first non-comment one, so it hit `value: z.unknown().optional(),` and never saw the reason. Splitting the object schema from the cast puts them adjacent, so the existing reason binds. No behavior change — the cast, the schema, and the reasoning are unchanged. Also lowers the rawJsonReads ratchet 6 -> 5 to match the current count, which had drifted down; leaving it high lets a removed raw read silently come back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(skills): point the orchestration error contract at its moved module #6150 branched before #6134, so skill-lifecycle.ts imports @/lib/workflows/orchestration/types — the module #6134 moved to @/lib/core/orchestration/types. Git merged a file deletion on one side with a new file referencing it on the other: no textual conflict, broken build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(knowledge): make lib/knowledge/orchestration the single implementation (#6154) * refactor(knowledge): make lib/knowledge/orchestration the single implementation Knowledge base create was implemented four times — the internal route, v1, v2, and the copilot tool — and the orchestration around the shared write had drifted. Extract it the same way lib/table/orchestration was: services write, orchestration decides which writes run, guards them, audits them, and returns a transport-neutral failure. Behavior converged, not preserved: - One chunking default (DEFAULT_CHUNKING_CONFIG). The agent defaulted minSize to 1 against the API's 100, so identical input produced differently-chunked knowledge bases depending on who created it. The agent path now chunks at 100. - Every successful mutation is audited inside the orchestration function. The copilot tool called recordAudit zero times, so agent-created knowledge bases, document uploads, updates and deletes left no audit trail at all. - Failures classify by class, not by message text. The knowledge service errors are OrchestrationError subclasses and storage-quota rejections throw a shared StorageLimitExceededError, replacing four separate message greps for "already exists" / "does not have permission" / "storage limit". delete_connector reported the opposite of what happened. It reached the route through an internal HTTP self-call that sent no query string, so the route's keep-documents default always applied while the agent told the user the documents had been removed. The self-call is gone — all four connector operations run in-process — and the orchestration returns the real counts. Also: - OrchestrationErrorCode gains 'payload_too_large' (413 / PAYLOAD_TOO_LARGE). Without it, dropping the storage-limit message match would have regressed the documented 413 on knowledge base create and document upload to a 500. - messageForOrchestrationError renders a route's own wording for an unclassified fault, so a driver's message no longer reaches the client on a 500. - v1 and v2 knowledge base update now forward actorUserId, which the service requires for a workspace move; both omitted it. - The connector DELETE route reads deleteDocuments through parseRequest. Its contract declared z.boolean(), which would have rejected the string a query param actually is. - Drop the 409 from POST /api/v2/knowledge/{id}/documents in the OpenAPI spec. Nothing on the upload path throws a conflict; it was only ever reachable by the message match this change removes. Behavior change worth noting: a v1/v2 PUT carrying only the workspaceId scope field and no actual updates now returns 400 rather than 200 with the unchanged knowledge base. Deliberately deferred: document update remains internal-only. Extracting performUpdateKnowledgeDocument makes exposing it on v1/v2 a contract and a route away, but that is a new public surface rather than part of this consolidation. * fix(knowledge): make connector create atomic and stop flattening failures Review round 1 on #6154. - Resolve the billing payer before the connector is committed, not after. A malformed attribution header rejected post-commit left a live connector behind a 500, and a retry created a duplicate plus duplicate sync work. Manual sync resolves before writing its audit for the same reason. - Let the source-config validator carry its own failure class. Collapsing every rejection to `validation` flattened the connector PATCH route's 401 (stale stored credential) and 409 (missing workspace context) into a 400. - Add `unauthorized` to OrchestrationErrorCode. It is the class that 401 was already expressing on this route, and the v2 vocabulary already had UNAUTHORIZED; only the shared union was missing it. - Report a knowledge base that exists but failed to archive as failed, with the reason, rather than as not found. The copilot delete loop folded every non-not-found failure into `notFound`, telling the user it was never there. - Route copilot failures through the same message helper the HTTP surfaces use, so an unclassified fault's raw text (a driver's failed SQL) no longer reaches the agent verbatim while the UI and public APIs get the generic wording. * feat(api): expand the public v2 files surface (#6160) * feat(api): expand the public v2 files surface Adds folder support, rename/restore, move, bulk archive, share, and content replace to /api/v2/files, so managing files by API no longer stops at upload + download + archive-one. Routes are thin: auth -> parse -> perform* -> serialize. Share and content replace get their orchestration extracted first so the session routes and the public ones cannot diverge on the effective-authType resolution, the EE public-sharing gate, or the storage-quota classification. Presigned upload stays session-only: presign does an advisory quota check and the real debit happens in the separate register step, so a caller that never registers leaves unaccounted bytes with no reaper. The buffered multipart path debits inside uploadWorkspaceFile's own transaction. * fix(files): classify folder and content failures instead of 500ing them Bugbot round 1. The v2 routes map errorCode straight to a status, so every manager failure that arrived unclassified became a 500 for what is really a caller-fixable 400 or 404. - Folder manager throws OrchestrationError: missing target/folder -> not_found, reparent cycle / self-parent / restore-into-archived-workspace -> validation. - File manager does the same for the in-transaction 'File not found' paths that the earlier pass missed. - updateWorkspaceFileContent's outer catch re-wrapped everything in a bare Error, which stripped the class off StorageLimitExceededError and the new not_found alike. It now rethrows a classified failure untouched and attaches cause to the generic wrap, so asOrchestrationError can still walk the chain. - Every remaining perform* gained the asOrchestrationError branch. - renameWorkspaceFile returned the pre-update read, so the v2 PATCH reported a stale updatedAt; it now returns the timestamp it actually wrote. Docs: upload auto-suffixes a duplicate name rather than rejecting it, matching the in-app uploader. The description claimed 409 and was simply wrong. * fix(files): surface a failed upload read-back as the real error getWorkspaceFile swallows a query failure and returns null unless throwOnError is set, so a transient blip on the post-upload read reported as 'file could not be read back'. Distinguish the two: a real null after a just-committed write is an invariant break, a query failure is itself. * revert(api): drop the dedicated v2 file-folder routes File folders already live in the shared folder table as resourceType 'file' (#6045 cut them over, #6051 dropped workspace_file_folders), and the remaining file-specific folder machinery is being folded into the generic folder engine. Publishing /api/v2/files/folders/** would pin that transitional split into a public contract we'd then have to keep or break. Files stay folder-aware — folderId/folderPath on the projection, folderId on upload, and the move route — because a folder id is a folder.id and survives the unification untouched. Folder management belongs on /api/v2/folders once that surface serves resourceType 'file'; until then there is no v2 way to enumerate file folders, which is the deliberate gap. The orchestration classification fixes stay: the internal routes and the copilot file-folder tools still call those perform* functions. * fix(files): classify upload failures instead of matching their wording Bugbot round 2. uploadWorkspaceFile had the same outer-catch rewrap that updateWorkspaceFileContent did, so a blown storage quota reached the route as a bare Error and the v2 handler recovered the status by substring-matching the message. Any rewording silently demoted a 413 to a 500. - uploadWorkspaceFile rethrows a classified failure untouched and attaches cause to the generic wrap. - FileConflictError is now an OrchestrationError('conflict'), so a duplicate name classifies like every other conflict. Its 'FILE_EXISTS' discriminator had no readers and is gone; the instanceof checks elsewhere still hold. - The v2 upload handler uses v2CaughtOrchestrationError, dropping all three string matches. Also documents that bulk-archive is best-effort: unknown or already-archived ids are skipped rather than failing the call, and deletedItems is what actually happened. That asymmetry with the single-id DELETE was undocumented. * feat(api): add search, filtering, and sorting to the v2 list endpoints (#6189) * feat(api): add search, filtering, and sorting to the v2 list endpoints One convention across every v2 list, documented on lib/api/contracts/v2/shared.ts: `search` (case-insensitive substring on the resource's natural name field), `sortBy` + `sortOrder` (per-resource enum, never a free string), and enumerated resource-specific filters. Reuses the sortBy/sortOrder pair v2 logs and v2 knowledge-documents already ship rather than inventing a third dialect alongside the Logs filters and the Tables predicate grammar. Every filter and sort is pushed into SQL. GET /api/v2/files previously read the whole scope and sorted/sliced it in JS; it now goes through a new queryWorkspaceFiles that filters, orders, and bounds the page in one query. Cursors are stamped with the sort they were minted under, so replaying one under a different sort is a 400 instead of silently duplicated or skipped rows. * fix(api): validate v2 cursor key values and compare timestamps at ms precision Two review findings, fixed at the root by making a keyset key own its cursor codec instead of hand-writing a decoder per sort. Cursor key values are caller-controlled, and matching the sort stamp and key count was not enough: an unparseable timestamp or a non-numeric size reached the query as an Invalid Date or NaN and surfaced as a 500. Each key now type- checks its own value and rejects a cursor it cannot hold, which both routes render as the documented 400. Timestamp keys now order and compare on date_trunc('milliseconds', col). Postgres keeps microseconds and defaultNow() populates them, but a cursor value round-trips through a millisecond-only JS Date — comparing the raw column against the truncated value re-admitted the page's own last row, duplicating it and stalling pagination outright at a page size of one. Reachable today via workspace_files.updated_at, which insertFileMetadata leaves to defaultNow(). * feat(api): complete the v2 workflows resource with versions and CRUD (#6184) * feat(api): complete the v2 workflows resource with versions and CRUD Adds version listing/detail plus create, update, and delete to the v2 workflows surface, which previously covered only execution and deployment. - GET /api/v2/workflows/[id]/versions — cursor-paginated, newest first - GET /api/v2/workflows/[id]/versions/[version] — version + pinned state - POST /api/v2/workflows, PATCH and DELETE /api/v2/workflows/[id] All six delegate to the existing orchestration and persistence helpers; no new domain logic. * fix(api): check folder containment before lock state; reject malformed version cursors assertFolderMutable walks a folder's ancestor chain without filtering on workspace, so inspecting it before containment let a caller tell a locked folder in someone else's workspace (423) from a nonexistent one (400). Create and update now assert containment first, matching the ordering import-workflow.ts already uses. A version cursor that decodes to JSON without a numeric version filtered every row out and returned an empty page with nextCursor null, which reads as a clean end-of-list. Malformed cursors are now a 400. * refactor(api): page workflow versions in the persistence helper listWorkflowVersions read every version row and the route filtered and sliced the result in memory, so the response was bounded but the query was not. It now takes optional limit/afterVersion, turning the cursor into a real keyset query; the route asks for limit + 1 and only trims the has-more probe. Both params are optional, so the internal, v1 admin, and copilot callers are unchanged. Also restores the untouched GET handler in [id]/route.ts to its original formatting — collapsing its signature had re-indented the whole body and buried the actual additions in whitespace churn. * feat(api): expand v2 tables with stateless multipart transfers (#6188) * feat(api): expand the public v2 tables surface Adds 16 operations so a v2 caller can do what the internal surface can: rename/move/lock a table, restore it, manage saved views, run enrichment columns, look up rows, and import/export with observable job control. Extracts lib/table/orchestration/import.ts (performTableCsvImport, performCreateTableFromCsv) and lib/table/export-stream.ts from the first-party routes, then repoints those routes at them, so v1 and v2 cannot drift on what an import or export actually does. events/stream, metadata and dispatches stay internal — they are editor state, not public API. * fix(api): make v2 table PATCH all-or-nothing and name the lock in every 423 Greptile P1: PATCH applied locks, rename and move as three sequential transactions, so a folder rejected mid-request left the earlier writes persisted while the response reported failure — and the schema-changed signal was skipped, leaving open clients on stale state. Every rejectable condition now runs before the first write, and the signal fires whenever anything did land. Cursor: v2TableLockError dropped the lock kind, so async import, column run, enrichment and table mutations returned a bare LOCKED. A table has four independent locks, so the caller could not tell which to clear. * fix(api): report the lock kind on classified 423s too, not just thrown ones The previous commit named the lock only where the rejection was thrown and caught at the route boundary. Where it instead arrives as a classified `errorCode: 'locked'` outcome — delete table, delete row, update column, and the table mutations — the kind was dropped, so those 423s stayed unactionable while their neighbours improved. The orchestration results now carry `lock`, and a shared `v2TableOrchestrationError` renders both arrival paths into the same `{ code, message, details: { lock } }` body. `details` is omitted rather than sent null when the kind is unknown, so a caller branching on it sees absence instead of a phantom value. * fix(api): make async table imports observable, not just startable `POST /import-async` pointed callers at `GET /api/v2/tables/jobs` to track progress, but that endpoint filters to `type = 'export'` — imports are derived onto the table itself, one write job at a time, and exports get a separate list precisely because they are excluded from that derivation. The public Table shape omitted those derived fields, so an async import could be started and cancelled but never observed to completion, failure, or progress. That is the gap the import/export/job-control set was meant to close. Table now carries `job` — id, type, status, rowsProcessed, error, or null when idle — and the import-async docs point at the table rather than the export list. * feat(api): make v2 table PATCH state which operations landed on failure Greptile held the PR at 4/5 on the residual non-atomicity and named two acceptable resolutions: make PATCH atomic, or have the contract adopt and expose partial-success explicitly. Atomicity would mean threading one transaction through renameTable, moveTableToFolder and updateTableLocks — three shared service functions with four non-test callers including the first-party route and two copilot tools — and deferring their per-operation audits to commit time. That is a refactor of shared write paths well outside this PR. So the contract states it instead. Every rejectable condition is already pre-validated, so a failure here is a genuine fault; when one follows a successful operation the error now carries `details.applied` listing what is live. Absent when nothing applied, so its presence always means "these changes took effect despite the error". Documented on the operation. `v2ErrorForOrchestration` gained the optional `details` this needs. * fix(api): make table lock flags read-only on the public v2 surface The new PATCH /api/v2/tables/[tableId] accepted a `locks` object, gated on workspace admin plus the table-locks feature. That still lets an API key clear the guard placed there to stop it: `write` is the floor for the endpoint, and admin keys are ordinary API keys, so a lock is no longer a boundary the key cannot cross. Locks stay readable on the table resource and enforcement is unchanged (a locked verb still returns 423). Changing one is now a first-party admin action only. The v2 body is declared here rather than reusing the first-party updateTableBodySchema, which keeps its `locks` field so the UI can still toggle them. It is .strict(), so a request carrying `locks` is rejected with a 400 naming the field instead of silently succeeding without applying it. * fix(api): keep reporting applied operations when the PATCH re-read fails The composite table PATCH promises that `error.details.applied` names the operations that are live despite an error, but `applied` was scoped inside the try. A rename or move that committed and was then followed by a throw in the final re-read — or a re-read finding the table archived — returned a bare 500/404 with no details, telling the caller nothing had landed. It would then retry into a duplicate-name conflict or repeat the move. `applied` is now function-scoped so every post-write exit carries it: the 404 on a missing re-read, a thrown lock error, a classified orchestration error, and the generic 500. `v2TableLockError` gains the same `extraDetails` parameter `v2TableOrchestrationError` already had. * feat(api): add workflow group writes to the v2 tables surface v2 exposed GET /groups but none of the writes, so the public API could run an enrichment or workflow column and read its binding, but never create one. A caller could add a plain data column and trigger the machine; wiring the two together still required the UI. Adds POST/PATCH/DELETE on /api/v2/tables/[tableId]/groups. The group is the unit that fills columns — one group feeds several — so creating one creates its output columns in the same call, matching the first-party shape rather than inverting it onto the column endpoint. Four departures from the first-party body, all public-surface concerns: - group.id is optional and server-generated. The UI mints an id to render optimistically; a public caller has no such need and a client-chosen id is a collision waiting to happen. - outputColumns[].workflowGroupId is dropped from the body and stamped from the resolved group, so it cannot disagree with it. - autoRun defaults to false. First-party defaults true so a UI add fills cells immediately; here it would make one POST fan out a metered run across every existing row. - A group naming neither a workflowId (type manual) nor an enrichmentId (type enrichment) is a 400 rather than a half-specified group the route has to guess about. Also rejects an outputColumns entry no group output feeds — the two arrays are joined by column name, and the first-party client builds both from one picker so it cannot desync, but a public caller can. Workspace containment on workflowId is asserted before it is persisted, on create and on any update that re-points the group; without it a table becomes a way to invoke workflows the key cannot otherwise reach. * improvement(api): make v2 table import and export async-only Drops the three synchronous entry points: POST /tables/[tableId]/import, POST /tables/import-csv, and GET /tables/[tableId]/export. Sync import tied a write to the lifetime of an HTTP request. The body *was* the data, so it carried a 10 MB cap that Next silently truncates past — a partial import reporting success. It also had no job, so a timeout mid-write left rows in place with nothing to poll and nothing to cancel. The async path reads the file from storage instead: upload via POST /api/v2/files for a key, start with POST /import-async, watch GET /tables/[tableId] -> job, stop with POST /job/cancel. Sync export carried no such hazard, but one shape per operation beats two: with both removed the surface has exactly one way to move a table in or out, and the CLI wraps the extra calls. This also removes the last multipart handling in v2 tables. Those were the only routes bypassing parseRequest — form fields were parsed by hand against separate form schemas, outside the contract system every other v2 write goes through. Create-a-table-from-CSV is now two calls: POST /tables, then /import-async with createColumns. csvImportModeSchema is append|replace, so there is no single-call create. Route baseline 1064 -> 1061. * docs(api): correct the import-async note about upload size limits The docstring claimed there is no synchronous upload endpoint and so no request-body size cliff. Both are wrong: POST /api/v2/files is a synchronous multipart upload with a 100 MB cap, and it is the only v2 upload path (presigned is deliberately absent). What async-only actually bought: the cap went 10 MB -> 100 MB, it fails on an explicit size check and a bounded body read rather than a proxy cap that silently truncates, authorization completes before any body is buffered, and the table write is a job that can be watched and cancelled. * feat(api): unify file and table transfers * improvement(api): make multipart transfers stateless * fix(api): make table import completion retries idempotent * feat(v2-tables): paginate the table list `GET /api/v2/tables` returned every table in the workspace in one response — it used the cursor envelope but hardcoded `nextCursor: null`, and had no `limit`. That was defensible when tables were only created through the UI; `POST /api/v2/tables` is public now, so a script can create them in bulk and the list has no way to ask for less. Adds `queryTables` alongside `listTables` rather than changing it, so the internal callers that genuinely want the whole scope are untouched — the same split `queryWorkspaceFiles` / `listWorkspaceFiles` already uses. Filter, order and slice all run in the query, so a `search` never costs a full-workspace read. A cursor whose values don't bind raises a validation error instead of being coerced to "no filter", which would have silently served page 1 under a resumed cursor. The keyset closes on `id` so a page boundary inside a run of equal names or timestamps stays stable. The shared `LimitQuery` doc component said "Maximum rows to return"; it now serves the table list too, so the wording is resource-neutral. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(api): add multipart knowledge document uploads * fix(api): keep usage admission at knowledge upload session creation * feat(knowledge): wire knowledge base uploads to multipart sessions * fix(knowledge): refuse to abort an upload once a document is bound * fix(uploads): prevent multipart cleanup races * Unify file creation and signed upload sessions (#6264) * feat(uploads): unify signed upload sessions * fix(uploads): preserve attachment storage semantics * feat(files): add authored file creation * fix(uploads): omit hoisted S3 metadata headers * feat(api): add file metadata endpoint * improvement(api): scope folders to resource paths (#6284) * improvement(api): scope folders to resource paths * fix(files): serialize folder resolution with uploads * fix(files): release folder lock before upload setup * fix(api): normalize folder paths and unblock resource mutations * fix(api): make resource cleanup and metadata consistent * improvement(uploads): persist multipart sessions in postgres * fix(db): store table row trigger timestamps in UTC * improvement(api): default folder deletion to non-recursive * fix(billing): unify chat usage source * improvement(logs): expose trace spans on log detail * fix(logs): parse list trace spans * improvement(api): replace workflow jobs with execution resources (#6294) * improvement(api): replace workflow jobs with execution resources * fix(api): preserve legacy jobs while preferring v2 executions * fix(api): make execution polling resume-aware * fix(ui): hide async examples for public workflows * fix(api): bridge resume queue visibility lag * feat(api): add v2 workflow resume endpoint * fix(api): project pending resume attempts * fix(api): prefer terminal logs over stale resumes * improvement(api): unify v2 resource query layers (#6319) * improvement(api): unify v2 resource query layers * fix(api): address v2 review findings * fix(api): preserve cancelled queue status * fix(api): guard cancelled job transitions * fix(api): close v2 resume and log gaps * feat(api): rename v2 executions to runs * feat(api): split credentials and secrets * feat(api): add workspace metadata and email attribution * improvement(api): consolidate public v2 route handling * improvement(files): centralize operations across APIs and Copilot (#6392) * improvement(files): unify rename authorization * chore(skills): add file operation migration guide * improvement(files): consolidate file operation authorization * improvement(files): extract shared operation foundation * improvement(api): simplify internal route declarations * improvement(files): centralize application authorization * refactor(api): share workspace file name validation * refactor(files): centralize copilot application calls * docs(skills): generalize application operation migration * improvement(api): centralize remaining v2 resource operations (#6412) * improvement(api): centralize v2 resource operations * fix(api): preserve custom tool conflict errors * improvement(api): migrate policy-sensitive v2 reads (#6410) * improvement(workflows): centralize v2 application operations (#6411) * refactor(api): migrate v2 knowledge operations (#6413) * refactor(api): migrate v2 knowledge operations * fix(knowledge): fail upload completion on dispatch errors * fix(knowledge): preserve upload retry and VFS errors * improvement(tables): centralize v2 application operations (#6414) * improvement(tables): centralize v2 application operations * fix(tables): preserve run validation and signals * feat(auth): add scoped internal executor delegation (#6459) * feat(auth): add scoped internal executor delegation * fix(auth): derive delegation lifetime from one timestamp * Include share status in file metadata * feat(auth): centralize delegated identity policy (#6462) * improvement(copilot): consolidate application adapters (#6450) * improvement(api): harden application route boundaries (#6451) * improvement(api): harden application route boundaries * fix(folders): reject creates at workspace cap * fix(knowledge): enforce trusted workspace scope (#6452) * fix(knowledge): enforce trusted workspace scope * refactor(knowledge): declare v2 body lifecycle * finish knowledge application migration * refactor(knowledge): compose copilot batch commands * fix(knowledge): parse connector query flags * fix(knowledge): finalize partial batch effects * fix(knowledge): align merged application boundaries * fix(knowledge): close application boundary review gaps * style(knowledge): satisfy branch biome checks * fix(knowledge): page connector documents in editor * refactor: enforce Copilot table application boundary (#6453) * refactor: enforce copilot table application boundary * fix(tables): finish application boundary migration * fix(tables): restore scoped copilot imports * fix(tables): compose copilot commands atomically * fix(tables): preserve workflow group scheduling * fix(tables): complete fixed copilot composition * fix(tables): reject enrichment output mutation * fix(tables): complete authorized application boundary * fix(workflows): migrate Copilot application boundary (#6455) * fix(workflows): migrate Copilot application boundary * fix(workflows): finish delegated application migration * fix(workflows): encode VFS folder aliases * fix(workflows): close application composition gaps * fix(workflows): preserve VFS validation errors * fix(workflows): complete application boundary migration * test(workflows): format canonical binding coverage * fix(workflows): scope executor metadata reads * fix(workflows): bind executor metadata targets * improvement(skills): align application operation guidance (#6532) * feat(api): expose v2 resource owners * fix(api): distinguish visible resource authorization failures (#6537) * feat(api): generate v2 OpenAPI from contracts (#6509) * feat(api): generate v2 OpenAPI from contracts * fix(api): preserve string boolean wire defaults * fix(api): document file download headers * fix(docs): use TypeScript CLI with Next.js * fix(docs): avoid client-rendered theme script * fix(api): document departed audit default * feat(api): replace legacy core docs with v2 * feat(api): generate v2 OpenAPI from contracts * feat(api): refine generated v2 OpenAPI docs * fix(docs): align localized v2 execution examples * fix(ci): restore Helm diff and sync audit mock * fix CI regressions after staging merge --------- Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Theodore Li <theodoreqili@gmail.com> Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com> Co-authored-by: Theodore Li <theo@sim.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-08-11 05:45:25 -04:00