912 Commits

Include renames

Author	SHA1	Message	Date
Waleed	3ff91f0439	improvement(docs): clean up leftovers from the code-block alignment PR (#6825) ... * improvement(docs): clear leftovers from the reverted revisions A cleanup pass over the final state. Every finding was residue from an approach this PR tried and abandoned, or a claim that stopped being true when it did. - Delete the copy-button svg sizing rule: a later rule sets `display: none` on that same element ungated, so sizing it was never observable. Superseded by the mask approach. - Drop the paragraph in page.tsx arguing about a custom Shiki factory. The factory was deleted; nothing configures one now. - Correct shiki-curl-json.ts, which still claimed the grammar "reaches the client path too". It does not — that was the justification for choosing a grammar over a transformer, so leaving it stated the opposite of the truth. Now records where it applies, where it does not, and why not to retry. - Correct the global.css section header, which claimed the component owns the shell while the next rule defines it here. - Qualify the `--copy-glyph` declarations with `:has(> svg[class*="lucide"])`, which the group's own comment asserts of every rule in it. - Correct `getCode`'s TSDoc: the gutter is a `::before`, and pseudo-element content never reaches `textContent`, so line numbers were never what the clone guards. It guards transformer-emitted `.nd-copy-ignore` nodes. - Compose `chipGeometryClass` and emcn's `ChipChevronDown` in the API example selector instead of restating their literals. - Merge the duplicated `div[role="region"]` rule. The tablist pair stays split: biome's `noDuplicateProperties` reads a nested `@variant` setting the same property as a duplicate and fails the build — recorded so it is not remerged. - Note that fumadocs ships its own gutter for `lines`-meta fences, which cannot be suppressed from here and would paint a second column. * fix(docs): drop a highlighter registration that can never fire fumadocs-openapi calls `renderCodeBlock` with a hard-coded `"json"` from both of its call sites (`request-tabs.js:76`, `response-tabs.js:48`), so the docs `CodeBlock` it routes through never receives a shell language. The `getHighlighter('js', { langs: [curlJsonBodyGrammar] })` registering the shell-scoped JSON-body injection therefore did nothing but await on every API sample render, and the docblock claiming the grammar covers those samples was wrong. - Delete the call and its imports. - State the grammar's real coverage: prose fences only, via `langs`. Both API reference paths are unreachable — samples are JSON, and the cURL usage tabs highlight client-side off fumadocs' own factory. - Correct `code-block.tsx`'s TSDoc, which still said API samples come from fumadocs' own renderer. They come through this component; `UsageTab` is the renderer that bypasses it. - Re-home a comment orphaned when two CSS rules merged — it had drifted onto the rule below and read as documenting it. - Drop a `.nd-copy-ignore` claim about transformers emitting those nodes; nothing here does, and upstream parity is the reason the clone exists.	2026-08-18 15:28:02 -07:00
Vikhyath MondretiandClaude Opus 5	3a03774e42	fix(forks): stop copying connector-managed knowledge base documents (#6818) ... * fix(forks): stop copying connector-managed knowledge base documents A fork copies a KB's documents but never its connectors, so a connector-sourced document arrives with `connector_id` nulled and its `external_id` intact. The sync engine keys every existing/tombstone/ exclusion lookup off `connector_id`, so that copy is invisible to it - never updated, reconciled, or purged - and `doc_connector_external_id_idx` does not constrain it either, since its `connector_id` is NULL. Attaching a connector in the child then re-ingests every page as a NEW row on top of the snapshot. Each fork hop re-copies the previous hop's orphans and adds one more generation, so a prod -> UAT -> staging chain leaves three rows per page and a knowledge search returns the same page three times, one of them serving content frozen at the fork date. Exclude connector-managed documents from all four doors a document can enter a fork through: the whole-KB content copy, the in-transaction placeholder pre-creation, the sync-only copy into an already-mapped KB, and the content fill (guarded for payloads planned by a pre-change worker mid-rollout). The placeholder path matters as much as the copy loop - filtering only the content phase would leave a permanently archived row behind a persisted `knowledge_document` mapping. Skipped on both sides, the reference clears like any other uncopied document's. A document whose connector was deleted already has a null `connector_id` (the FK is ON DELETE SET NULL) and is static in the source too, so it still copies. One count(*) per copied KB logs what was left behind, since a fully connector-synced KB now forks to zero documents. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): keep the skipped-document count from failing a copied KB The connector-managed count feeds a log line, but it sat inside the KB's try block, so a transient failure on a COUNT(*) would roll back a copy that had otherwise succeeded and clear every reference to it. Move it into a helper that swallows its own error. Counting is not copying: only the copy itself may fail a resource. Test proven red by removing the catch - the mutation reports a knowledge-base failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): clean up full-KB placeholders planned before the exclusion The mapped-KB fill guarded a pre-change plan, but the full-KB path did not: a placeholder planned by an old worker for a connector-managed document is simply no longer returned by the page query, so nothing fills it and it stays archived behind a live mapping that a remapped document-selector still resolves to. Report those child ids as failed documents so the shared cleanup clears their references and drops the rows, and delete their persisted identity so a later sync does not resolve to a row cleanup removes. Keyed on the SOURCE being connector-managed, which can never become copyable, so it cannot race a concurrent attempt mid-fill the way a "source is gone" check could. The mapping drop is now one helper shared with the mapped-KB catch. Test proven red by removing the reconciliation block. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): make the stale-plan probe best-effort The probe ran inside the KB try, so a transient SELECT would reach the catch, roll back a complete copy, delete the child base, and clear every reference to it. Weighing it as "load-bearing, so fail closed" was wrong: the probe runs on EVERY copied KB that has referenced documents, while the state it repairs exists only inside a rollout window. Failing closed traded a common-path outage against a rare-squared one. It now swallows its own failure with a loud error log, leaving that pre-existing state in place rather than destroying a good copy. Test proven red by removing the catch - the mutation reports the KB failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-18 14:23:19 -07:00
Waleed	1c69372cba	feat(cli): follow a run, wait for one, and tail the log (#6813) ... * feat(cli): follow a run, wait for one, and tail the log Three commands the surface was missing, each polling or streaming something the generated command layer cannot express. `workflows run --follow` renders the SSE the execute route already emits, so a multi-minute agent run stops printing nothing until it ends. It rides on the generated `run` leaf rather than a sibling command — same operation, one different response encoding — and delegates to the handler it replaced, so every non-follow invocation still runs the generated path. Answer text, thinking and tool calls go to stderr; only the final envelope reaches stdout, so redirecting still yields the result. Reasoning and tool frames need the `X-Sim-Stream-Protocol` header, which is sent only when asked for, because negotiating also switches answer text to live chunks the server may retract. `workflows runs wait` closes the loop `--async` opens. Terminal is completed, failed or cancelled; `redacting` is not, since a run whose output is still being scrubbed is not yet a run you can read. A time pause keeps polling because the server resumes it, and a human pause stops with the resume command rather than burning the bound and calling it a timeout. Distinct exit codes keep cancelled and paused from reading as failure. The bound is `--wait-timeout` and not `--timeout`, because SIM_TIMEOUT_SECONDS already bounds one request and two knobs of the same name hide each other. `logs follow` tails runs as they arrive. Dedup keys on run id, not on the timestamp: a schedule fan-out starts many runs in the same millisecond, so a timestamp watermark either drops the siblings or reprints them. JSON output is one object per line, because a follow never closes an array, and the table header is printed once so columns stay aligned across polls. * fix(cli): disclose a truncated burst, and clear a stale retry notice Two review findings in `logs follow`, both verified against the code first. The page budget bounds one poll so an enormous burst cannot stall the follow, but on reaching it the live cursor was discarded: the remainder is older than everything collected and the next poll restarts at the newest page, so those runs were never printed and nothing said so. The budget stays — draining without one trades a bounded poll for unbounded buffering in a process meant to run for hours — but hitting it now warns on stderr, naming the count and pointing at `sim logs list`. That notice is written even off a terminal, because a piped log is where an unexplained hole is hardest to spot. The retry notice was cleared after the empty-rows check, so a poll that recovered but found nothing left "retrying in Ns…" on screen while the follow was already healthy. Clearing now happens as soon as a poll succeeds. The second test needed two failures to be worth anything: the teardown clears the line either way, so what separates fixed from broken is whether a bare erase lands before the second notice or only at the end. The first version passed against the bug. * test(cli): pin that a mixed page is the watermark, not a truncation A page holding a run already printed proves the follow caught up, so the truncation warning must not fire there — that is how every healthy poll terminates, and warning would report a hole on the ordinary path. The straggler sharing that page is still collected, because the filter takes every unprinted row on it rather than only those above the known one. * fix(cli): say when the requested backlog was larger than a page holds The logs API clamps `limit` into 1–1000 rather than rejecting it, so `logs follow -n 5000` came back with 1000 rows, anchored the floor to that partial page, and said nothing. The seed already knew — it computes whether a live cursor remained — but the caller discarded the answer. Guarded on both halves. Fewer rows than asked for is only a shortfall when more were waiting: a workspace holding ten runs answers `-n 50` with ten and nothing is missing, so warning on the row count alone would fire on every small workspace. The cursor is what separates the two.	2026-08-18 11:43:16 -07:00
Waleed	c17043a8b7	improvement(docs): align code blocks with the platform design system (#6810) ... * improvement(docs): align code blocks with the platform design system Docs code blocks rendered in stock `github-light`/`github-dark` on fumadocs chrome, sharing no colors, typeface, metrics, or corner radius with the app. - Add Sim Shiki themes transcribed from emcn's Prism token colors, shared by the MDX pipeline and fumadocs-openapi (which highlights through its own instance, so the API reference was left on the GitHub palette). - Use the mono stack the app actually renders. `tailwind.config.ts` points `font-mono` at `--font-martian-mono`, but nothing defines that variable, so every code surface in the product resolves to the system stack. - Give blocks the platform's field chrome — `rounded-lg`, a `--border-1` hairline, a `--surface-5`/`--code-bg` fill — and the 13px/21px metrics of `Code.Viewer`. The rule keys on `figure.shiki` because two renderers emit these figures and that is the only join point they share. - Number every line, from the same tokens as the in-app gutter. Padding sits on `.line` rather than fumadocs' `--padding-left`: that property is re-declared on the inner `pre` for API samples, which dropped the digits on top of the code. - Collapse tabbed fences into one box with the strip as the title row, and align the inline-code chip with the app's markdown renderer. - Reuse emcn's `Button`, `useCopyToClipboard`, and chip chrome constants instead of re-deriving them, and drop ~90 lines of `!important` overrides, including a rule that could never match. * fix(docs): stop line numbers overlapping code, unify the copy glyph The gutter opened its column by setting `padding-left` on `.line`, which never applied: fumadocs' own rule is `.shiki:not(.not-fumadocs-codeblock *) .line`, and `:not()` carries its argument's specificity, putting it at (0,3,0). Every code block rendered its line number on top of the first characters. - Drive fumadocs' `--padding-left` / `--padding-right` instead of overriding `.line`. Declared on the figure, the viewport, and any inner `.shiki`, because the variable is inherited and the nearest declaration wins — the class sits on the figure alone for prose fences but on the figure and the inner `pre` for API samples, and `--padding-right` is also written as an inline style on the viewport. - Route API request/response samples through the docs `CodeBlock` via fumadocs-openapi's `renderCodeBlock`, so they carry the emcn copy control rather than fumadocs' lucide clipboard. - Mask the emcn glyph over the one block `renderCodeBlock` cannot reach — the usage tabs hardcode `ClientCodeBlock` and `OperationClientOptions` exposes only `APIExampleSelector` — so the copy icon is identical everywhere. * improvement(docs): reserve the gutter column without numbering one-liners A line number on a single-line shell command has nothing to reference, and the CLI pages are mostly single-line commands. Dropping the gutter on those blocks was the original behaviour, but it made adjacent fences start their code 28px apart wherever a command sat next to its output. Reserve the column on every block so all code shares a left edge, and paint the digit only when the fence has more than one line. * fix(docs): drop the gutter entirely on single-line fences Reserving the column but leaving it blank gave one-line commands a 44px indent with nothing in it, which reads as a rendering fault rather than as alignment. Gate the column and the digit together, so a single-line fence keeps fumadocs' default padding and a multi-line one gets both. * fix(docs): stop the copy-button CSS restyling emcn's own Button The rules added for fumadocs' copy button matched on `aria-label` alone, so they also hit the emcn `Button` this app renders — re-declaring geometry, radius, color, and a `background: none` that killed its hover, and pinning docs to today's `buttonVariants` values with no failure signal if those change. - Qualify every one with `:has(> svg[class*="lucide"])`, the same scoping the mask rules already used, so they reach only the block fumadocs renders. - Stroke the masked glyph at 1.25 to match `Button size='icon'`, which overrides the icon's authored 1.55. The two copy glyphs were rendering at different weights — the mismatch the mask exists to remove. - Drop `.line::after { content: none }`: it cannot outrank fumadocs' (0,4,1) rule, and no fence in `content/` uses the `lines` meta it guarded against. - Drop the `!important` and the redundant viewport selector on `--padding-left`; nothing declares it between the figure and the region, and nothing contests it at equal specificity. `--padding-right` keeps both — its inline style is real. - Use emcn's `cn` where emcn class constants are merged, so they go through the merger that knows the `text-micro|caption|small|md` scale. - Correct the comments the review disproved: two claimed the API reference still renders fumadocs' CodeBlock, which `renderCodeBlock` changed. * fix(docs): keep the gutter padding override belt-and-braces A simplify pass removed the `!important` and the viewport selector from `--padding-left` as provably redundant, and on the numbers they are: fumadocs declares the property at (0,2,0) while these selectors are (0,3,1) and (0,4,1), and nothing declares it on the viewport. Restore both anyway. Getting this wrong paints the line numbers on top of the code — a regression this PR already shipped once — and the specificity of `:has()` and `:not()` is easy to miscount in exactly that direction. The comment now records both that the override is redundant on paper and why it stays. * feat(docs): highlight the curl JSON request body as JSON A `curl -d '{…}'` payload is one single-quoted string to a shell, so the same JSON that renders with colored keys in a response sample rendered as one flat block of string color in the request sample directly above it. Fixed with a TextMate injection rather than the two approaches that don't work: - `{ include: 'source.json' }` attaches the JSON grammar but its object pattern only assigns `support.type.property-name.json` — the scope that colors keys — when it owns the opening brace. Entering mid-string, keys stay string-colored, which is the whole difference. So the key/value/array patterns are written out and name that scope directly. - A Shiki transformer tokenizes it correctly but is a function, and the request tabs highlight in the browser off a `shikiOptions` object passed through RSC, where functions cannot cross. A grammar is plain data and reaches both sides. An injection has to be registered on the highlighter, not passed per call, so the API page moves to `createAPIPage` from `fumadocs-openapi/ui/base` with our own factory, and `ApiShikiProvider` hands that same factory to the client code blocks — both public API. The MDX pipeline preloads it through `langs`. The opening brace requires `}`, a quoted key, or end-of-line after it, which keeps `awk '{print $1}'` out; the end-of-line case is needed because Oniguruma matches line by line. Verified against `jq '.[0]'`, `awk '{print $1}'`, `grep -o 'foo'` and `echo '{}'` — none are re-colored. * fix(docs): paint the code fill on the viewport, not the figure The request and response panels on an API reference page rendered on different backgrounds. Sampled from screenshots: the request panel showed the page background (#ffffff light, --bg dark) while the response panel showed the code surface (--surface-5 / --code-bg). The fill was left to show through from the figure or the tab group, and those diverge per renderer. fumadocs gives a standalone figure `bg-fd-card` but an in-tab figure `bg-fd-secondary`, and this app forces `--color-fd-card: transparent` on API reference pages — so zeroing the in-tab figure's fill, expecting its group to supply one, left the request panel transparent while the response panel's `bg-fd-secondary` group kept ours. Paint it on the scroll viewport instead. That is the innermost box all three renderers wrap code in, so it cannot diverge, and it no longer matters what any ancestor sets. * fix(docs): hide the code tab strip's scrollbar fumadocs makes the strip `overflow-x-auto`, and an endpoint with ten status codes overflows it in the API reference's narrow rail — leaving a scrollbar across the bottom of a 34px header, which reads as the header being clipped rather than as something scrollable. Hidden the way the platform hides it on a scrolling tab strip: emcn's `TabStrip` carries `overflow-x-auto [scrollbar-width:none] [&::-webkit-scrollbar]:hidden`. The code viewport below keeps its scrollbar. There the overflow is content, and the platform's own `Code.Container` shows one for the same reason — hiding it would hide that a line continues. * fix(docs): keep fumadocs-openapi's server graph out of the client bundle The Vercel deployment went red at the commit that added `ApiShikiProvider`, and stayed red for three commits. That component is `'use client'` and imported `ClientCodeBlockProvider` from `fumadocs-openapi/ui/base` — an entry that also pulls `remark`, `remark-rehype`, `@fumari/json-schema-ts` and `github-slugger`. Importing it from a client module forces that whole server graph into the browser bundle. Measured in `.next/static/chunks`: `json-schema-ts` in 1 chunk, `github-slugger` in 3, `remark-rehype` in 4. A local build tolerates the weight; a deployment with size limits does not. `ClientCodeBlockProvider` lives in a `"use client"` module that the package does not expose through its `exports` map, so there is no client-safe path to it. Back to `createAPIPage` from `fumadocs-openapi/ui`, dropping the custom factory and the provider. After: `json-schema-ts` 0 chunks, `github-slugger` 1, `remark-rehype` 1 — the remainder is fumadocs' own client-side markdown. The server path keeps the injection by registering it on the shared highlighter `highlight` already resolves. What is given up is the API reference's cURL usage tabs, which highlight in the browser off fumadocs' own factory. Prose fences keep it, and that is where the `curl -d '{…}'` examples live — getting-started, authentication, workflows/deployment, passing-files, triggers/webhook, in every locale.	2026-08-18 11:12:44 -07:00
Waleed	edc25aa976	docs(helm): document null as the way to remove an inherited env key (#6801) ... * docs(helm): document null as the way to remove an inherited env key Setting `app.env.KEY: ""` cannot clear a key that `app.envDefaults` sets: the Secret template drops empty values, and the deployment template treats an empty override as "not overridden" and still inlines the default. Helm's own `KEY: null` deletion is the supported mechanism and already works. The empty-string behavior is load-bearing, not a bug — every key under `app.env` ships as a "" placeholder, and ten collide with a real `envDefaults` value (NEXT_PUBLIC_APP_URL, BETTER_AUTH_URL, ...), so "" has to read as "unspecified" or a default install would blank them out. - README: document `null`, with the --reuse-values and Argo CD valuesObject caveats; correct the claim that `app.env` always wins over `app.envDefaults` - values.yaml + self-hosting docs: same guidance where operators look - sim-helm skill: record why an unset list is the wrong shape here - tests: lock in that null removes a key and "" does not * docs(helm): correct the verify command's chart path and scope the required-secret claim - The verify snippet used a `sim/sim` repo alias that this chart never publishes; every other instruction installs from the local `./helm/sim` path, so the command could not run as written - Nulling a boot-critical key only fails at template time with the chart-managed Secret. `existingSecret` mode skips that validation entirely (the chart cannot read a pre-created Secret), and under ESO the key must instead be mapped in externalSecrets.remoteRefs.app * docs(helm): say null must be applied in every layer that sets a key `null` deletes a key from the map it is applied to, not from the pod. A key set in both `app.env` and `app.envDefaults` survives a null on the app.env entry alone — the deployment then inlines the envDefaults value again. Under ESO a retained `externalSecrets.remoteRefs.app` mapping keeps syncing the key regardless of app.env. - README and self-hosting docs: drop the "works in all three secret modes" shorthand and spell out that every layer setting the key must be nulled, including the ESO remote mapping - tests: cover both halves — nulling only app.env restores the envDefault, nulling both actually removes the key - chart 1.5.4; staging took 1.5.3 in the meantime	2026-08-17 18:16:35 -07:00
Waleed	9dc828f36a	fix(email): greet SMTP relays with a qualified hostname instead of [127.0.0.1] (#6799) ... * fix(email): greet SMTP relays with a qualified hostname instead of [127.0.0.1] Nodemailer derives the EHLO greeting from os.hostname() and substitutes the address literal [127.0.0.1] whenever that name contains no dot. Kubernetes pod hostnames never contain one, so every k8s deployment introduced itself to the relay as loopback and strict relays refused the session before any mail moved. Send the domain the app is served from instead, as RFC 5321 4.1.4 asks, with SMTP_EHLO_NAME to override it for relays that expect a different identity. * fix(email): parse EHLO address literals and drop a port from the app domain Review round 1. The bracketed branch matched a character class rather than an address, so [::::] and [13] reached the relay as a greeting it would refuse. Parse the address with node:net instead, which also admits the RFC 5321 IPv6: form. getEmailDomain reports a URL host, so a deployment served on a non-default port failed the qualified-name check and fell back to nodemailer's default — [127.0.0.1] again on Kubernetes, the exact failure this change exists to fix. Strip the port before validating. * fix(email): accept any casing of the IPv6 literal tag, and stop owning SMTP_EHLO_NAME in setup Review round 2. RFC 5321 tags the IPv6 address-literal form, and RFC 5234 makes ABNF string literals case-insensitive, so [ipv6:2001:db8::1] is as valid as [IPv6:...]. The exact-prefix check routed it to isIPv4 and discarded it. Drop SMTP_EHLO_NAME from the email capability's optional fields. SMTP_SECURE, the same kind of optional transport knob on the same provider, is not modelled there either, and claiming the field obliged the setup wizard to prompt for it — a field whose entire purpose is to stay unset now that the default is right.	2026-08-17 18:09:02 -07:00
Waleed	d17a11f29b	feat(jotform): trigger a workflow on every new form submission (#6802) ... * feat(jotform): trigger a workflow on every new form submission Jotform's only webhook event is a new submission, so the block gets one trigger. Deploying it registers the callback on the form through the API and undeploying removes it again. Two things about this provider needed handling: Jotform posts submissions as multipart/form-data, which the shared webhook body parser did not read — the delivery died as a 400 before any handler saw it. The parser now flattens a multipart body the same way it already flattens a urlencoded one, reducing an uploaded part to its filename so a stray file cannot inflate the execution input. The form's webhooks are identified by their position in the form's webhook map, so an id captured at registration goes stale the moment any other webhook on that form is removed. Nothing persists it; cleanup re-resolves the id by matching the callback URL. Registration checks the same way, because Jotform answers a rejected request with the unchanged list rather than an error. Answers are exposed as the parsed `rawRequest` rather than re-keyed by question label — the labels are not unique, and the payload shape is only documented as the raw q{qid}_{slug} map. The trigger's region field is named `apiRegion` so it does not collide with the block's own advanced-mode `region`. * fix(jotform): make webhook registration idempotent and URL matching tolerant Validated the trigger against Jotform's API reference and a captured delivery (zulip's multipart fixture), which confirmed every mapped field — formID, submissionID, formTitle, username, ip, type, pretty, rawRequest — and turned up three things worth correcting. Jotform keeps a form's webhooks as a plain list and does not treat the URL as a key, so posting one it already holds leaves the form delivering every submission twice. Registration now consults the list first and only posts when the URL is absent. The documented POST sample returns the new entry as "0", renumbering the rest, which is further reason nothing persists an id. URL matching no longer lets a trailing slash decide the outcome. Jotform stores the URL verbatim in every sample seen, but an exact match failing would hard-fail deploy, and Pipedream's client normalizes the same way. The rawRequest description claimed the field holds the submitted answers. A real payload also carries slug, buildDate, submitSource and jsExecutionTracker, and a file answer appears under the bare slugified label as upload URLs rather than under a q{qid}_ key — which is also why filtering to q-prefixed keys would silently drop file answers. * fix(jotform): keep the callback when an active deployment still needs it Redeploying prepares the replacement webhook row alongside the live one and a workflow keeps its path across deployments, so both rows resolve to a single callback on a single form. Registration adopts the callback already present instead of posting a duplicate, which left the retired row's cleanup deleting the one the new row had just adopted — the trigger went silent after a redeploy that changed the trigger config. Teardown now skips when another webhook row belonging to an active deployment resolves to the same form and callback URL, matching how the Telegram handler skips deleteWebhook while an active deployment still uses the same bot. A genuine undeploy has no such row and still cleans up.	2026-08-17 18:08:38 -07:00
Waleed	0e84e92d39	fix(cli): bound, trace, and explain the requests the CLI makes (#6798) ... * fix(cli): bound, trace, and explain the requests the CLI makes Four transport gaps, all of which failed silently. A request had no timeout, so a connection that was accepted and then never answered hung the terminal indefinitely. `SIM_TIMEOUT_SECONDS` now bounds one, defaulting to 3600s — deliberately above every timeout the server itself applies, since a synchronous workflow run is allowed 3000s on a paid plan and a tighter default would abort real work and report it as a transport failure. `0` removes the bound, for a self-hosted deployment that runs executions without one of its own. The caller's abort signal is composed with the timeout rather than replaced, so neither masks the other. Node ignores HTTP(S)_PROXY unless NODE_USE_ENV_PROXY opts in, and only from v22.21 and v24.5, so on a network that reaches the API only through a proxy every command failed to connect while the variable that would have fixed it was already set. The CLI cannot enable that from inside the process — Node reads it at startup — so it says what to do rather than bundling an HTTP stack for a setting the platform now owns. An API key was sent to any http:// endpoint with no signal. Now a warning, not a refusal: http is the documented way to reach a local dev server, and a deployment terminating TLS at a gateway is real. Loopback stays silent. `SIM_DEBUG=1` traces method, URL, status and duration. Bodies and headers are deliberately absent — the request carries the API key, and `secrets set` carries the secret itself. All four write to stderr, so a piped stdout stays parseable. * fix(cli): make the request bound safe on every runtime it supports Two ways the new timeout could fail before the request was made. `AbortSignal.any` arrived in Node 20.3 and this package supports Node 20, so composing a caller's abort signal with the timeout threw a bare TypeError on the earliest 20.x releases. It is now used when present and composed through an AbortController when not. `AbortSignal.timeout` rejects a fractional millisecond outright, and past 2^31-1 ms it does not fail at all — it clamps to 1ms, so the longest timeout anyone asked for became the shortest. The value is now rounded and refused above what Node can actually wait, pointing at 0 for an unbounded wait. Also unstubs env vars between tests: `stubEnv` is not undone by `unstubAllGlobals`, so a SIM_TIMEOUT_SECONDS set for one test configured every test after it. * fix(cli): correct the proxy version table, and classify a timeout mid-body `runtimeCanProxy` treated any release between 22 and 24 as capable, so on Node 23 — which reached end of life before the backport — a configured proxy was ignored and the CLI stayed silent about it, which is the exact failure the warning exists to report. The table is now the two lines that shipped the support, and anything after them. `AbortSignal.timeout` keeps firing after `fetch` resolves, so a bound that elapsed while the body was still being read — a large `files get` — escaped the client's own handling and printed a raw TimeoutError stack. The top-level handler now names it, which covers the streaming path as well as the JSON one. A user's own Ctrl-C raises AbortError and is deliberately left alone. * fix(cli): report a timed-out download as a timeout `files get --output-file` streams the body to disk, and `streamToFile` converted anything the stream threw into a write failure. So a request bound elapsing mid-download read as `Could not write <path>: ...`, sending the reader to check permissions and free space for a timeout they can raise, and hiding the one instruction that resolves it. The predicate and that instruction now live beside the timeout that raises them, so the client, the top-level handler and the download path all say the same thing. The wrapping stays where it is: the staged-download cleanup runs off that failure, and rethrowing past it would leak the temporary directory. * fix(cli): keep a sub-millisecond timeout bounded Zero is how this function says "no bound", so rounding a positive SIM_TIMEOUT_SECONDS down to zero inverted the request: anything under 0.0005s asked for the shortest possible timeout and got none at all, leaving a stalled request to hang. Introduced by the rounding that fixed the fractional-millisecond rejection. Floored at 1ms for every positive value; only a literal 0 still disables.	2026-08-17 18:07:36 -07:00
Waleed	0b4d34137b	feat(secrets): add optional descriptions to workspace secrets (#6796) ... * feat(secrets): add optional descriptions to workspace secrets Workspace secrets already have a backing credential row with a description column, but nothing surfaced it. Teammates had no way to record what a secret is for. - Add a Description field to the secret detail page, matching the integrations credential page, gated on workspace-secret admin - Fold the value and description editors into one Save/Discard pair and one unsaved-changes guard; two guards cannot coexist, since each seeds its own same-URL history entry - Match descriptions in the secrets settings search - Expose description on GET/PUT /api/v2/secrets and in the CLI Descriptions are workspace-only: env_personal credential rows are per-workspace mirrors of one user-global secret, so one saved there would exist in a single workspace, and a personal secret has no teammates to inform. The API rejects a description on personal scope rather than silently dropping it, and omitting it on PUT leaves any existing description untouched so a value rotation cannot erase it. * fix(secrets): address review findings on secret descriptions - Patch the credential detail cache optimistically on update. `onMutate` cancelled the detail query but only patched the lists, so a detail-backed editor stayed dirty after a successful save until the refetch landed — long enough for Discard to restore the pre-save value over the committed one, and for Back to open the unsaved-changes guard. - Memoize `useSecretValue`'s returned callbacks and object, per the hook convention, so the composed form's save/discard stop churning per render. - Reject a description on a personal secret in the domain layer rather than only at the v2 boundary. The internal credential update path accepted one for any type, writing data every reader hides. - Normalize an empty description to null so the API and UI agree. - Correct the secrets documentation, which described a Display Name field the detail view does not have and omitted the scope rule. - Drop the CLI's copy of the 500-character bound; it can't import the contract, so a copy only drifts from the message the API already returns. - Collapse a redundant save guard and align the description write gate with the render gate. Leaves the integrations credential page byte-identical to staging. * fix(secrets): keep the API docs example and CLI column order stable Backward-compatibility fixes for anyone who never sets a description. - Move the blank-to-null normalization out of the contract and into the route. A Zod `.transform()` on any property drops the whole request schema's OpenAPI examples, which had silently removed the Set Secret request example from the published docs. - Append the CLI `description` column instead of inserting it before `updated`. `--output text` is positional, so inserting would shift every field an existing script cuts. - Reject a description on a personal secret with a message that says so, rather than dropping the field and falling through to the generic "no updatable fields" error.	2026-08-17 17:33:35 -07:00
Waleed	38075ad977	fix(sap_concur): align the integration with SAP Concur's documented API (#6790) ... * fix(sap_concur): align the integration with SAP Concur's documented API Validated all 70 tools, the block, and both proxy routes against SAP's published API docs. Auth: - add the password and companyUuid to the token cache key so a request with the wrong password can no longer be served a cached token minted from someone else's - wire the documented company-level flow (username = company UUID, credtype = authtoken) so companyUuid actually scopes a token - expand the datacenter allowlist to the documented set (adds glz, apj1, usg, the impl hosts, and the www- twins) and drop the undocumented cn host; validate the returned geolocation by shape instead of membership - coalesce concurrent token fetches so a fan-out mints one token - forward Retry-After so 429 retries pace off Concur's own hint - handle the errorMessageList, SCIM detail, and legacy Error.Message shapes instead of falling through to a generic HTTP message - pin redirects and cap the response body Block: - collapse six contextType subBlocks that disagreed on their default, so a new block no longer seeds MANAGER for every operation - clamp contextType to each operation's documented set - stop requiring a userId and contextType that the default operation's tool does not accept, and scope the receipt fields to the upload ops - reach six params that had no subBlock, and pass userId on travel request updates so a stale value cannot impersonate Tools: - correct response shapes that resolved to undefined: budget headers, budget categories, allocations, receipts, SCIM nextCursor, and the delete endpoints that return a bare boolean - use the Travel Request Amount schema (currency, not currencyCode) - narrow the four XML-only travel tools to a documented string payload and request application/xml - surface real errors instead of a JSON parse failure when the proxy returns a non-JSON body - cap receipt uploads at the documented sizes before downloading Adds 106 tests covering the token cache, geolocation validation, path traversal, and error extraction. * fix(sap_concur): drop the removed forwardId subblock via a migration Removing the `forwardId` subblock without a migration entry breaks deployed workflows that still carry a value under that key. It fed a `concur-forwardid` request header that is documented nowhere in Concur's Receipts v4 or Image v1 references, so it was never honored. There is no replacement subblock and the value is an opaque caller-chosen string rather than a secret, so it is dropped outright. * fix(sap_concur): stop swallowing upload response-read failures The upload route caught every error from the bounded response read and continued down the success path, so a size-limit breach or a stream failure surfaced as an upstream success with a null or header-only body. Concur returns Content-Length: 0 on a successful image-only upload, and readResponseTextWithLimit already returns an empty string for that without throwing, so dropping the catch keeps the legitimate empty-body case working while letting real read failures reach the route's handler. * fix(sap_concur): unblock company auth and correct the body wand prompt The password grant marked username required, so the company-level flow — which sends the company UUID as the token username and has no user login — could not be configured at all, even though the request schema and token fetch already accept companyUuid without a username. Username is now optional for that grant and the server-side check reports which of the two is missing. Relabels the password and companyUuid fields to say what they carry in the company flow. The shared body wand prompt also still described several payloads the way they looked before this branch: quick expenses in PascalCase rather than v4 camelCase, travel requests and expected expenses using currencyCode where the Request v4 Amount schema uses currency, the standard SCIM SearchRequest URN instead of Concur's, startIndex as a search parameter when it is unsupported, and a cash advance shape that does not match the documented request. A wand-generated body was therefore rejected for most of the create operations it covers. * fix(sap_concur): keep Concur's status when an error body fails to read Removing the blanket catch from the upload read fixed one failure mode and introduced its inverse: a cap breach or stream error while reading a non-success body threw before the route reached the branch that preserves Concur's status, so an upstream 4xx surfaced as a Sim 500 and could trigger a retry the caller should not make. Both routes now split the two cases. On a success status the body is the result, so a read failure still propagates. On an error status the body only supplies the message, so a read failure resolves empty and the upstream status survives, with the message falling back to the generic HTTP-status form. Adds 21 tests covering both helpers over success, error, empty-body and boundary statuses; inverting the status check turns 14 of them red.	2026-08-17 16:27:23 -07:00
Waleed	60097c89b4	fix(cli): default to the host that serves the API (#6791) ... `sim.ai` answers /api/** with a 301 to `www.sim.ai`, and the client refuses to follow redirects — a 301 rewrites a POST into a bodyless GET, so following one turns a write into a silent no-op and hands the API key to whatever host Location names. Defaulting to the apex therefore failed every command for anyone who never set an endpoint. Before the refusal shipped it was quieter and worse: reads succeeded while writes did nothing. Also trims the provider catalogue from eleven inferred columns to seven. `docsUrl`, `helpText`, `requiresClientGeneratedCredentialId` and the nested `fields` are what you read once you have chosen a provider, not what you scan to choose one, and they pushed the table well past a terminal. Both ids stay: `credentials connect` names an OAuth provider by `serviceId`, `credentials create` matches a service account on `providerId`.	2026-08-17 16:25:58 -07:00
Waleed	c8f559ae77	fix(workflows,connectors): close pre-merge audit findings (#6783) ... * fix(workflows,connectors): close pre-merge audit findings Recover subblock values orphaned by the id renames in this release, and stop truncated knowledge-base listings from reporting themselves complete. - Add operation-scoped subblock id migrations so a saved workflow's stored value survives a rename. Cloudflare create/update DNS record, ServiceNow read record, and Okta deactivate/delete previously lost their stored value: the create path substituted a seeded default (an A record where the user chose CNAME, and unproxied where they chose proxied), and the update path silently no-opped while reporting success. A migration is used rather than a legacy-id fallback so no subblock id carries two value spaces at runtime. - Webflow, Zendesk: a listing that stops for a reason the connector cannot rule out now reports as capped instead of exhausted. A malformed envelope, an unfollowable continuation link, or an absent collection list previously read as a complete listing and let deletion reconciliation hard-delete every document past the truncation point. - Sentry: pin the listing window in the request rather than inheriting the server default, so the range cannot silently narrow into hard deletes. - Fork sync: a parent re-pick no longer writes a blank over a hidden optional dependent's stored target value, and a required field stays on screen once it is filled. Add hook-level coverage for the submitted payload. - Fork file copy: a file whose name is already taken in a reused target folder is de-duplicated instead of dropped. - Delete an orphaned Shopify OAuth route that built a credential from unsigned cookies. It had no writer, no caller, and no inbound link. - Tailwind: drop two content globs that scanned 5.4k files to emit one unused rule, keeping the ones that fix brand tile icon color. - Correct the API route-count baseline, add an Evernote docs redirect, align library copy with the language rules, and fix a stale turbo filter. * fix(connectors,forking): trim the audit fixes to their minimum A legitimacy review found several changes closed no live defect, and two introduced problems of their own. - Zendesk: narrow the cursor fix to a signal change. Treating a missing meta envelope as truncation had also made the walk follow links.next and keep paginating, and the ticket cursor has no page-depth valve, so a source advertising a next page with no meta could loop without terminating. The page-fetch set now matches the previous behavior; only the flag is new. - Zendesk: drop the search next_page branch. The existing count check already caps every case where a missing key could lose documents. - Webflow: drop the empty-collections flag. The sync engine already blocks the first sync on an empty listing and reconciles only when a second sync agrees, which handles a transient fault better and still removes documents when a source is genuinely emptied. The flag short-circuited that and suppressed reconciliation permanently. Restore the previous loud failure on a non-array envelope, and drop the unreachable collection-id filter. - Webflow: soften a docstring that claimed pagination.total is always present. It is documented optional, so its absence proves nothing either way and treating it as unprovable truncation is the fail-safe reading. - Sentry: drop the pinned statsPeriod. Sentry's issue search floors every query at 90 days in the executor regardless of the request, and the endpoint this release moved away from hit the same floor, so there was no window to close. Keep the tests and the docstring recording that. - Fork copy: drop the renamed counter, which no caller reads. - Repair check-block-registry, which stopped exempting migrated subblock ids when the migration map became an array — `in` was testing array indices. - Drop mdx from a Tailwind content glob that emits nothing, and loosen an exact compiled-SQL assertion to the invariant it was pinning. * fix(migrations): keep a ServiceNow write body off the read projection Review findings from the first round. - A legacy ServiceNow block can hold a Create/Update Record JSON body under `fields` while its stored operation is Read Records: the id served both value spaces before the rename, and a subblock value is not cleared when the operation changes. The scoped migration moved that body onto `readFields`, where it would reach the wire as sysparm_fields. Migration entries can now carry a `whenValue` predicate for the case where the stored operation alone cannot separate two value spaces, and the ServiceNow entry uses it to move only a plausible comma-separated projection. - Type the fork copy test harness instead of using `any`, without weakening it: every predicate shape it does not model still throws rather than matching. - Correct the dependent-omission comments. Omitting a parent-invalidated field preserves the target's stored value on Save and across an undo, where the parent nets out unchanged; on a Sync the written state is source-derived, so what it prevents there is an explicit blank reaching the fields the remap's clearing pass does not cover, nested tool params in particular. Okta's migration scope is left as-is: `okta_remove_user_from_app` and the sendEmail split shipped in the same release, so no saved block can hold legacy state for it, and widening the scope would promote an activation-era value onto the deactivation switch. Tests document the boundary. * chore(forking): move the fork-sync changes to their own PR The dependent-omission fix and the fork file-copy de-duplication are reviewed separately in #6787. They are the only changes here that overlap #6776, and they carry their own design tradeoff, so they should not ride along with the unrelated audit fixes in this PR. * fix(migrations): separate a ServiceNow write body from a projection by parsing The guard tested for a `{` or `[` prefix, so a stored scalar body — `true`, `"short_description"`, `42` — read as a field list and was promoted onto `readFields`, where it would go out as sysparm_fields. A Create/Update Record body is JSON and a projection is a bare comma-separated field list, which is never valid JSON, so parsing is the whole test rather than a guess at its opening character. Ambiguity still resolves to "not a projection", leaving the value where the Create/Update control owns it. * test(connectors,credentials): tie two assertions to what they actually prove - Webflow: a non-array collections envelope reaching `for...of` throws, which is the intended loud failure. Assert the spec-mandated TypeError plus a single request and no write-back, rather than matching V8's wording. - Credentials: the second guard test cannot observe "not deleted" — the proxy driver replays canned rows — so name it for what it does verify, that the reference check carries no workspace predicate and an empty RETURNING logs nothing. Making the driver decide the outcome would fake the database. - Drop `vi.importActual`; a plain `drizzle-orm/pg-proxy` import works now that `drizzle-orm` is un-mocked. * fix(migrations): identify a ServiceNow projection by its own shape Recognising a write body was the wrong way round. A saved body is not always well-formed: it can be a half-typed draft or carry an unquoted block reference, so neither "opens with a brace" nor "fails to parse as JSON" identifies one — and a body misread as a projection is moved to readFields with its original key dropped, losing the draft. Match the projection instead: a comma-separated list of ServiceNow field names, which are word characters plus the dot of a dotted walk. A brace, quote, colon, angle bracket or interior space fails that shape. Parsing then removes the bare scalars that satisfy it by accident.	2026-08-17 16:02:58 -07:00
Waleed	ae2147645c	fix(cli): resolve findings from a full command-surface audit (#6788) ... * fix(cli): resolve findings from a full command-surface audit Exercised all 147 commands against a live deployment. Fixes the defects that surfaced, plus the docs and generator drift they exposed. Transport - Stop following redirects. A bare domain that 301s to www silently converted POST to GET and dropped the body, so reads worked while every write failed with a misleading validation error and login returned 405. Both the client and the device flow now explain the redirect and name the endpoint to configure, rather than carrying credentials off-origin. - Report a non-JSON response as one instead of printing the HTML page. - Name the personal-API-key remedy on a workspace-key refusal, reading the machine-readable code the API actually sends. - Drop union-branch noise from validation errors that contradicted itself. - Show paging progress on stderr for multi-page fetches. Output - Clamp record values for table only. text is the format built for pipes, and it was truncating signed URLs and tool source mid-value. - Infer timestamp, duration, bytes and boolean formatting for API-owned keys so undeclared commands stop printing raw ISO and float ms. Skips user-defined table cells and leaves json/yaml on the raw payload. - Render a declared-but-absent field as an em dash; billing credits were vanishing silently. Paths, naming and validation - Percent-encode folder paths per segment and decode them for display, so a folder reads and types as the name shown in the app. - Reject a malformed endpoint where it is set and where it resolves, instead of crashing with a URL parse trace. - Request the detail level logs list's own columns need; its workflow column could never populate. - Rename three commands that described themselves wrongly and align two flags with their siblings. Old spellings still work: hidden, warned on stderr, and kept out of help and docs. - Verify whoami against the API, separating a bad key from an unreachable endpoint, and report the workspace by name. - Correct the --yes help text, which advertised skipping a prompt that does not exist. Docs - Teach the docs generator that a flag required by the runtime is required, and that hidden commands are not documented. * fix(cli): clear the paging progress line when a page fails Progress is written without a trailing newline so it can be overwritten in place, and both paging loops cleaned it up only on success. A page that threw part-way through left `fetched 1200…` on the line the error was then printed onto, so the two ran together. * fix(cli): name a working API root when an endpoint redirects The suggested endpoint was the redirect target's origin, which drops a path prefix. A self-hosted deployment reached at https://host/sim was told to set https://www.host — not an API root, so following the advice replaced one broken endpoint with another. Derive it by stripping the request's own path from the target instead, so a prefix survives, and say nothing about --set-endpoint when the target resolves to the endpoint already configured: a trailing-slash or path normalization redirect keeps the origin, and naming the value the caller already has explains nothing. The login poll shared both faults and now shares the helper.	2026-08-17 15:55:04 -07:00
Waleed	746a4496ba	chore(deps): upgrade next to 16.3.1, its optimizer no longer deletes live code (#6777) ... * chore(deps): upgrade next to 16.3.1, its optimizer no longer deletes live code 16.3.0 was reverted in #6242 because its Turbopack optimizer modelled a bare `return <asyncCall>()` tail call inside an async function as returning the promise object, propagated that always-truthy fact through the caller's `await`, and deleted everything after the resulting `if`. That shipped two dead code paths to production: the whole `POST /api/credentials` create path, and the insert inside `upsertAsyncToolCall`. We reported it as vercel/next.js#96595. The fix — "[turbopack] Collapse nested promises in the analyzer" (vercel/next.js#96601) — folds `Promise<Promise<T>>` to `Promise<T>` in the analyzer, and was backported as #96675 and released in 16.3.1. Verified before taking the bump: - The minimal reproduction from the issue no longer reproduces on 16.3.1. All four routes keep their code; on 16.3.0 `/api/broken` lost everything after the `if`. - A production build of `apps/sim` on 16.3.1 still emits the markers whose disappearance was the original signal: `credential_connected` (43 files), `acquireOrganizationUserMutationLocks` (28), and the `upsertAsyncToolCall` insert-path warning (10). The `return await` hardening added to both sites in the revert stays as is, and so does the TypeScript toolchain configuration. 16.3.1 published 2026-08-13, so it is inside the 7-day `minimumReleaseAge` supply-chain window until 2026-08-20 and needs an exclusion to install. The alternative is sitting on 16.2.12, whose successor we already reverted once, so the entries go in dated and come out on the next touch of the file. The mermaid and js-yaml exclusions aged out on 2026-08-11 and 2026-08-07 and are dropped here per that same rule. * fix(deps): keep the musl and win32 SWC binaries in the lockfile The release-age exclusion only listed the four @next/swc platforms that package.json pins, but next declares all eight as its own optionalDependencies, so all eight are normally resolved into bun.lock. A gated optional dependency does not fail the install — bun drops it silently — so the first install stripped both musl variants and both win32 variants from the lockfile. That left the Alpine devcontainer and any Windows machine with no SWC binary to resolve. Adding the remaining four to the exclusion list restores all eight entries at 16.3.1. Worth knowing for the next time this happens: bun.lock is sticky here. Once an optional dependency has been dropped, re-running the install — even with --force, even with the age gate switched off entirely — does not bring it back, because the resolution is not reattempted. The lockfile has to be regenerated from a base that still contains the entries, which is why this restores bun.lock from staging before re-applying the bump.	2026-08-17 14:04:01 -07:00
Waleed	75718ab39f	fix(execution): stop a cancelled run reporting success when its wait swallows the cancellation (#6775) ... * fix(execution): stop a cancelled run reporting success when its wait swallows the cancellation Cancellation reaches a running execution over Redis pub/sub, which is at-most-once. The engine turns that into `status: 'cancelled'` via `signalCancelled`. But the wait handler also polled the durable Redis cancellation key itself, and on a hit it broke out of its sleep and returned an ordinary successful block output. The engine's `cancelledFlag` stayed false, so a cancelled run finished as `success: true` — and with a block after the wait, kept executing. Whichever detector fired first won. The engine's pub/sub path normally wins by about one round trip; when the wait's own 500ms poll landed inside that window the cancellation was lost. Consolidate detection in the engine, which is the only component that can project run status: extend the once-at-start durable backstop into a poll that runs for the life of the run and routes through `signalCancelled`. The wait handler and loop orchestrator now observe only `ctx.abortSignal`, which the engine aborts, so no leaf can observe a cancellation the engine has not seen. The loop orchestrator additionally used to ignore `abortSignal.aborted` whenever Redis was enabled, so a mid-loop timeout or client disconnect was invisible to it, and it awaited a Redis round trip on every iteration. Handlers that abort their own I/O off `ctx.abortSignal` are unaffected: that surfaces as a throw, which the cancelled branch of `run` already classifies. * docs(wait): correct the in-line wait ceiling to 5 minutes The Wait page claimed a 10-minute cap for a synchronous wait in three places. `MAX_INPROCESS_WAIT_MS`, the block description, the sub-block hint, and the validation error all say 5 minutes.	2026-08-17 01:55:48 -07:00
Waleed	b38e4e2f91	docs(integrations): add missing manual intros; fix light brand tiles rendering white glyphs (#6774) ... * docs(integrations): add manual intro sections to eight integration pages * fix(styling): scan blocks and ee in the tailwind content globs * docs(snowflake): correct the unload-data capability to a table source	2026-08-17 00:49:54 -07:00
Waleed	31521d6abf	fix(connectors): validate and repair the knowledge-base connector fleet (#6757) ... * fix(connectors): validate and repair all 61 knowledge-base connectors Audits every KB connector against its provider's live API documentation and fixes what the audit found. The dominant defect class is deletion reconciliation: the sync engine hard-deletes any stored document absent from a "full" listing, and most connectors had a path where a truncated or errored listing failed to set `syncContext.listingCapped`. Highest-impact fixes: - linear: `getDocument` was dead code. The query declared `$id: ID!` where the schema is `issue(id: String!)`, so every call failed variable validation. - salesforce: `v62.0` was substituted into a `{version}` template that already contains the `v`, so every REST call 404'd. SOQL `LIMIT` was also used as a page size, silently capping every sync at 200 records. - notion: only the first level of blocks was fetched, so tables indexed empty. - microsoft-teams: `/messages` returns messages without replies, so no threaded content was ever indexed. - gmail: an empty page discarded `nextPageToken`, which reads as a complete empty listing and hard-deletes every stored thread. - confluence: the CQL path paginated with `start` and `totalSize`, neither of which exists on that endpoint, so label-filtered syncs stopped after one page. - box: `supportsRefreshTokenRotation` was unset, so Box's rotated refresh token was discarded and every credential died on its second refresh. Removes the Evernote integration entirely: the classic EDAM API is deprecated, its sandbox is decommissioned, and developer tokens are no longer obtainable. Makes SFTP host-key verification mandatory, and adds an attendee-PII opt-out to google-calendar and google-meet (default on, so existing sources are unchanged). Bumps the contentHash namespace for notion, google-docs and hubspot so existing documents re-hydrate once and actually receive the content fixes above. * fix(connectors): close swallow-into-empty and cursor-taper regressions Ship-gate pass over the connector audit. Every finding was re-verified against the provider's live documentation or machine-readable spec before being acted on; several pass-3 edits were reverted rather than extended. Correctness fixes: - fireflies: a 2xx with an unparseable body returned an empty listing instead of throwing. fireflies runs a full sync every time, so a fault persisting across two syncs would have tombstoned all indexed docs. - linear: same shape via `data.issues || {}` on a non-nullable connection. - greenhouse: a 403 from a key without scorecard permission was treated as transient, appending `:partial` to the hash. That never matches the list stub, forcing full re-hydration of every candidate on every sync forever. - google-meet: `fetchParticipants` carried a 404 swallow copied from its transcript siblings, freezing every speaker as "Unknown". - airtable, asana, ashby: reverted page-size tapers applied over opaque cursor tokens. The cap was already enforced server-side. - google-docs: response byte cap resolved to 800MB and could never fire. - google-forms, google-vault, notion, sharepoint, dropbox: `getDocument` now throws on transient failure instead of returning null, which the engine reads as absence. Security: - Retry headers are attached non-enumerably. TypeScript `private` is compile-time only, so `SecureFetchHeaders.setCookies` was an own enumerable property that the logger serialized into sync logs. Docs and dead code: - Corrected six fabricated doc citations (github, jira, jsm, linear, google-meet, dropbox) and removed the Evernote integration entirely. * refactor(jira): type the ADF node helpers with unknown instead of any * fix(connectors): throw on misconfigured typeform/zendesk sources instead of returning null A null from getDocument reads as documented absence, so on an add the document is dropped with neither a failure counter nor a log. Both listDocuments paths already throw on the same missing config. * fix(connectors): keep confluence CQL page size constant; unify monday API version The CQL search endpoint paginates by opaque cursor, and Atlassian does not document that a cursor issued against one limit survives a request asking for a different one. Narrowing limit to the remaining budget was the same pattern reverted on airtable, asana, and ashby. The page size is now constant and the cap is applied by trimming the returned page, which keeps the cap exact without varying the request. Monday OAuth getUserInfo hardcoded API-Version 2024-10 while every other monday surface reads MONDAY_API_VERSION, defeating the single-source pin. * fix(connectors): act on the final validation sweep Findings from a read-only /validate-connector pass over all 59 changed connectors, verified against provider specs before acting. Silent-drop fixes (a fulfilled null from getDocument records no failure and no log, so the document vanishes): - ashby: candidate.info returning success with an unusable payload. Ashby sets contentDeferred, so this path is live. - azure-devops: an unresolvable branch, likewise live. - dropbox: 409 covers the whole LookupError union, and restricted_content and locked both mean the file still exists. Only not_found is absence. - docusign: fetchFormValues swallowed every non-OK status, baking a permanently incomplete document since the hash is metadata-only. typeform: 'all' sent response_type=started,partial,completed, but Typeform documents only partial and completed. An unknown enum member risks a 400 that fails the whole sync, and staging omitted the parameter entirely, so this shipped as a regression. Now requests the widest documented set. github: removes a utf-8 blob branch justified by a misattributed quote — that sentence describes the encoding REQUEST parameter of Create a blob; the GET response is documented as always base64. Also corrects two comments that hid a real drop: >1 MB files under vnd.github+json 403 rather than returning encoding: none. hubspot: routes HTML detection through a shared anchored helper. The loose pattern matched angle-bracketed prose such as an email address, and htmlToPlainText deletes the span and collapses line structure. This matters now because the hubspot:v2: bump rewrites every live document once. youtube: drops an invented channel-ID format quote. * fix(connectors): converge incidentio partial hash on settled statuses A 403 from a key without incident_updates permission, or a 404, returns on every sync. Marking those incomplete appended :partial to a hash that then never matched the listing stub, so the incident re-hydrated forever without converging. Only a transient failure may mark content incomplete now, which matches how greenhouse already treats the same class. * fix(connectors): flag WIQL truncation unconditionally; stop swallowing docusign form-data failures azure-devops: the 20,000-item WIQL ceiling was probed by asking for a matching item with an id beyond the largest returned. That probe is unsound — buildWiql orders by ChangedDate DESC while ids are assigned in creation order, so the highest-id match is almost always inside the returned window. The probe came back empty for genuinely truncated projects, left the listing unflagged, and let deletion reconciliation remove every indexed item outside it. Flag unconditionally instead; the cost is a project sitting exactly at the ceiling not reconciling deletions until a full resync. docusign: fetchFormValues threw on a non-404 status and then caught its own throw, returning []. The earlier fix was a no-op. The catch now rethrows, so a transient failure produces a failed row instead of a permanently incomplete document under a metadata-only hash. * fix(connectors): restore airtable AI Text indexing; floor sentry maxIssues airtable: staging rendered object cells with a JSON.stringify fallback, so AI Text values and nested lookup arrays reached the index. This branch replaced that with a fixed key-probe list to stop attachment-URL hash churn, but the probe list has no fallback — aiText ({state,isStale,value}) and nested arrays rendered to the empty string and vanished from every document, and the content-derived hash never moved when the text regenerated. Read `value` last, after the existing probes, and recurse on nested arrays. The generated text is stable rather than an expiring signed URL, so this does not reintroduce churn. sentry: maxIssues now feeds the request limit, and Sentry rejects a non-integer. validateConfig accepts a fractional entry, so a config that saved cleanly would fail every sync at listing time. Staging was immune only because it sent a hardcoded page size.	2026-08-17 00:21:04 -07:00
Waleed	cc1a278d73	fix(integrations): close defects found by an independent cold audit (#6767) ... * fix(integrations): repair Update SLO and advanced OData filters An independent audit — eight cold readers, one per integration, given no prior findings — checked the eight integrations merged to staging today. Two defects broke an operation outright; both are fixed here. datadog: Update SLO rewrote every non-metric SLO to `metric`. The SLO Type dropdown carried a `metric` default and its condition covered both create and update, so an untouched control reached mergeSloUpdatePayload as an edit. A metric SLO requires `query`, and the merged body carries monitor_ids or sli_specification instead, so Datadog rejected it — Update SLO was unusable on monitor-based and time-slice SLOs, with no way to express "keep the current type". Update now has its own control defaulting to "Keep current". microsoft_ad: list_users, list_groups and list_service_principals emitted $count=true only alongside $search, so any $filter using an advanced operator (ne, not, endsWith, startsWith on non-indexed properties) returned 400. Graph requires $count=true with ConsistencyLevel: eventual for those. list_devices already did this correctly; the other three now match it. Also from the same audit: Datadog path IDs are trimmed before encoding in all 20 URL builders rather than 2, matching the existing get_monitor test. * fix(integrations): cloudflare, crowdstrike, and mssql audit findings From the independent cold audit. Cloudflare: DNS analytics no longer emits fabricated min/max telemetry (Cloudflare documents both as always empty); purge_everything defaults to specific-purge and errors when combined with target lists; three unsourced description claims corrected; Array.isArray guards on four older list transforms. CrowdStrike: IOC sort placeholder corrected to the dot form (created_on.desc, not the nonexistent created_timestamp); the 500-indicator cap relabelled as a Sim bound rather than a CrowdStrike one; credential failures return 401 rather than 500; prevent_no_ui noted as unenumerated. MSSQL: introspect no longer lets the model choose the database; row and byte caps on reads; introspection collapsed from 4N+2 to 6 fixed queries; WHERE and identifier guards run before the connection opens so rejections are 400 not 500; SAVE TRANSACTION, OPEN/CLOSE key, DEALLOCATE and ADD SIGNATURE added to the statement screen as two-token phrases; encrypt wording corrected to say TDS 7.4 encryption is negotiated, not guaranteed. * fix(splunk): publish the real output tables and read the errors Splunk sends The docs generator parses tool source text and resolves a shared `outputs` const only from the family's `types.ts`, so Splunk's helpers in `utils.ts` were invisible to it: seven operations published the block's union of every output instead of their own. Run Search and Get Search Results each shipped a ~50-row table naming savedSearches, alerts, indexes, and apps they never return, Cancel Search Job lost `messages`, and the four list tools lost `total`/`offset`. Inline the four helpers into each consuming tool and delete them, since relocating a shared const only moves the trap. Also: - Add a `splunk-errors` extractor for the documented `{messages: [{type, text}]}` envelope and set it on all twelve tools. A rejected SPL string, the most common failure, previously fell through to the status text and reported "Bad Request". - Read `searchEarliestTime`/`searchLatestTime` with `asNumber`. The job entry documents them as bare epoch numbers, so `asString` returned null for every `output_mode=json` response. - Project the `<messages>` block of the XML job-control response. It is the only payload that endpoint returns, so `cancel_search_job.messages` was always empty. - Mark the nullable job outputs optional, matching the transform. - Default `run_search` to `max_count=1000`. A oneshot search has no paging escape hatch and Splunk's own default is 10000 rows in one buffered response. - Add suggested skills to `SplunkBlockMeta`, grounded in `tools.access`. The regenerated tool metadata also picks up the Cloudflare and MSSQL output changes from the previous commit, which were never synced. * fix(okta,servicenow): apply integration audit findings Cherry-picked from fix/okta-servicenow-audit-followups (6db0f54), whose base predated the earlier audit round; the duplicate isOktaFlagEnabled that produced is resolved in favour of the existing richer helper, which already accepts 'yes'/1/'on' as well as true/'true'. okta: get_logs no longer advertises hasMore forever. A System Log query with no 'until' is a polling query, and Okta always returns a next link for one, even on an empty page — so any loop driven by hasMore never terminated, including the one our own shipped skill instructs the agent to run. errorCauses is now surfaced, so a failed write reports the real reason instead of the useless 'Api validation failed: profile'. sendEmail routes through one coercion helper across all four lifecycle tools. update_group's declarative fallback throws rather than silently truncating an extensible group profile. servicenow: attachmentLimit and limit no longer overwrite each other. Neither assignment was scoped to an operation, so all 12 paginated operations could silently return a row count the user never asked for — defeating the block's own design, which gave attachmentLimit a unique id precisely to avoid this. All seven approval states are published by ServiceNow and are now reachable from the filter, with the space-vs-underscore punctuation documented. The five legacy generic tools route through the shared response helpers, and the folder's only 'any' is gone. Block skills now name the semantic operations. * chore(integrations): regenerate catalog and docs artifacts * fix(integrations): disclose MSSQL truncation and keep Okta's poll cursor Three defects the review round found in the audit fixes themselves. MSSQL capped a recordset and then reported it as complete: `executeQuery` computed `truncated`/`truncationReason` but all five statement routes returned only `message`, `rows`, and `rowCount`, so a caller could not tell paging was required. A shared `toRowsResponseBody` now folds the reason into `message` for an agent reading the status line and exposes the two fields for a caller that branches on them. The byte ceiling also admitted a single oversized row as a lone exception, so one `nvarchar(max)` value serialized an unbounded body — the ceiling bounded everything except the case it exists for. A row is now admitted only when it still fits, and the drop is disclosed rather than read as an empty table. Okta's `get_logs` nulled `nextCursor` alongside `hasMore` on an empty polling page. Terminating the loop is right, but the cursor is the resume handle Okta tells callers to persist, so a scheduled workflow that hit one quiet interval restarted from `since` and re-delivered events it had already processed. The two answer different questions and now diverge. Cloudflare's purge block no longer lets the invalid combination be built: the four target fields are hidden once Purge Everything is selected, so the tool's guard is a backstop rather than a reachable hard error. * fix(okta,servicenow): stop sending requests the APIs reject Okta documents `since` and `after` on the System Log as mutually exclusive, so `get_logs` lets the cursor win rather than sending both — the shape a scheduled poll that persists the cursor would otherwise send. Seven boolean query params reached Okta interpolated raw, so an agent tool call supplying `yes` was rejected. Each now routes through `isOktaFlagEnabled`, keeping its existing send-or-omit behavior. A cleared ServiceNow limit/offset/quantity stayed `''` through the block mapper and was appended as a valueless `sysparm_limit=`. The mapper now resolves a blank to undefined, and the tools skip a blank as well. * fix(integrations): mssql guard gaps and Entra query, scope, and output findings MSSQL read-only screen - Screen RENAME, documented T-SQL DDL for Azure Synapse dedicated SQL pools and Analytics Platform System, which are reachable over TDS with exactly the connection fields this block exposes. `SELECT 1 RENAME OBJECT dbo.t TO t2` was a schema change passing an operation advertised as read-only. - Screen the Service Broker family: RECEIVE as a word, and END/MOVE/GET CONVERSATION and SEND ON CONVERSATION as two-token phrases, since END closes every CASE. RECEIVE is a destructive read and END CONVERSATION WITH CLEANUP drops a conversation's messages. MSSQL routes and block - Build the insert statement before connecting, matching update and delete, so a bad identifier answers 400 instead of burning a TLS+login and returning 500. - Declare `truncated`/`truncationReason` on the block, which the tools declare and the routes emit but the block left unreferenceable. Microsoft Entra ID - Pair `$count=true` with `ConsistencyLevel: eventual` conditionally. Graph documents `hasMembersWithLicenseErrors`, `isLicenseReconciliationNeeded`, and `identities/any(i:i/issuer)` as filterable only *without* advanced query parameters, and documents advanced queries as unsupported in Azure AD B2C tenants, so the unconditional pair broke filters that previously worked. When continuing from a nextLink the pairing is read off the link itself. - Request `LicenseAssignment.Read.All` instead of `Directory.Read.All`. The latter was needed by `GET /subscribedSkus` alone, whose permission table names the former as least privileged and does not list the ReadWrite scope we hold. - Enumerate the block's real output keys instead of a single `response` object no tool emits. * fix(splunk,datadog): stop truncating searches and send mute/unmute as query params Splunk run_search: revert the `max_count=1000` default added last pass. It was wrong on both halves. Splunk documents the parameter as "the number of events that can be accessible in any given status bucket. Also, in transforming mode, the maximum number of results to store" — so for a non-transforming oneshot it bounds status buckets, not the response, and for a transforming search (`| stats`, `| timechart`, which is what the block's own skills generate) it capped results at 1000 where Splunk would have stored 10000, silently. The block's `maxCount` placeholder already read `10000`, contradicting the code. Send `max_count` only when the caller sets it and restate the description in Splunk's own terms, matching create_search_job. The real guidance — a oneshot buffers the whole result set, so use Create Search Job + Get Search Results for anything large — moves into the tool description and the search-splunk-logs skill. Datadog mute/unmute: send `scope`, `end`, and `all_scopes` as query parameters. `MuteMonitor` and `UnmuteMonitor` declare no `requestBody` in the authoritative spec (docs.datadoghq.com/resources/json/full_spec_v1.json — the generated datadog-api-client-go v1 schema omits both operations and is a subset, not the authority); all three parameters are `in: query`. Sent as a JSON body they are dropped, so a scoped, time-boxed mute becomes an indefinite mute across every scope and unmute's "all scopes" never applies — answered with a 200 and the full monitor object, so nothing surfaces. Datadog list_monitors: imply `page=0` when a page size is set without a page. Datadog "returns all monitors without a `page_size` limit" when `page` is absent, so Page Size was inert from a control that reads as a bound. `page` is not defaulted when neither is set — that would silently truncate a caller relying on the documented return-everything behavior. Also: - Note in get_fired_alerts that `name=-` returns every saved search's fired alerts and the endpoint documents "Request parameters: None", so there is no count/offset to bound it. - Fix the Splunk block's `messages` output blurb: `[{type, text}]` holds for the search and job-control operations, but get_search_job returns an object. - Generalize the Datadog block's numeric coercion (`datadogPageNumber` → `datadogNumber`) over all 32 bare `Number()` mappings, so a typo or unresolved reference is omitted rather than sent as `NaN`/`null`, and an explicit `0` survives the old truthiness guard. - Disclose create_event's documented 18-hour `date_happened` ceiling, and that send_logs' `ddsource: "custom"` is a Sim default rather than a Datadog one. * chore(integrations): regenerate tool metadata and docs * fix(integrations): resolve confirmed findings from cold block audit Cloudflare: clear purge_cache advanced targets across operations; send action_parameters/ref/logging on rate-limit rule updates; migrate off the deprecated batch zone-settings endpoint; correct MX/URI priority wording; stop coercing blank numerics to 0. CrowdStrike: seed includeHidden to match Falcon's documented default. Microsoft Entra ID: resolve a UPN to an object ID for app role assignment; wrap 21 array outputs in items.properties so nested paths resolve. Okta: route assign_user_role's notification flag through isOktaFlagEnabled. ServiceNow: drop the triage skill's claim of a default limit that does not exist. Splunk: always assign coerced numerics so raw values cannot leak through the executor's raw-input merge. * fix(editor,credential-group): mask secrets outside short-input and stop a per-option abort from failing a shared query config.password only reached the short-input renderer, so eight credential fields rendered in plaintext: private keys on ssh/sftp/pi/kalshi, the Secrets Manager payload, the STS web-identity and SAML assertions, and the Browser Use variables table. long-input, code, and table now honor the flag. Code fields mask through the highlighter because react-simple-code-editor paints its textarea transparent; the table masks every column but the first so key/value rows stay distinguishable. A registry-walking audit test fails both on a password flag sitting on a type that cannot honor it and on any of the eight fields losing its flag. credential-group threaded a per-option AbortSignal into the fetch registered under the workspace-wide credential group list key, so closing one option panel rejected every co-observer with an AbortError that is not a React Query cancellation. The shared fetch now runs on its own lifecycle signal. * fix(mssql,editor): measure the response cap in UTF-8 and stop search from unmasking secrets capRecordset sized rows with JSON.stringify(row).length, which counts UTF-16 code units while the emitted body carries raw UTF-8. CJK is the worst case at 3 bytes per unit, so a recordset admitted as 10 MB serialized to 28 MB. Rows are now measured with Buffer.byteLength, serialized once each, with array punctuation charged exactly and a reserve held back for the response envelope. Workflow search revealed masked credentials without the user touching the field: the search panel keeps focus in its own input and only scrolls the match into view, so typing a guess painted a private key on screen. The index is built client-side from values already in page memory, so this was never a privilege boundary, but masking exists to prevent incidental display and a screenshare-visible reveal defeats it. Focus is now the only reveal, applied through one shared policy across all four renderers. * test(editor): drop PEM-shaped fixtures from the masking tests The masking fixtures carried a literal OPENSSH private key header, which GitGuardian flags as a committed secret even though the body was only the base64 of "openssh-key-v1". The fixtures now use an obvious marker string, and the assertions derive their match text and dot counts from the fixture instead of restating its bytes. * refactor(editor): drop the dead isSearchHighlighted prop No renderer consumed it. The editor computed it at two call sites and sub-block passed a hardcoded false into renderLabel's slot for it, so even the one function that declared a parameter never saw the real value. Its only live effect was in the memo comparator, where an unconsumed value changing forced a re-render for nothing. The name stays in the masking audit's forbidden-inputs list, which guards against a search signal being wired back into a masking decision.	2026-08-16 23:25:16 -07:00
Waleed	0844d4166b	feat(jotform): add Jotform integration (#6772) ... * feat(jotform): add Jotform integration Adds 43 tools covering forms, questions, submissions, reports, webhooks, labels, and account operations, plus the block, icon, and generated docs. Request shapes are pinned against the API's own curl samples and the official SDKs: PUT /form/{id}/properties and PUT /form/{id}/questions each take a named envelope while PUT /form and the bulk-submission PUT take their payload bare, and submission answers accept both the nested object and the documented {qid}_{subfield} shorthand. Skips the deprecated folder endpoints in favor of labels, and leaves out endpoints whose response shape the docs do not publish. * fix(jotform): harden the error envelope against quoted codes and non-JSON bodies Jotform quotes `responseCode` on some endpoints and not others, so a typeof-number test skipped the check on the quoted ones and turned an auth failure into a successful tool result with empty output. Also caps the raw body fallback, since an upstream gateway can answer with an HTML page instead of the documented envelope. * fix(jotform): stop duplicate question labels overwriting derived answers Question labels are not unique — a form can carry two questions both labelled "Email" — so keying the derived `values` map on the label alone dropped all but the last and handed downstream workflows a confidently wrong answer. Every occurrence of a repeated label is now suffixed with its question ID, rather than only the later ones, so the result does not depend on answer order and a newly duplicated label reads as absent instead of as an arbitrary winner. The id-keyed `answers` record was already complete and is unchanged. * fix(jotform): make the label-keyed answer map collision-proof Question labels are free text, so the disambiguation key added in f9026cf was not itself safe: a question literally labelled "Email (3)" lands on the key generated for a duplicate "Email" at qid 3, dropping one of them. Any key already taken is now widened again until it is free. Accumulates in a Map rather than an object literal on the way out, since a question labelled `__proto__` assigned onto `{}` sets the prototype instead of an own property and disappears from the map entirely.	2026-08-16 23:13:28 -07:00
Waleed	aeb5624cc8	fix(integrations): close regressions found in the final validation sweep (#6764) ... * fix(integrations): close regressions found in the final validation sweep An independent read-only audit of the eight integrations merged to staging today found defects in every one, most of them side effects of the surgery those PRs performed on already-shipped code. Data loss and destructive paths: - cloudflare: restore the shipped subBlock ids on read filters so existing workflows keep their DNS/zone/purge filters. Losing them made list_dns_records return the entire zone with success: true, which a downstream delete fan-out would then target. The colliding write controls are renamed instead, chosen by blast radius. - cloudflare: refuse an update_ruleset_rule that would tear down the rule it edits. PATCH is a replace, so an omitted action_parameters unbound the WAF managed ruleset and every override under it. - cloudflare: split the hidden `enabled` control so a value set while drafting can no longer disable a live WAF or rate-limiting rule. - cloudflare: stop `name` leaking into update_dns_record and renaming a live record. - okta: stop a blank name overwriting a stored group name via the LLM path. The block guard covered only the UI. Broken on the default path: - microsoft_ad: update_user sent accountEnabled: "" on its own default, so every call left at "No Change" failed. Same tri-state defect already fixed for forceChangePasswordNextSignInWithMfa; `visibility` fixed alongside it. - cloudflare: `domain` is required for self_hosted (the default app type), ssh, vnc and rdp; add saas_app/target_criteria and drop dash_sso, which has no request variant. Silent wrong results: - datadog: list_monitors inherited Create Monitor's tag filter and returned a filtered list as if complete. - servicenow: `fields` carried both a JSON body and a projection on the three legacy generic operations. The regression test for this fed already-JSON and could not fail; it now feeds a real projection. - splunk: cancel_search_job reported failure on success by parsing an XML body as JSON; readSplunkJson now tolerates it. - okta: sendEmail === true dropped a string 'true', silently skipping the deactivation email. Security: - mssql: add writetext/updatetext/readtext to the statement screen. \bupdate\b cannot match UPDATETEXT, so both were reachable through the read-only path. - crowdstrike: chunk repeated-query ids. At the published caps a single request built a ~68 KB query string, past typical proxy limits. Also: splunk count=0 unbounded read, splunk pagination totals, the `nobody` placeholder that reintroduced the namespace bug by copy-paste, okta cursor and activate controls split per operation, servicenow sysparm_having syntax and two required controls no longer pre-seeded with consequential values, datadog block outputs reconciled with tool outputs, and 16 escaped apostrophes that corrupted the published Entra docs. One scope removed from microsoft_ad (User.Read.All). Directory.Read.All and GroupMember.ReadWrite.All were proposed for removal and verified still required; a test now asserts they stay. * fix(integrations): surface corrupt Splunk bodies and partial CrowdStrike deletes Narrows readSplunkJson's non-JSON tolerance to XML. The dispatching and job-control endpoints answer in XML, but a body that is neither empty nor XML was meant to be JSON, so swallowing its parse failure handed get_search_results an empty envelope and reported a lost result set as a search with zero events. Annotates a batched CrowdStrike delete that fails partway with the IDs its earlier batches already removed. Falcon cannot roll those back, so a bare failure left the caller unable to tell what was gone and a blind retry re-targeted IDs that no longer existed. Registers the Cloudflare subblock-ID migration the registry-stability check requires. The suffixed read-filter IDs never shipped in a release and every block already materializes the restored IDs, so they are dropped rather than renamed onto values the collision guard would discard anyway. * fix(integrations): close the three defects Bugbot found in the sweep An execute rule sent an explicit empty action_parameters object past the new guard, because presence was checked rather than emptiness. `{}` is the same payload Cloudflare's schema default produces, so it unbound the managed ruleset the guard exists to protect. Datadog's new List Monitors pagination used a bare `Number()`, so a typo or an unresolved reference in either advanced field reached Datadog as a literal NaN — the pattern this same sweep fixed for Entra `top` and the Splunk numerics. Okta's block still marked the group name required on update, blocking a description-only update that the tool, its merge helper, and the API all accept. * fix(integrations): confine the Splunk XML tolerance and the CrowdStrike commit list Splitting the XML tolerance out of readSplunkJson into readSplunkDispatchJson puts it only on the three dispatching and job-control tools that need it. The results path can no longer read any non-JSON body as an empty envelope, so a 2xx HTML interstitial surfaces instead of reporting a search that matched nothing. The dispatch reader anchors on the one documented `<response>` root, so an interstitial fails there too. A batched delete now records the IDs Falcon echoed in `resources` rather than the IDs that were requested. A batch can answer 200 while reporting per-ID failures, and naming those as deleted told the caller to drop still-live indicators from the retry. * test(crowdstrike): pin batched partial-delete parity with an unbatched request A 2xx envelope carrying per-ID errors is a partial success, not a failure — failedWithoutResources fails the operation only when nothing came back at all. The batched path already reports it exactly as a single request does, with deletedIds naming what Falcon confirmed and errors naming what it refused. Pin that so the contract is not mistaken for a swallowed failure. * fix(splunk): read the dispatch XML envelope instead of discarding it A dispatch answering in the documented XML form was replaced with an empty object, so create_search_job and dispatch_saved_search threw a missing-sid error after the remote job had already been created — stranding a job the caller could no longer poll or cancel. The envelope is now projected onto the same `{ sid }` shape output_mode=json produces, so the search ID survives. Matching only the opening tag also accepted a body cut off mid-transfer, which on a cancellation reported a truncated response as a successful cancel. The pattern now spans the closing tag, so a truncated envelope falls through to JSON.parse and throws.	2026-08-15 21:25:23 -07:00
Waleed	025ea4d2bd	fix(docs): serve JSON-LD in the HTML, fix sidebar spacing, and tighten the CLI guides (#6763) ... * docs(cli): use -g for the install, and cut the prose that was not pulling weight `--global` is valid but `-g` is what every comparable CLI documents, and the long form only came from the package README. Also drops the yarn tab: it read `yarn global add sim`, which works on Yarn 1 only — Yarn 2 removed global installs, so that command fails for anyone on a modern Yarn. Adds `npx sim` for running without installing. The guides had accumulated design rationale that belongs in code comments rather than user docs — why the filter grammar is JSON, why the config section naming is asymmetric, why an unexpected error keeps its stack trace. Surveying how gh, Vercel, Turborepo, Deno, Bun and Supabase write theirs, none carry that kind of justification, and callouts are reserved for content whose absence produces a wrong result rather than for general asides. So: 1016 lines to 763, and 12 callouts to 3. The three that remain are the pairing-code check, that `sim logout` does not revoke the key, and the `--limit 100` default on `batch-delete`/`batch-update`, which silently truncates a larger match. Troubleshooting drops the entries whose error message already contained its own fix and keeps the seven whose cause is not obvious. * fix(docs): render JSON-LD as native script tags so it reaches the HTML All four structured-data blocks — WebSite, TechArticle, BreadcrumbList, SoftwareApplication — were rendered with `next/script`, which never emitted a script tag. Measured on a production build, `/api-reference/getting-started` contained zero `<script type="application/ld+json">` elements; the payload existed only in the `__next_s` client-injection queue and the RSC flight data, so anything reading the served HTML saw no structured data at all. React was also logging "Encountered a script tag while rendering React component" on every page. `next/script` is for loading and executing JavaScript. JSON-LD is data, and Next's own guidance is a native `<script>` in the component. `serializeJsonLd` already escapes the `<` character to its unicode form, which is the sanitization that guidance calls for, so only the element changes. Same build, after: three valid tags per page with `WebSite` in `<head>`, and the injection queue gone entirely. * fix(docs): scope the flush-separator rule to a container's first separator `[data-separator]:not([data-separator] ~ [data-separator])` was meant to keep the first sidebar group flush against the top padding, but `~` only reaches siblings, so it also matched the first separator inside every expanded folder. Under Self-Hosting, "Install" lost its top margin and crowded the "Architecture" link above it — 25px of gap where "Configure" and "Operate" below it had 40px. `:first-child` expresses the intent directly. Only the four sidebar roots open with a separator; every nested folder starts with a page, so the intended case still goes flush and nothing else changes. * fix(docs): move the flush-separator rule onto the separator component Keeps the styling with the component that owns it, per the repo standard, and lets the global rule be deleted outright rather than corrected — `global.css` now only loses a rule in this PR. Tailwind's `first:` variant compiles to the same `:first-child` selector, so behavior is unchanged: the build emits `.first\:mt-0:first-child{margin-top:0}` and the prerendered HTML carries the class on the separator.	2026-08-15 20:39:34 -07:00
Waleed	257029a60c	feat(microsoft_ad): licensing, security, audit, role, and device operations (#6742) ... * feat(microsoft_ad): licensing, security, audit, role, and device operations Deepens the Microsoft Entra ID block from 12 to 36 tools against the Microsoft Graph v1.0 reference: license assignment and tenant SKUs, password set/reset, sign-in session revocation, authentication methods, sign-in and directory audit logs, app role and directory role assignments, service principals, device reads, and conditional access policy reads. Device write (device-update, device-delete) is deliberately excluded. Both document Directory.AccessAsUser.All as their only delegated scope, with the higher-privileged read documented as unavailable, so supporting them would mean requesting tenant-wide act-as-the-user directory access for two operations that additionally require the caller to hold Intune Administrator. Also drops an undocumented ?$select= from create_user that was silently nulling department and accountEnabled in the response. * fix(microsoft_ad): resolve OData filter and search by owning operation The params mapper assigned result.filter from each filter subBlock in turn, so the last non-empty one won regardless of the selected operation. Because a subBlock keeps its value after the operation changes, a filter written for one endpoint was sent to every other collection operation — invalid OData against a different Graph resource, or a silently wrong page. Resolves the filter and search terms from an explicit operation-to-field map instead, so each operation reads only the field it owns. * fix(microsoft_ad): clear non-owning filter and search on the merged inputs The executor merges { ...inputs, ...transformedParams }, so declining to copy a stale filter is not enough — the serialized value survives the merge and still reaches the tool. Advanced-mode subBlocks are serialized on non-emptiness alone and never have their condition evaluated, so the value is present even when the field is hidden. Write filter and search on every operation, as undefined when the operation owns neither, so the merge clears them. * fix(microsoft_ad): clear the MFA flag and let paged user operations continue without a User ID The set_password MFA dropdown only wrote its key when non-empty, so the "No Change" empty string survived `{ ...inputs, ...transformedParams }` and reached Graph in place of a boolean. Assign it explicitly, including as `undefined`, the same way `filter` and `search` are handled. `list_user_app_role_assignments` and `list_user_devices` page by `@odata.nextLink`, and both tools already treat `userId` as optional once a continuation URL is supplied. Drop them from the required set when Next Page is filled in so pagination-only runs pass block validation. Also note on the reset_password output that a generated password reaches workflow outputs, run history, and the model, matching how other tools that return secrets document exposure. * fix(microsoft_ad): require the service principal ID only on the first page Every other single-resource ID field pairs its condition with a matching required rule; servicePrincipalId had none, so a first-page run could pass block validation with an empty ID and fail inside the tool instead. Require it unless a continuation URL is supplied, matching the paged per-user operations. * fix(microsoft_ad): reject a continuation URL from a different collection Every paged operation reads the one shared Next Page field, and a subBlock keeps its value after the operation changes. Paging /users and then switching the block to /devices short-circuited back to the user page, silently returning the previous collection instead of the selected one. Assert the continuation URL's terminal path segment against the collection the tool actually reads, which also rejects a nextLink pasted from an unrelated response.	2026-08-15 19:39:40 -07:00
Waleed	fed891f69d	docs(cli): add a CLI docs section generated from the command tree (#6762) ... * docs(cli): add a CLI section, generated from the command tree The `sim` CLI shipped with no coverage in the docs site. Adds a fourth top-level tab for it, and moves Academy last. The command reference is generated. `sim` exposes 147 leaf commands across 33 groups, most of them derived at runtime from the v2 route contracts, so a hand-written reference would be wrong the week after it was written. The generator walks the command tree `buildProgram()` hands to commander — the same tree the terminal parses — rather than re-deriving it from the contract, which would be a second implementation free to describe commands nobody can invoke. `check:cli-docs` is a zero-arg `check:*` script, so the existing audit runner picks it up and stale pages fail CI. Generating against the real tree surfaced a collision it had been hiding: `bulkUpdateKnowledgeDocuments` and `updateKnowledgeDocument` both derived to `sim knowledge documents update`. Commander resolves a duplicate to the first match, so the bulk form shadowed the single-document one and its flags were unreachable while still appearing in `--help`. The bulk form is now `batch-update`, matching how `tables rows batch-delete`/`batch-update` already handle the same REST overload, and the generator fails on any duplicate path so the next one cannot land silently. Five hand-written guides cover install, auth, configuration, output formats, and scripting. Also corrects two commands in the package README that do not exist as documented (`tables columns <tableId>`, and `--sort score:desc`, which is JSON). * docs(cli): document every flag from the contracts, add troubleshooting and a single-page reference The command reference was structurally complete but said almost nothing: 223 of 377 flags rendered as "Set sort by" because the CLI only ever read flag help from its own contract overrides, and fell back to restating the flag name. The prose already existed. The v2 route contracts carry 931 `.describe()` calls and the OpenAPI specs publish all of them — 327 parameters and 282 body properties, 100% coverage — but the generated operation table dropped every one, carrying only a per-operation summary. It now carries the field descriptions, the path-parameter descriptions, and positional help, so `--help` and the docs explain a flag the same way the API reference does. Placeholder descriptions are now zero, and 147/147 commands, 377/377 flags and 130/130 arguments are documented. `check:cli-docs` fails on a request field with no `.describe()` rather than letting it render as documentation that says nothing. Also in this pass: - Commands are root-level sidebar entries under a Commands heading rather than a folder, and headings are the command's description, so the table of contents distinguishes entries at the first word instead of repeating "sim knowledge documents …" fourteen times. A guard fails the build if two descriptions on a page collide, since they would share an anchor. - A single-page `Complete reference` carrying all 147 commands, for in-page search and for agents fetching `/cli/reference.mdx`. It keys on exact command paths because descriptions are only unique within a group. - A troubleshooting page, with every message copied from the source. - Table columns are sized by a local component; the flag column was starved while descriptions kept most of the row empty. - The prerelease install channels are dropped from the docs and the package README, which is what npm renders. * fix(docs): match the CLI tab by path segment, and escape backslashes before pipes `pathname.includes('/cli')` also matches `/integrations/clickup` and `/integrations/clickhouse`, so both existing integration pages lit the CLI tab and unlit Documentation. Matching is now per path segment. Anchoring to the start would not work either — a non-default locale prefixes the path, as in `/ja/cli` — so the segment is matched wherever it sits. Table cells now double a backslash before escaping pipes. A value ending in one turned `a\` + `|` into `a\\|`, which the table parser reads as an escaped backslash followed by an unescaped pipe, splitting the cell early. Nothing in the command surface contains a backslash today, so this was latent rather than visible. The reference page's global options table is two-column and was being wrapped in `CommandTable`, which sizes the second column for the `Required` cell of the three-column tables and crushed the description into 5.5rem. It now matches the overview page, which leaves that table unsized.	2026-08-15 19:25:34 -07:00
Waleed	6a29a9e2f4	feat(mssql): add Microsoft SQL Server integration (#6739) ... * feat(mssql): add Microsoft SQL Server integration Add a Microsoft SQL Server block backed by six tools (query, execute, insert, update, delete, introspect), mirroring the existing PostgreSQL and MySQL integrations. Connections go through the `mssql` (Tedious) driver: `connectionTimeout` is top-level while `encrypt`, `trustServerCertificate`, and `instanceName` live under `options`, and `port` is omitted when a named instance is used. Values are bound as `@paramN` via `request.input()`; no user value is interpolated into SQL. Identifiers are bracket-quoted after validation and WHERE clauses run through the shared injection guard. Introspection reads INFORMATION_SCHEMA plus the `sys.indexes` catalog views for tables, columns, primary keys, foreign keys, and indexes. The icon is a placeholder database cylinder drawn with `currentColor` until the real brand mark lands. Requires `bun install` for the new `mssql` / `@types/mssql` deps. * feat(mssql): use the SQL Server brand mark on a white tile * fix(mssql): pin the validated IP and correct the introspection catalog reads Tedious exposes `options.connector`, a hook that replaces its own resolve-and-connect path, so the connection can be pinned to the address `validateDatabaseHost` already approved instead of re-resolving the hostname. `server` stays the hostname because tedious derives the TLS `servername` from it independently of the connector, so SNI and certificate validation survive the pin. This brings MSSQL in line with the PostgreSQL and MySQL tools. Named instances are dropped: tedious resolves them with a UDP SQL Server Browser lookup issued outside the connector, and node-mssql deletes `port` whenever `instanceName` is set, so no configuration leaves a named instance pinned. A named instance is reachable through its static TCP port. Introspection fixes: - index key columns now filter on `key_ordinal > 0`; INCLUDEd columns and partitioning columns both report `0` and were being returned as key columns, ordered ahead of the real ones - foreign keys resolve through `sys.foreign_keys` / `sys.foreign_key_columns` rather than `INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS`, whose join to `TABLE_CONSTRAINTS` has no row when a key references a unique index and so dropped the key entirely - `is_unique` is a `bit`, which tedious maps to a boolean, so it is coerced rather than compared - schemas come from `sys.schemas`, which needs only `public` and carries no metadata-visibility caveat The WHERE-clause guard also covers `WAITFOR TIME`, `OPENQUERY`, `OPENXML`, the legacy `master..sys*` compatibility views, and extended and OLE-automation procedures beyond `xp_cmdshell`. Regenerates the docs and catalog artifacts the icon change left stale. * chore(mssql): commit the lockfile entries for mssql and its tedious dependency tree * fix(mssql): make the Query operation genuinely read-only The block label, tool description, and docs all present Query as SELECT-only while the route ran whatever T-SQL it was given, so an agent picking mssql_query because "it is only a SELECT" could delete rows. Screen the statement for mutating keywords with string literals stripped, which also catches the WITH ... DELETE form that a leading-token check would miss. Also switch the tool barrel to absolute imports per the repo convention. * fix(mssql): compose the shared WHERE guard and close the semicolon-less batch gap The local validateWhereClause re-derived an older copy of the shared patterns and scanned raw text, so it missed a bare 1=1 and false-positived on prose in a quoted value. Delegate to validateSqlWhereClause, which masks string literals first, and keep only the SQL Server surfaces it has no reason to know about. T-SQL needs no statement terminator, so every semicolon-anchored stacked-query check reads straight past `id = 1 DROP TABLE dbo.users`. Screen for a bare statement-introducing keyword to close that; word boundaries leave ordinary column names like updated_at and deleted_at untouched. Export maskSqlStringLiterals so the dialect layer masks the same way the shared guard does rather than carrying a weaker single-quote-only copy. * fix(mssql): screen administrative T-SQL and reject batches in the read-only path The previous round left two keyword lists maintained separately, and both were short: DBCC, KILL, CHECKPOINT, USE, and DENY were in neither, so `SELECT 1; DBCC SHRINKDATABASE(...)` and `id = 1 DBCC SHRINKDATABASE(...)` both got through. Collapse them into one MSSQL_STATEMENT_KEYWORDS shared by the query and WHERE screens so a keyword cannot be covered in one place and missed in the other, and add the administrative commands. Also reject any second statement after a semicolon in the Query path outright. That closes SELECT 1; <anything> structurally instead of by naming the anything, so the blacklist no longer has to be exhaustive to hold. * fix(mssql): reject SQL comments in the read-only query path A block comment placed inside a keyword splits it as far as a lexical scan is concerned, so keyword coverage cannot settle whether the server rejoins the halves. Refuse comments in the Query path instead of modelling the tokenizer. A SELECT sent through this operation has no need for one, and Execute Raw SQL still accepts them. Masking leaves comment markers intact, so a literal containing -- still passes. * fix(mssql): close the masker-desync bypasses and correct the catalog reads Every T-SQL screen runs over the shared literal masker, which was written for the MySQL dialect. Three ways to desynchronise it let real SQL hide inside what the masker believes is a string, two of which survived the existing even-quote check: - a backslash before a quote. T-SQL has no backslash escape, so the server closes the literal where the masker swallowed the quote and runs the rest as code. `a='x\' DELETE FROM dbo.t WHERE b='y'` holds four quotes and masks the DELETE out of the keyword screen entirely, so the read-only Query operation would run it. - a double quote inside a bracketed identifier, which the bracket rule missed because it only looked for single quotes. - any unbalanced double quote or backtick, which the parity check did not cover. All three now fail closed. Introspection also filters hypothetical and disabled indexes, which were reported as if they were live, and resolves the referenced side of a foreign key through sys.schemas so a cross-schema reference is no longer an ambiguous bare table name. Values bound through request.input are serialized when they are nested JSON, which the driver otherwise rejects with a bare "Invalid string.". * test(mssql): cover the block param merge and the operation-to-tool map Asserts on the merged `{ ...inputs, ...buildParams(inputs) }` the generic tool handler forwards rather than the mapper's return, since a key the mapper omits keeps its raw subBlock value through that merge. Pins the TLS toggles to their string form end to end — a switch subBlock would serialize `'false'`, which is truthy, and the route contract would coerce the user's off into on — and checks that duplicate subBlock ids agree on their seeded default. * fix(mssql): release a pool whose connect failed, and allow a keyword with no trailing space Only a pool handed back to the route reaches its `finally`, so a pool whose connect rejected leaked its tarn resources — one per attempt when a bad credential is retried. It now closes itself, and a failure to close cannot mask the connect error the caller needs. The read-only screen also anchored on `\s` after the opening keyword, which refused valid reads like `SELECT*FROM dbo.users` and `SELECT(1)`. A word boundary accepts those while still refusing `SELECTX`, and cannot loosen the screen — the keyword and batch checks run over the whole statement regardless. * chore(mssql): regenerate tool metadata and the integration catalog after rebase Artifacts rebuilt with the generators rather than hand-merged, so they carry both the mssql entries and the tools that landed on staging in parallel. * fix(mssql): reject a parenthesised or negated constant tautology in a WHERE clause The shared guard recognises `OR 1` but not `OR (1)`, `OR ((1))`, `OR NOT 0`, or `OR NOT (FALSE)` — a parenthesis or a NOT between the operator and the constant hides it. Both patterns require the constant to be the whole parenthesised term, so a real disjunct such as `OR (1 = priority)` is untouched. This narrows the gap rather than closing it, and is not meant to close it: an always-true expression is not lexically decidable in general, which is why the WHERE screen stays documented as defense-in-depth rather than a boundary. * chore(mssql): regenerate tool metadata after rebase onto staging Rebuilt with the generators so the artifacts carry the servicenow and crowdstrike tools that landed on staging alongside the mssql entries. * fix(mssql): screen trigger and state statements, and drop the space anchor on Execute DISABLE and ENABLE were missing from the shared statement list, so `SELECT 1 DISABLE TRIGGER dbo.audit ON dbo.users` passed the read-only screen as a semicolon-less batch and turned auditing off. SET, BEGIN, COMMIT, and ROLLBACK are added with them, since session and transaction state are reachable the same way. FETCH is deliberately left out: OFFSET ... FETCH NEXT is the standard paging clause, and screening it would reject the ordinary paged SELECT. Execute Raw SQL anchored its allowlist on `\s`, which refused `EXEC(@sql)` — the ordinary form of dynamic SQL, on the one operation meant to run it. It now uses `\b`, matching the read-only screen.	2026-08-15 19:13:45 -07:00
Justin Blumencranz	7f936dc02a	feat(tooling): enforce docs freshness and modernize agent skills (#6756) ... * feat(docs): fail CI when generated integration docs are stale * fix(docs): don't flag delete-then-recreated trigger pages in check mode * docs(skills): require docs:check in the integration authoring skills * chore(skills): migrate agent commands to native skills * fix(skills): clean orphaned Claude projections	2026-08-15 19:08:47 -07:00
Waleed	8a44621382	feat(cloudflare): add WAF rulesets, rate limiting, Zero Trust Access, R2, Workers, and Tunnels (#6740) ... * feat(cloudflare): add WAF rulesets, rate limiting, Zero Trust Access, R2, Workers, and Tunnels Extends the Cloudflare integration past DNS/zones/cache with the security and Zero Trust surface: - Rulesets engine (zone-scoped): list rulesets, get a ruleset, read a phase entry point, and create/update/delete rules. WAF managed-rule overrides are surfaced through the http_request_firewall_managed entry point, since Cloudflare has no dedicated overrides endpoint. - Rate limiting (zone-scoped) via the current Rulesets-based http_ratelimit phase, not the deprecated rate_limits endpoint. - Cloudflare Access (account-scoped): applications, application policies, groups, identity providers, and service tokens. - R2 buckets, Workers scripts/routes, and cloudflared Tunnels. Destructive operations (delete application, delete policy, revoke service token, delete rule, delete bucket) spell out their blast radius, and every tool branches on the envelope's success flag rather than the HTTP status. Security events are intentionally omitted: Cloudflare exposes them only through the GraphQL firewallEventsAdaptive dataset, whose field list is not documented outside schema introspection. * fix(cloudflare): correct docs drift and remove any from the tool layer Validation pass over all 47 Cloudflare tools against developers.cloudflare.com. - Two tool descriptions still escaped a quote as \'. That reaches the model verbatim and truncates the generated MDX cell — the get_zone_settings `value` output row was missing from the published docs entirely. Both are now template literals, and the row is back. - list_rulesets ignored pagination. The endpoint pages by cursor via result_info.cursors.after (not page/per_page), so a zone with many rulesets silently truncated with no way to page. Expose per_page + cursor and return the next cursor. - The managed-ruleset override description claimed action and enabled were the overridable properties. They are the ones the Rulesets engine documents at every level, but individual managed rulesets add more: an OWASP Core Ruleset rule override also takes score_threshold. Corrected in both the tool output description and the block's action-parameters wand prompt. (sensitivity_level is a DDoS override, not a WAF one — deliberately absent.) - list_tunnels/get_tunnel dropped the documented `metadata` field. - list_r2_buckets appended order=name whenever any filter was set. `order` only qualifies `direction`, and `name` is its sole documented value. - Path-interpolated IDs are trimmed, so a pasted ID with trailing whitespace no longer 404s. - Replaced every `any` in the integration with checked types: a shared CloudflareEnvelope plus per-resource raw payload interfaces, read through readCloudflareResponse. The mappers in utils.ts were the widest hole — typing them caught four real output-shape mismatches (identity provider read_only, service token enabled, DNS record meta/priority, certificate geo_restrictions) that `any` had been hiding. - BlockMeta only described DNS and zone work. Added templates and skills for the WAF, rate limiting, and Zero Trust Access surfaces the block now has. Confirmed against the docs and left unchanged: rulesets/rate limiting are zone-scoped and Access/R2/Workers scripts/Tunnels are account-scoped while Workers routes are zone-scoped; tunnels live under /accounts/{id}/cfd_tunnel; the ratelimit object is a sibling of action/expression, not nested in action_parameters; every rate limiting period and mitigation_timeout option matches the documented set; R2 delete returns an empty result so echoing the requested bucket name is correct; app-nested Access policy endpoints are current, not deprecated; and every tool fails on a 200 carrying success:false. * fix(cloudflare): stop per-operation subblock defaults colliding on a shared id Subblock initial values are seeded into block state keyed by subblock id — both stores/workflows/utils.ts and lib/workflows/defaults.ts assign `subBlocks[subBlock.id]` in a plain forEach — so two controls sharing an id leave one stored value and the last definition in file order wins. Four ids were duplicated with differing defaults: - `type` was defined four times. The Access "Application Type" control is last, so every new block seeded `type = 'self_hosted'` and the three DNS record controls inherited it — Create DNS Record sent a Zero Trust application type as its record type. The subblock added on this branch broke a default on tools that shipped long before it. - `status` was defined three times. The empty tunnel filter is last, so List Certificates lost its `all` default. - `proxied` was defined three times. An empty filter is last, so Create DNS Record lost its explicit `false`. - `action` was defined twice. The rate limiting dropdown is last, so the ruleset-rule action input was seeded `block`, quietly making "block live traffic" the default for a WAF custom rule the user never configured. Give the colliding controls their own ids and map them back to the tool params per operation, ahead of the coercions that read them, so each operation keeps its own default. The other 17 duplicated ids agree on their value and are left shared. Adds tests covering each separated default plus a sweep asserting no id carries two different seeded values, so a future duplicate goes red. * fix(cloudflare): generate array include rules and allow bootstrapping a phase ruleset The Access policy include wand asked for a JSON object while the tool parses the field with parseJsonArrayParam, so generated rules failed validation. Switch it to json-array, whose prompt reinforcement omits the object braces. Rate limiting and WAF custom rules could only be appended to an existing ruleset, but a zone that has never had a rule in a phase has no entry point ruleset and returns 404, leaving no way to add the first rule. Add cloudflare_create_ruleset for the documented POST /zones/{id}/rulesets bootstrap, seeded with optional initial rules. * fix(cloudflare): correct verified API defects and stop filters leaking into writes Independent re-validation of all 48 tools against developers.cloudflare.com turned up defects that the shipped tools would have hit on their happy path. Delete DNS record reported every success as a failure. That endpoint is the one Cloudflare v4 response with no envelope — its documented body is `{"result":{"id":...}}` with no `success` — so `!data.success` was always true. Branch on an explicit `=== false` instead. The two replace-semantics PATCH endpoints could silently destroy live config. Update rate limit rule defaulted a missing action to `block`, converting an existing `log` or challenge rule into a hard block on real traffic; update ruleset rule left action and expression optional and had no `ratelimit` or `logging` passthrough, so updating a rate limiting rule stopped it rate limiting. Both now require the fields the replacement needs, and the ruleset rule carries the two nested objects through. Access applications were unbuildable for most types: `domain` was required, but it does not exist on the saas, app_launcher, warp, biso, dash_sso, infrastructure, mcp, mcp_portal, or proxy_endpoint request variants. The application type enum was also six values behind. Access group `is_default` is an array of rule objects, not a boolean. Purge cache merged every supplied target into one body, but the purge body is a one-of over the five target kinds; it now names the conflict instead. The remaining fixes are documentation drift: the priority field is MX and URI only (an SRV record carries priority inside its content), the certificate status filter documents only "all", the Worker tag filter takes tag:allowed pairs, and the managed-rule override list conflated the DDoS-only sensitivity_level with the WAF rule-level set. Separately, controls that share a subBlock id share one stored value, and `shouldSerializeSubBlock` short-circuits on `mode: 'advanced'` before it evaluates `condition` — so a hidden list filter was reaching a write. A `list_dns_records` content filter could overwrite a record's content, cache tags could be written onto a DNS record, and the zone status enum could reach the tunnel list, whose enum is disjoint. Filters that differ from the value they collided with now carry their own id, remapped through one table before any coercion. Sharings that mean the same thing everywhere are unchanged. Aliases are cleared by explicit assignment rather than destructuring, because the executor merges the mapper's output over the raw inputs and a merely omitted key survives as its raw subBlock string. The tests assert on that merged result, and three mechanical invariants now go red on a new collision: no id spans a read filter and a written value, no dropdown id carries two option sets, and no hidden advanced control feeds an operation that cannot render it. That last one found the name filter reaching three list operations. * docs(cloudflare): point self_hosted_domains at its replacement Cloudflare deprecated the field in favour of destinations, which the tools already surface. The output stays — Cloudflare still returns it — but the description now says which one to read. * refactor(cloudflare): drop a dead exception from the empty-type guard create_dns_record now takes its record type from the recordType control, whose dropdown has no empty option, so the operation can never reach this guard with an empty type. Clear it unconditionally. * fix(cloudflare): point the canvas sentences at the renamed filter controls The list filters that were split off their write-side twin kept their old ids in canvasPresentation, so seven clauses referenced a control that is no longer visible for that operation — check:canvas-sentences catches exactly this, and a broken clause fails silently on the card rather than throwing. * fix(cloudflare): stop the rate limiting action defaulting on a replacing update Making action required on update_rate_limit_rule was only half the fix: the Action dropdown still seeded block for the update operation too, so an update that edited only the threshold kept sending block and converted a live log or challenge rule into a hard block — exactly the harm the required flag was meant to prevent. The update now has its own control with no seeded value, so the action is something the caller states rather than inherits. The certificate status filter also still offered Active and Pending, which Cloudflare does not document for that endpoint; the only documented value is all, and omitting it returns active packs. * fix(cloudflare): stop the Access replacements seeding a type and a decision Same class as the rate limiting action: both Access updates are full replacements, and the shared controls seeded self_hosted and allow for the update operations too. Editing only a policy's include rules would silently convert a live deny, bypass, or non_identity policy to allow — widening who gets in — and editing an application would rewrite what it IS. Each update now has its own required control with no seeded value, so the type and the decision are stated rather than inherited. Regression tests cover both, and the canvas sentence follows the renamed decision control.	2026-08-15 18:50:05 -07:00
Waleed	cabd2e2fc1	feat(datadog): extend to 40 tools and align every operation with the published OpenAPI specs (#6745) ... * feat(datadog): add incidents, SLOs, dashboards, synthetics, Cloud SIEM, and APM tools Extends the Datadog block from 12 to 39 operations, all verified against Datadog's published OpenAPI specs: - Incidents (v2, public beta): list, get, create, update, add todo - SLOs (v1): list, get, create, update, delete, history - Dashboards (v1): list, get, create, delete - Synthetics (v1): list tests, get test, latest results, trigger, pause/resume - Cloud SIEM (v2): search signals, get signal, update triage state, assign, list detection rules - APM: search spans (v2), list Service Catalog definitions (v2) Adds tools/datadog/utils.ts so every tool builds its URL from the configured site/region and shares the JSON:API-aware error extraction, and handles the v1 flat vs v2 envelope shapes and cursor pagination per endpoint. * fix(datadog): align every operation with the published OpenAPI specs Validated all 39 shipped operations (plus the 12 pre-existing ones that had never been spec-checked) against the DataDog v1 and v2 OpenAPI schemas. - `POST /api/v2/downtime` requires `monitor_identifier`, so a downtime created without a monitor id was rejected. Default to the `*` monitor tag. - A one-time downtime schedule declares `additionalProperties: false` and accepts only `start`/`end`; the timezone moves to `display_timezone`. - `GET /api/v2/downtime` has no `monitor_id` filter, and the response carries no `disabled` attribute. Downtime ids are UUID strings, not numbers. - Drop scaffold types for operations that do not exist (metric metadata, event query, monitor update/delete/unmute, host listing) along with their fields. - Note that monitor mute is no longer published in the v1 specification. - Add browser Synthetic test results, which the browser-specific endpoint returns with its own camelCase step-count shape. - Replace every `any` with a spec-derived interface, keeping the polymorphic service-definition schema opaque. * fix(datadog): remove remaining any types and declare every returned output field Replace the six surviving `Record<string, any>` request-body and response-cast sites with concrete spec-derived shapes, and declare the output fields that transformResponse already returned but outputs omitted: - create_downtime / list_downtimes: timezone, created, modified - create_monitor / get_monitor: options, creator - list_monitors: message, priority, options, created, modified, creator - query_logs: content.attributes, content.tags - update_security_signal_state / _assignee: type; assignee also gained the archiveReason/archiveComment pair its sibling already declared - query_timeseries: series gained the items shape it never described * fix(datadog): stop dropping downtime targeting inputs in the block mapping create_downtime accepts monitorTags, timezone and muteFirstRecoveryNotification, but the block exposed no inputs for them and never forwarded them. Monitor-tag targeting silently fell back to the `*` tag, so a downtime meant for one team's monitors muted every monitor in scope. Adds the three advanced sub-blocks and wires them through. Also routes list_downtimes' currentOnly through toSwitchBoolean. A switch yields the strings 'true'/'false', and 'false' is truthy, so turning the toggle off still sent current_only=true. Every other switch in the block already used the helper; this was the last raw one. * fix(datadog): correct metric type codes, stop SLO update data loss, drop unpublished mute Independent re-validation of all 39 operations against the DataDog/datadog-api-client-go generator specs (v1 and v2 openapi.yaml) rather than the client-rendered docs site. Correctness: - submit_metrics sent inverted MetricIntakeType codes (gauge as 0/unspecified, rate as 1/count, count as 2/rate), silently changing how Datadog aggregated every submitted series. The spec enum is 0 unspecified, 1 count, 2 rate, 3 gauge; an unrecognized type is now omitted so Datadog infers it. Also stops stamping an invented `resources: [{name:'host'}]` default and now forwards `interval`, which Datadog requires for count and rate metrics. - update_slo replaced the whole SLO with only the fields the caller filled in, so editing one field erased description, tags, query, monitor_ids, groups, thresholds, and timeframe. PUT /api/v1/slo/{slo_id} is a full replacement, so the stored SLO is now read first and the supplied edits are overlaid onto it, with the read-only fields stripped. - update_incident admitted empty strings, so a blank input could blank a stored incident title or fail as an invalid date-time. - query_timeseries reported a failed query as success: Datadog returns 200 with a non-ok `status` and the reason in `error`. - create_monitor swallowed malformed options JSON and created a monitor with no thresholds. - send_logs rebuilt each entry from a fixed field list, discarding the custom attributes Datadog accepts as additionalProperties, and padded absent optional fields with empty strings. Removed: - mute_monitor. /api/v1/monitor/{monitor_id}/mute is absent from the v1 spec entirely, there is no unmute counterpart to reverse it, and downtimes are the supported mechanism. Contract accuracy: - Security signal search advertised relative times ("now-1h"); the spec types filter.from/to as format: date-time. Descriptions, placeholders, and wand prompts now produce ISO-8601. - list_incidents advertised an `include` value ("integrations") that is not in the spec enum, and neither incident tool trimmed the comma-separated list, so "users, attachments" 400d. - Invalid "ok" group state dropped from both monitor descriptions. - time_slice removed from SLO create input, which cannot build one without an SLI specification. - DatadogSite gains ap2, uk1, and us2.ddog-gov.com. Pagination and errors: - list_downtimes silently truncated at Datadog's default 30 with no way to page; adds page[limit]/page[offset] and surfaces totalCount. - query_logs returned a cursor it had no way to accept back. - Error extraction consolidated onto datadogErrorMessage, which now also reads the dictionary-shaped errors of the SLO delete conflict. Ten tools were reading `.detail` off plain strings or the raw entry off objects, degrading every failure to a bare status line. - Debug logging removed from list_monitors. Adds 29 regression tests, each verified to fail when its fix is reverted. * fix(datadog): add SEV-0, document page-size caps, drop unsourced output defaults - The severity dropdown omitted SEV-0, which IncidentSeverity allows and both incident tool descriptions already advertised. - Page-size descriptions now state Datadog's documented default of 10 and cap of 100 instead of an arbitrary example, so an agent does not request an out-of-range page. - trigger_synthetics_tests emitted an explicit null for a string-typed optional output, and update_synthetics_status reported 'live' on the error path regardless of what the caller actually requested. * fix(datadog): keep mute_monitor and add the missing unmute counterpart Reverses the removal in the previous commit. Absence from the datadog-api-client-go generator spec showed the endpoint is unpublished there, not that it is retired: Datadog's official Python client still implements it on master as `Monitor.mute(id, scope=, end=)` and `Monitor.unmute(id, scope=, all_scopes=)` (datadogpy datadog/api/monitors.py), which `_trigger_class_action` resolves to `POST /api/v1/monitor/{id}/mute` and `/unmute` with exactly those body fields. mute_monitor has also been in the block since #2175 in December, so dropping it would have broken existing workflows for an endpoint that two independent sources agree is live. The genuine defect was that muting was a one-way trapdoor: Sim could mute a monitor but had no way to reverse it. Adds datadog_unmute_monitor, sharing the monitor ID and scope inputs with mute, so the operation is recoverable from the same block. Also: mute no longer discards the response body (it now reports the monitor id, name, and state), routes errors through datadogErrorMessage, encodes the monitor ID in the path, and stops dropping an explicit `end` of 0. * fix(datadog): make downtime targeting explicit and reach downtime pagination from the block Addresses the review findings on the previous round. - create_downtime accepted both a monitor ID and monitor tags but `monitor_identifier` is a oneOf, so it silently kept the ID and dropped the tags, muting a different set of monitors than the caller asked for. It now rejects the ambiguous combination. - create_downtime ran Number.parseInt on the monitor ID with no validation, so a non-numeric value became NaN and serialized as null inside monitor_identifier. It now uses the same parseMonitorIds guard the SLO path already had, naming the offending value. - list_downtimes gained limit/offset in the tool but the block exposed neither, so no block-driven call could page past Datadog's default. Adds the two sub-blocks and wires them through the params mapper. - The block did not declare the totalCount the tool now returns, so nothing downstream could bind to it. * fix(datadog): tolerate non-string list inputs and keep the shipped mute subblock ids Both defects were introduced by this branch. - splitCommaList called .split on its argument, so routing create_downtime's monitorId through it turned a legitimate numeric input into a TypeError before the request was built. A <Block.output> reference to get_monitor or list_monitors resolves to a number, and an LLM tool call can pass a number or an array, so the helper now normalizes all three shapes. The previous Number.parseInt path had accepted a number by coercion. - Adding the unmute operation renamed the mute subblock ids scope/end to muteScope/muteEnd. Workflow state is persisted by subblock id, so every existing Mute Monitor block would have kept the old keys and silently lost its scope and end time. Restored the shipped ids; both are still unique block-wide and no operation reads another operation's value. * fix(datadog): compare downtime targets after parsing, not before A whitespace-only Monitor ID is truthy as a raw string but parses to no monitor, so the oneOf conflict guard rejected a valid tag-targeted downtime whenever the untouched Monitor ID field carried blank text. Both sides are now compared after parsing.	2026-08-15 18:48:46 -07:00
Waleed	9e67655b23	feat(servicenow): semantic incident, change, catalog, approval, CMDB, and knowledge tools (#6747) ... * feat(servicenow): add semantic incident, change, catalog, approval, CMDB, knowledge, and directory tools The ServiceNow block only exposed generic Table API CRUD, so every real task started with "which table is that on?". This adds 27 semantic tools that wrap the same Table API plumbing under the names customers actually use. - Incidents: create, get by number or sys_id, search, update, resolve, close, and append a work note or customer-visible comment. - Change: create, get, list, update, move state, and list change tasks through the documented Change Management API. - Service catalog: browse items, order one via the Service Catalog API order_now endpoint, and list or get requested items. - Approvals: list pending approvals for an approver, approve, and reject. - CMDB: search CIs on any class, read a CI with its inbound and outbound relations through the CMDB Instance API, and list cmdb_rel_ci rows. - Knowledge: search and read articles through the Knowledge Management API. - Directory: find a user by email or user name and list group members, which is what fills assigned_to and assignment_group. Reference fields are the usual source of confusion, so every semantic read defaults to sysparm_display_value=all — a reference comes back as both its sys_id and its label — and every semantic write exposes sysparm_input_display_value so a display name can be written instead of a sys_id. Coded state values are exposed as labelled dropdowns built from one constants module rather than raw integers. The shared instance-URL, Basic Auth, sysparm, envelope, and error handling now live in tools/servicenow/utils.ts, and the existing eight generic tools were moved onto it rather than keeping their own copies. * fix(servicenow): stop per-operation subblock defaults colliding on a shared id Subblock initial values are seeded into block state keyed by subblock id, so two subblocks sharing an id leave one stored value and the last definition wins. Three ids were duplicated with differing defaults: - `displayValue` was defined twice, unset for the generic Table API tools and `all` for the semantic ones. The semantic definition won, so a new block set to Read Records or Aggregate Records sent `sysparm_display_value=all` — a wire change to two already-shipped tools. - `state` was defined four times. The Approval State definition won, so every new block carried `state=requested`, which Create Incident wrote to the incident and Move Change State used instead of its own `-5` default. Give the colliding controls their own ids and map them back to the tool params per operation, so the generic tools keep their original request shape and each semantic operation keeps its own default. Also correct descriptions that overstated what the API does: the LIKE operator is not documented as case-sensitive, List Requested Items has no requester filter, and the Change Management API task shape differs from the Table API. Adds tool tests covering the refactor invariants for the eight pre-existing Table API tools and the display-value separation. * feat(servicenow): read a change request's real next states from the instance The change tools describe state transitions using the base-system codes, which only hold on an instance that has not customized its change model. ServiceNow publishes an endpoint that answers the question directly for the record in hand, so use it rather than keep assuming. GET /api/sn_chg_rest/change/{sys_id}/nextstates returns the states reachable from the change request, the instance's own state-value-to-label map, and, for model-driven changes, each transition with the conditions it has and has not met. The tool flattens the per-target-state grouping ServiceNow returns (each transition already carries from_state and to_state, so nothing is lost) and derives the states whose conditions currently pass. Also record the sourcing for the coded values in constants.ts: the change states and close codes are published as a table, but the incident state codes are not — only 6 (Resolved) appears in the docs — so mark the rest as defaults rather than guarantees. Note that sysparm_input_display_value also reinterprets date and time values in the caller's timezone instead of GMT, which matters for the change start and end dates. * docs(servicenow): stop asserting undocumented coded values in placeholders The additional-fields examples used hold_reason with a coded value of "1". ServiceNow documents the On hold reason choices by label only — Awaiting Caller, Awaiting Change, Awaiting Problem, Awaiting Vendor — and publishes neither the column name nor the codes, so the example was asserting something unsourced. Use a field whose value is caller-supplied instead, and record the On Hold requirement on the incident state control using the labels the docs actually give, including that Awaiting Caller makes Additional Comments mandatory. * fix(servicenow): drop phantom parent fields from the catalog order output order_catalog_item read parent_id and parent_table off the order_now response. Those fields belong to submit_producer, a different Service Catalog endpoint; the documented order_now result is sys_id, number, request_number, request_id, and table. Both outputs were therefore always null. * fix(servicenow): correct what knowledge search returns as an article id Search results carry a table-prefixed identifier — "kb_knowledge:9e528db1..." — not a bare sys_id, while GET /knowledge/articles/{id} accepts only a bare sys_id or a KB number. The output described it as a sys_id and the tool description told callers it was what they needed to fetch the article, so chaining the two tools on that field would fail. Point callers at the KB number instead. Relevancy score is documented as a number, not a string. * docs(servicenow): cite the page that actually documents approval statuses The approval state constants pointed at the classic-approvals landing page, which does not list the statuses. Approval status is documented separately and names four — Requested, Approved, Rejected, and Not Requested. * fix(servicenow): stop constant interpolation leaking into tool descriptions The docs generator and the client-facing integration catalog read tool descriptions from source rather than from the evaluated module, so a template literal like `state ${INCIDENT_STATE.RESOLVED}` shipped to users verbatim: `apps/sim/lib/integrations/integrations.json` and the published ServiceNow integration page both rendered `${INCIDENT_STATE.RESOLVED}` instead of `6`. Inline the base-system coded values in the description text; the constants stay in use everywhere behavior depends on them. Also drops an escaped `\'` in the `inputDisplayValue` description for the same reason, and adds a standing guard test asserting no subBlock id carries two different seeded defaults — the invariant behind the per-operation defaulting bug, now checked structurally rather than only through the four per-operation cases. * refactor(servicenow): type the shared response boundary instead of any `parseServiceNowResponse` returned `any`, so every tool reading `data.result` did unchecked property access — a shape change on the instance side would have produced a wrong-typed output silently rather than a type error. Introduces `ServiceNowEnvelope` (`result?: unknown`) as the parser's return type and narrows the record index signatures from `any` to `unknown`. Adds `toRecordObject`, `readString`, and `readNestedNumber` so the tools that read individual fields narrow deliberately at the point of use. This surfaced five genuinely unchecked reads: Order Catalog Item, Get Knowledge Article, and Search Knowledge were declaring `string | null` / `number | null` outputs while emitting whatever the instance sent, and Get Change Next States assigned an unvalidated object to `Record<string, string>`. Each now coerces or drops a non-matching value rather than passing it through. * fix(servicenow): publish the shared tool params and stop offering inert controls The docs generator reads tool source rather than importing it, so the shared `params.ts` consts the semantic tools spread were dropped from every published Input table — 27 of 35 ServiceNow tools listed no instance URL, username, or password at all. Follow a spread into the module it is imported from so those rows are published; ten other integrations gain the rows they were missing for the same reason. Two controls were dead on arrival: Additional Fields was offered on Move Change State and Add Incident Comment, and neither tool read it. Wire it through the change transition, which needs it, and drop it from the comment tool, whose body is exactly one journal field. Every coded-value control was a select-only dropdown, so a customized instance's state or close code was unreachable — sharpest on Move Change State, whose target state is required and whose real codes come from Get Change Next States. Make them comboboxes. Also correct two doc claims ServiceNow does not publish (the incident state citation pointed at a page that does not exist and compares the legacy incident_state field; closing an incident is not documented as requiring itil_admin), replace Record<string, any> with checked narrowing that surfaced two unsound widenings, and document that List Change Tasks returns a fixed {value, display_value} shape under `tasks` rather than `records`. * fix(servicenow): stop one subblock id from carrying two value spaces Subblock values are stored per block keyed by id, so an id reused across operations keeps its value when the operation changes. Incident and change shared `state`, and `closeCode`, `closeNotes`, `comments`, and the knowledge search phrase were each reused for a different value space — so an incident state could be written onto a change request, an incident close code sent as a change close code, or an encoded query searched as knowledge text. Give each value space its own subblock and republish it to the tool param from the operation that owns it, the way targetState and approvalState already work. The generic Table API ids stay exactly as they are, since renaming one would orphan the stored value of every workflow already using those shipped tools. The previous guard only compared seeded defaults, which is why this class stayed hidden; the new one asserts against the merged params a tool actually receives. * fix(servicenow): point the canvas sentences at the renamed subblocks The split of the colliding subblock ids left the operation sentences anchored on ids that no longer exist, so those clauses would silently drop from the card. * fix(servicenow): validate collection members and split the fields projection toRecordArray cast every member of a successful response, so a null or scalar in a collection was handed to the next block as a record while the tool reported success and its declared output said that could not happen. Members that are not plain objects are now dropped, and knowledge articles and change transitions get the same narrowing. The two response types that described an unverified inner shape now say what is actually checked. The 'fields' subblock also carried two value spaces: a JSON body on Create and Update Record, a comma-separated projection everywhere else. Operations added since read a separate returnFields control, so a body can no longer arrive as a projection or the reverse. The shipped ids are untouched, since renaming one orphans the stored value of every workflow already using those tools.	2026-08-15 18:33:30 -07:00
Waleed	4bc89c9256	feat(crowdstrike): add alerts, host response, IOC, Spotlight, RTR, and case tools (#6746) ... * feat(crowdstrike): add alerts, host response, IOC, Spotlight, RTR, and case tools CrowdStrike Falcon shipped only three read-only Identity Protection sensor tools. This adds 20 tools across the response and investigation surface SecOps teams actually automate against. Alerts (current Alerts API): query, get details, update status/assignment/ tags/comment/visibility. Hosts: contain, lift containment, hide, unhide. Host groups: query, get details, add/remove hosts. IOC Management: query, get, create, update, delete. Spotlight: query vulnerabilities, get vulnerability details. Real Time Response: init session, execute a read-only command, poll command status, delete session. Case Management: query cases, get case details. Every endpoint, request field, and response field is taken from CrowdStrike's published surface (developer.crowdstrike.com API reference, FalconPy endpoint definitions, and the swagger-generated gofalcon models). Required API scope is documented in each tool description. Deliberately not implemented: - Detects API: decommissioned 2025-09-30, superseded by Alerts. - CrowdScore Incidents API and behaviors: decommissioned 2026-03-09 and removed from the developer center entirely. Case Management is CrowdStrike's replacement, so its two documented read operations are implemented instead. - Case create/update/merge: the swagger types case `status` and `severity_info.level` as bare strings with no enum, so a correct write cannot be built without guessing. CrowdStrike answers 200 with a populated `errors` array for partial failures. Responses now surface those per-item errors, and an empty result set carrying errors is reported as a failure rather than silently succeeding. The route's shared Falcon client, response normalizers, and operation dispatch move into colocated modules so the handler stays readable at 23 operations. * fix(crowdstrike): correct the RTR read-tier commands and stop dropping the IOC delete filter Validation pass over all 23 tools against CrowdStrike's swagger-generated SDKs (gofalcon falcon/models + falcon/client, FalconPy _endpoint/*.py) turned up four real defects. The Execute RTR Command dropdown offered `csrutil` and a bare `reg`. Neither is a read-tier base command: CrowdStrike's own swagger description for RTR_ExecuteCommand enumerates cat, cd, clear, env, eventlog, filehash, getsid, help, history, ipconfig, ls, mount, netstat, ps, and "reg query". `csrutil` appears nowhere in CrowdStrike's published surface, and `reg` alone is not a base command — the registry variants are "reg query" (read) and "reg set"/"reg delete" (Active Responder). Both entries are corrected everywhere they were repeated: dropdown, tool description, and param description. Delete Indicators showed a Filter input, declared the param, accepted it in the contract, and implemented CrowdStrike's documented filter-takes-precedence rule in the route — but the block never mapped the field into the tool call, so the filter was silently discarded and a filter-only delete failed validation. The `filter` case is now mapped alongside the ID list. A 200 carrying only envelope errors was reported as HTTP 200 with success:false, which reads as a success to anything inspecting status. Failures now adopt the per-item error code the envelope supplies, falling back to 502. Alert updates gain a first-class Remove Tags By Prefix field. The spelling was previously unresolvable, so it was left to the raw action-parameter escape hatch; CrowdStrike's swagger settles it as `remove_tags_by_prefix` in both the PatchEntitiesAlertsV2 and PatchEntitiesAlertsV3 descriptions. Case Management and Spotlight scopes now name the OAuth scope string (case-templates:read, spotlight-vulnerabilities:read) alongside the label the Falcon API client UI shows, so either rendering is findable. * fix(crowdstrike): stop blank sensor filters reaching Falcon and expose the RTR outputs The falcon.ts/normalize.ts/operations.ts split routed query_sensors through the shared buildUrl helper, which skips only undefined. An empty filter or sort string therefore emitted `?filter=` / `?sort=` where the pre-split route omitted the param, sending Falcon an empty FQL expression. Reject blank values in the contract instead, matching the newer operations. Also surface the ten RTR fields the tools already return but the block never declared, and broaden the block metadata past the original sensor-only surface. * fix(crowdstrike): fail query operations on error-only envelopes and guard IOC pagination Falcon can answer 200 with an errors array and no resources. The detail operations already treated that as a failure, but the five query branches returned an empty successful result, so a failed alert query read as a valid no-match to the calling workflow. Blank FQL rejection now covers the alert, host-group, indicator, vulnerability, and case contracts too, not just sensors, and Query Indicators rejects offset combined with after instead of forwarding a pagination pair CrowdStrike refuses. * fix(crowdstrike): stop blank inputs reaching Falcon and restore the dropped output docs The executor merges `tools.config.params` over the raw block inputs, so a key the mapper omitted kept its raw subBlock value — and an untouched subBlock is stored as `null`, which the route contract rejects. Query Alerts with an empty Filter, Update Alerts without every optional field, and Delete Indicators without an audit comment all 400'd before reaching CrowdStrike. Seed every optional key as `undefined` so omission is authoritative, which also stops a value left over from another operation riding along. Shared output consts in `outputs.ts` were silently dropped from the generated docs: the generator scans tool source and resolves consts only from `types.ts`, so `errors`, `affected`, and `pagination` rows vanished from 17 tool pages and every nested property row with them. Inline the literals. Against CrowdStrike's own generated SDKs and developer portal: - add csrutil, ifconfig, users, and the eventlog subcommand forms to the read-tier RTR base commands, matching PSFalcon's ValidateSet - add detection_suppress/detection_unsuppress and cap host actions at the documented 100 ids - cap the IOC search limit at the documented 500, not 2000 - correct the Cases scope to "Cases: Read"; case-templates guards a different collection - type the IOC payload so a blank string cannot clear a stored field on PATCH - send `MsaRangeSpec` bounds capitalized, as the spec serializes them - fail the sensor and RTR-session-close paths on a 200 whose envelope carries only errors, and surface partial sensor errors - give Delete Indicators its own filter so a stale alert query cannot widen it - drop the pre-selected network-isolating host action * fix(crowdstrike): correct the RTR command tier, IOC update contract, and US-3 region Independent re-validation against gofalcon's swagger-generated models and CrowdStrike's developer center turned up several wire-level errors. - Real Time Response advertised "eventlog backup"/"export"/"list", "reg query", ifconfig, and users as base commands. base_command names a command family and subcommands belong in command_string; the eventlog write variants are Active Responder commands that would fail on scope under this Read-scoped tool, and ifconfig/users appear in neither authoritative list. The block now offers the 16 documented read-tier families and the contract enforces them. - Indicator updates accepted an entry with no id, which cannot name a record, and accepted type/value, which the update model does not expose. Creates accepted an entry with no type, value, or applied_globally -- the one property CrowdStrike marks required, and the one that decides fleet-wide scope. - CrowdStrike documents that PATCH overwrites any omitted field with a blank value. The contract can only catch blanks, so the update tool now tells the caller to read the indicator first and resend its full field set. - Added the US-3 commercial region, which was missing from every cloud list. - Aggregate queries silently dropped percents and filters_spec. - Deleted the response-envelope body unwrap: no endpoint this integration calls returns that shape, and getFalconErrorMessage never honored it anyway. - Softened the Detects and Incidents claims to what the sources actually state. A tool description longer than the docs generator's 600-character id-search window silently publishes as an empty string; three descriptions had crossed it. Shortened them and added a test that fails before the catalog goes blank. * docs(crowdstrike): name the endpoint and Identity Protection scope on the sensor tools The three sensor tools were the only ones in the family that named neither their endpoint nor their OAuth2 scope, and none of them said these are the domain controllers Falcon Identity Protection monitors rather than Falcon endpoint sensors -- a distinction an agent choosing between them and the Hosts tools has no other way to make. Identity Protection Entities: Read is also a separate product entitlement from Hosts and Alerts. * refactor(crowdstrike): say which ID caps are CrowdStrike's and which are Sim's Every bulk-ID limit claimed CrowdStrike as its source, but only the sensor (5000), host action (100), indicator batch (200), and Spotlight (400) caps are published. The alert, host group, indicator, and case caps are Sim's own bound on request size, and the validation message now says so instead of attributing a limit CrowdStrike does not document.	2026-08-15 18:19:20 -07:00
Waleed	cbbcca970c	fix(okta): stop partial updates erasing stored profile data (#6751) ... * fix(okta): stop partial updates erasing stored profile data Post-merge audit of the Okta integration (follows #6741), verified against the OpenAPI spec bundled in okta-sdk-golang/.generator. Two updates could silently destroy data: - `update_group` targets `PUT /api/v1/groups/{groupId}`, which Okta documents as `replaceGroup` — it swaps the profile wholesale. Sending only the two fields the tool exposes erased the stored description on every rename, and dropped every org-defined custom attribute along with it. The tool now reads the group and overlays the supplied fields before replacing, matching the read-modify- write `salesforce_update_custom_field` already uses for the same hazard. - `update_user` gated its profile fields on `!== undefined`, so an empty string reached Okta and blanked the stored value. The block strips blanks before they get there, but the tool is `user-or-llm` and a model routinely emits `""` for a field it has nothing to say about, so the guard belongs on the tool. Also corrected: - `forgetDevices` defaults to true at Okta, so the unseeded switch rendered off while remembered factors were in fact being cleared. - Group rules take a plain keyword on `search`, not the SCIM-style expression the shared Search field's wand generates, so they get their own field. - `get_logs` dropped `limit=0`, which the spec documents as valid. - `get_user` emitted an activation timestamp under `activated`, which the block declares as the lifecycle boolean; the timestamp is now `activatedAt`. - Descriptions that overstated what an endpoint does: `list_users` omits DEPROVISIONED users, `delete_user` deactivates before it deletes, `delete_group_rule` answers 202, and `excludedGroupIds` is always empty because Okta does not support group exclusions. * fix(okta): forward the abort signal through the group read-modify-write * test(okta): rename the shared body-builder helper * fix(okta): key the send-email and search mappings off the operation * docs(okta): use TSDoc for the new block annotations	2026-08-15 17:18:28 -07:00
Waleed	852906ec91	feat(splunk): add Splunk Enterprise and Cloud integration (#6743) ... Adds a Splunk block with 12 REST operations: run search (oneshot), create/get/cancel search job, get search results, list/get/dispatch saved searches, list/get fired alerts, list indexes, and list apps. Bearer-token or basic auth, with optional /servicesNS namespace scoping. Every tool was validated against the Splunk REST reference. Results use search/v2/jobs/{sid}/results because the v1 endpoint is deprecated and disabled from Splunk Enterprise 9.0.1. A half-specified namespace fills the missing node with the documented - wildcard rather than nobody/search, which would have hidden user-private objects. Dispatching endpoints fail loudly instead of reporting success with a null sid, and Create Search Job rejects exec_mode=oneshot since that mode returns results rather than a search ID. The results and control endpoints tolerate an empty body. saved/searches sends the f field filter the reference prescribes for it.	2026-08-15 17:07:59 -07:00
Waleed	d45dad7e8b	feat(okta): add System Log, MFA, sessions, apps, roles, and group rules (#6741) ... Expands the Okta block from 18 to 44 operations, covering the System Log, MFA factors, sessions, applications, administrator roles, and group rules. Adds shared helpers for the SSWS auth header, Okta error parsing, and the Link-header `after` cursor, and routes every tool through them so there is one auth and error path. All eight list operations now return `nextCursor` and `hasMore`. Makes the block's param transform authoritative over the serialized inputs: the executor merges it on top of them, so a key the transform omits keeps the raw subBlock string. Assigning `undefined` is what actually drops it, which is what keeps a non-numeric `limit` from reaching Okta verbatim and stops a blank field in a partial `update_user` from overwriting the stored value with an empty string.	2026-08-15 16:47:02 -07:00
5 people	337a53f12c	feat(cli): Sim CLI with AWS-style profiles and a platform key exchange (#6147) ... * improvement(api): pull in the v2 external endpoint surface Cherry-picks improvement/v2-endpoints (98c85677f5) onto the current base. The v2 surface standardizes one response family across every endpoint: `{ data }`, `{ data, nextCursor }`, and `{ error: { code, message, details? } }`, rendered through apps/sim/app/api/v2/lib/response.ts. v1 auth and rate limiting are reused as-is; the workspace-access and enterprise-audit checks are split into `resolve*` cores returning structured failures, with thin v1 wrappers that render the old `{ error }` body so v1 behavior is unchanged. The branch's own /api/v2/tables/** is dropped. Staging's tables v2 (#6067, typed predicate grammar + POST /api/v2/tables/[tableId]/query) supersedes it and lands in the following merge; the two are reconciled onto the shared envelope separately. Conflict resolutions: - v1/middleware.ts: keeps resolveWorkspaceRequestActor alongside the new resolveWorkspaceAccess/resolveWorkspaceScope split - v1/audit-logs/auth.ts: keeps the newer targetOrganizationId parameter and isOrganizationBillingBlocked check inside the structured resolver - bun.lock: taken from HEAD; the branch's lock churn was unrelated lucide-react hoisting Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * feat(usage): accept X-API-Key on usage-logs list + export /api/users/me/usage-logs and /export now use checkHybridAuth — the same auth /api/users/me/usage-limits already accepts — so external monitors can read summary.bySourceCredits (the source breakdown of usage-limits' aggregate currentPeriodCost) instead of estimating Copilot spend by subtraction. Workspace-scoped keys are pinned to their own workspace's slice of the ledger: the filter defaults to the key's workspace and an explicit mismatch 403s. Both endpoints documented in openapi-core.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(cli): sim CLI with AWS-style profiles and a platform key exchange Adds `packages/sim-cli` (`@sim/cli`, bin `sim`) and extends the existing CLI key handoff so it can mint the credential the public API actually accepts. ## Key exchange The handoff already existed but only minted *copilot* keys, which do not authenticate `/api/v1` or `/api/v2` — those want a Sim platform key. The approval now carries a `scope`: - `copilot` (the default, so terminals built against the original flow are unaffected) mints as before - `platform` mints a Sim API key: workspace-scoped when the approver is a workspace admin, personal otherwise Scope and workspace are fixed at *approval*, not at poll: the poll is unauthenticated by necessity, so the browser is the only moment a human is present to consent and the only place a permission can be checked. The poll echoes back what was granted rather than what was asked for, so the CLI cannot file a copilot key under a platform profile and fail later with an opaque 401. Picking a workspace and scoping a key to it are kept separate. The terminal has no key yet, so it cannot list workspaces — the browser picker is the only place that choice can be made, and the pick comes back as the profile's default whether or not the key is bound to it. Otherwise a non-admin would pick a workspace by name and then have to go find its id by hand. Personal-key creation moves into `lib/api-key/orchestration` so the settings route and the exchange share one issuer. ## CLI Profiles work like the AWS CLI: `~/.sim/config` for settings (`[profile dev]`), `~/.sim/credentials` for keys at 0600 (`[dev]`), selected via `--profile` / `SIM_PROFILE`. Each setting resolves flag → env → file → default, and `sim whoami` reports the winning source so a surprising value is explainable. CI can skip login entirely with `SIM_API_KEY` + `SIM_WORKSPACE`. Commands cover the v2 surface pulled in earlier: workflows, logs, files, and knowledge, with `--output json` passing the API's own shapes through for `jq`. `sim tables` is deliberately absent — that surface is still in flux. ## Drift fixes The v2 routes were authored a month ago and had fallen behind their services: `checkActorUsageLimits(userId, workspaceId)` → the billing-attribution flow (which also restores correct payer attribution for workspace keys on KB upload and search), `processDocumentsWithQueue` gained a required argument, and the deploy/rollback param objects had stale fields. Caught by a cold type-check — an incremental run had reported these files clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * feat(billing): dedicated v2 usage endpoints; keep internal usage routes session-only Replaces the earlier X-API-Key enablement on /api/users/me/usage-logs with a dedicated public surface, so the internal Billing-settings endpoints can evolve with the UI while external monitors get a stable versioned contract: - GET /api/v2/billing/usage — current-billing-period summary with bySourceCredits (the source breakdown external monitors need to watch e.g. Copilot consumption without estimating by subtraction), plus limitCredits and plan - GET /api/v2/billing/usage/logs — cursor-paged credit ledger in the v2 envelope - workspace-scoped keys are pinned to their own workspace's slice; personal keys read the account ledger The public wire is credits-only: usage-logs rows now carry a hasCost boolean instead of dollarCost (the Billing UI only needed the >0 signal), and the rateLimit block is removed from the usage-limits response and docs (deploy-modal tab relabeled accordingly). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(cli): generate the CLI's v2 API from the route contracts, add tables The same endpoint was being described in three hand-maintained places: the Zod contracts the routes validate against, the OpenAPI documents, and the CLI's own TypeScript interfaces. Two of those are now derived. ## Generation `scripts/generate-v2-cli-api.ts` reads `apps/sim/lib/api/contracts/v2/**` and emits `packages/sim-cli/src/generated/v2-api.ts`: request/response types for all 44 operations plus an operation table (method, path, path params) the client dispatches through, so a route that moves or changes verb moves the CLI with it. The contracts are the right source because the routes validate against them — a shape that disagrees with a contract is a shape the server would reject. Zod 4's `z.toJSONSchema()` handles all 110 schema slots; the JSON-Schema-to-TS emitter is hand-rolled over that known-narrow subset and throws on anything unrecognized rather than degrading to `any`, since silence is how a generated client drifts. `packages/*` must not import `apps/*`, so the generated file is plain type declarations with no imports and the script does the crossing at build time. `check:cli-api` fails CI when the file is stale. The generated directory is excluded from biome: the pre-commit hook runs `check --write`, which would otherwise reformat generated output and fail that check with an unrelated message. ## OpenAPI: checked, not generated The docs specs carry ~1000 hand-written descriptions and ~400 examples that Zod schemas do not encode, so generating them would trade real documentation for mechanical accuracy. `check:openapi-drift` reconciles structure instead — every v2 path and method must exist on both sides — keeping the prose while still failing on divergence. Both currently agree on all 44 operations. ## Tables `sim tables list|get|columns|rows|insert|delete-rows`, built on the generated types. Rows go through the POST query endpoint even unfiltered, since it is the only shape carrying the predicate. Row columns are discovered at runtime and unioned across the page, so a sparse row cannot hide a column. Deletion requires an explicit `--row`/`--filter` selector *and* `--yes`; an argument-less call would otherwise empty the table. Path params are percent-encoded — an id containing `/` or `?` would otherwise retarget the request. The four existing command groups drop their hand-written interfaces for the generated ones. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * fix(cli): make the generated v2 API a fixed point of the formatter The pre-commit hook rewrote the generated file immediately after it was committed, so `check:cli-api` then failed in CI reporting contract drift that had not happened — the only difference was quote style. The biome.json exclusion added alongside it does not help: lint-staged runs `biome check --write` on explicit paths, which bypasses `files.includes`. It implied protection it never provided, so it is removed. The generator now pipes its output through `biome format --stdin-file-path` instead, making the emitted file conformant by construction. The hook has nothing left to change, and the check compares like with like. A formatter failure throws rather than emitting unformatted output, since falling back silently would reopen the same loop. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * fix(cli-auth): wait for the workspace list before allowing approval The picker fell back to "No workspace (personal key)" while the workspace query was in flight, and Connect stayed live through that window. A fast click approved a personal key with no default workspace — when the same click a moment later would have issued a workspace-scoped key. The fallback read as an answer rather than a pending state, so the card could promise one outcome and deliver another. Connect is now disabled until the list resolves, the trigger shows a loading label (a placeholder would not show, since the fallback always counts as a selection), and the explanatory line no longer asserts the personal-key outcome before it is known. Failure is treated as degraded rather than fatal: the picker disables but Connect stays enabled and the copy says a personal key will be issued, so a transient list failure cannot strand a waiting terminal. Tests cover the pending, loaded, admin-binding, and error states; the two loading assertions fail against the previous implementation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * fix(cli-auth): name minted keys by timestamp, not date A second login on the same day failed with `A workspace API key named "CLI (2026-07-30)" already exists` — after the user had already approved in the browser, so the whole handoff was wasted and there was no way to complete it without renaming the existing key. Key names are unique per owner, so the name has to be unique per login. Now `CLI (2026-07-30 15:42:07Z)`: second precision, UTC so it is unambiguous in a shared workspace key list and sorts chronologically. The comment claiming a same-day collision was desirable (so logins would reuse one key) was wrong — nothing reuses the key, the mint just fails. A collision at second precision now means something genuinely unexpected, so it is still surfaced rather than retried under a suffixed name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * feat(docs): validate OpenAPI specs against the Zod contracts in CI The specs in apps/docs are hand-authored because they carry what Zod never defines — error envelopes, status codes, prose, examples — so they can't be generated; check:openapi validates them instead: - spec integrity: $refs resolve, operationIds unique, 2xx documented, no orphaned component schemas - v2 conventions: every /api/v2 operation documents 401 + 429 and every 4xx/5xx resolves to the canonical { error: { code, message } } envelope - contract cross-check: contracts are auto-discovered from lib/api/contracts/v2 (each carries its method + path); doc<->contract coverage both ways, query/body/response field diffs via z.toJSONSchema - examples: documented request/response examples must parse with the matching contract's actual Zod schemas First run caught real drift, fixed here: 16 stale orphaned schemas in the core spec, the v2 billing ops referencing v1-shaped error components, deploy/rollback examples missing the required nullable lifecycle keys, CreateTableBody missing folderId, a legacy-grammar delete-rows example, and four knowledge document ops missing their required workspaceId query param. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(docs): recursive field diff in check:openapi + the deep drift it found A mutation test showed the doc<->contract field diff only compared top-level properties, so a typo inside the { data } envelope passed. The diff now descends through matching object properties and array items (both sides must expose a property set — passthrough contracts and prose-only docs end the descent instead of false-positive), with the Zod JSON-schema root doubling as the $defs context. Deep drift it immediately caught, fixed here: select-column config (options/multiple) missing from every tables column schema, AddColumnBody hand-rolling a third column shape (now composed from ColumnInput, with position/workflowGroupId as the per-op extensions the contracts actually admit), chunking strategyOptions undocumented, and the deployment lifecycle fields (activeDeployment/latestDeploymentAttempt) missing from DeploymentState. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(security): close the triggerType rate-limit bypass on workflow execute Caller-supplied triggerType flowed unchecked into preprocessExecution, whose checkRateLimit default turns OFF for 'manual'/'chat' — so any API-key caller, and any anonymous public-API caller billed to the workspace owner, could execute unthrottled by sending {"triggerType":"manual"} (async runs also skipped the worker-side check via admissionCompleted). External callers may now only send the redundant 'api' value; internal JWT callers ('workflow'/'mcp') are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * refactor(execution): extract enqueue/status/cancel into shared libs Prepares the v2 execution surface: handleAsyncExecution's queue logic moves to lib/workflows/executor/enqueue-execution.ts (slot/claim semantics encoded in a discriminated outcome, not HTTP statuses), the execution-status read to execution-status.ts, and the order-sensitive cancel machinery to lib/execution/cancel-workflow-execution.ts. The v1 routes re-render identically — their suites pass unmodified. Also: preprocessExecution gains rateLimitCounter ('sync'|'async') and its 429 now carries code RATE_LIMIT_EXCEEDED + retryAfterMs (previously indistinguishable from the concurrency 429 and Retry-After was discarded); and the duplicate cancel contract in contracts/logs.ts is unified on the full 5-value reason enum — its narrower copy made requestJson throw a client ZodError when cancelling a paused HITL run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(execution): callable execution service + structured error classifier executeWorkflowService composes the same libs the v1 route holds inline (call-chain guard, execution-id claim, LoggingSession, preprocessing, deployed-state load + file-field processing, timeout-bound executeWorkflowCore, output hydration/compaction) for the deployed-state caller class — the seam the v2 execute route and in-process internal callers share, making the HTTP endpoint syntactic sugar. classifyExecutionError stops discarding the block context that buildBlockExecutionError already attaches at throw sites: failed runs now yield {message, code, blockId, blockName, blockType} with a stable append-only code enum (TIMEOUT/CANCELLED/USAGE_LIMIT_EXCEEDED/ INVALID_INPUT/BLOCK_EXECUTION_FAILED/CHILD_WORKFLOW_FAILED/ OUTPUT_TOO_LARGE/EXECUTION_FAILED), so callers route on error class instead of substring-matching messages — the single place raw errors are interpreted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): POST /api/v2/workflows/[id]/execute Thin route over executeWorkflowService: X-API-Key or anonymous public-API auth (sync/stream only for anonymous), strict body with body-flag async (no mode headers on v2), SSE passthrough for stream, and the execution resource response — executionId always present, in-band run failures are status:'failed' with the structured {message, code, blockId, blockName, blockType} error, sync timeout is status:'failed' + TIMEOUT instead of v1's 408, and a Response block's payload stays inside output (authors never control response status/headers on this origin). Async debits the async bucket and the 202 statusUrl points at the v2 executions resource. Adds CLIENT_CLOSED_REQUEST/SERVICE_UNAVAILABLE to the v2 error codes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): v2 executions status + cancel with queued backfill GET /api/v2/workflows/[id]/executions/[executionId] is the single status URL for sync and async runs: before the async worker writes the durable log row, status is backfilled from the job queue (deterministic job id) as 'queued'/'running' — closing v1's 202-to-pickup 404 window — and failed runs carry the structured error object. POST .../cancel renders the shared cancellation lib in the v2 envelope with the tightened 5-value reason enum. Both authenticate via the shared resolveV2WorkflowAccess (X-API-Key, authz masked as 404, allowPersonalApiKeys honored). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(execution): workflow tool + MCP bridge run in-process workflow_executor (workflow-as-agent-tool) short-circuits in executeTool through WorkflowBlockHandler — the same invocation boundary canvas child workflows use — mirroring the deployed_block_executor precedent. The MCP serve bridge calls executeWorkflowService directly instead of fetching its own execute endpoint; deployment-version pinning, MCP response-size rejection, and the actor override become typed options instead of header sniffing. Both callers drop the double admission slot and duplicate top-level log row the HTTP hop cost, and failed child runs now surface the structured error + child executionId so parents and MCP clients can route on error class and hand providers a reproducible handle. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(infra): CORS + CSP coverage for the v2 execute path /api/v2/workflows/:id/execute gets the same wildcard-origin, credential-free CORS policy as v1 (the default credentialed policy would block browser API-key calls and open a cookie CSRF surface) with X-Sim-Stream-Protocol allowed and no X-Execution-Mode (async is body-selected on v2), plus the COEP/COOP/CSP header block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(ui): deploy modal + copilot advertise the v2 execute surface All 20 API-tab snippets move to POST /api/v2/workflows/{id}/execute with the nested {"input": ...} body, async as the "async": true body flag (X-Execution-Mode gone), status polling against the v2 executions resource, the third tab renamed Usage and pointed at /api/v2/billing/usage, and {data} envelope unwraps in the printed responses. Fixes the latent baseUrl derivation (endpoint.split('/api/workflows/')) that would have silently built garbage URLs under a v2 endpoint, and deletes dead code (exampleCommand across 3 sites, getAsyncExampleTitle). Copilot deploy/manage/serializer endpoint builders and the api_trigger bestPractices example follow (the latter also drops its hardcoded staging host). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * docs(api): document the v2 execution surface Adds execute, execution status, and cancel to openapi-v2-workflows.json with the structured ExecutionError schema (append-only code enum + block attribution) and the ExecutionResource contract, documenting the rules that differ from v1: modes are body-selected, a failed run is HTTP 200 with status 'failed', an executionId always means data (never the error envelope), queued status is visible immediately, and Response-block payloads stay inside output. Registers the three pages in the generated workflows meta.json and bumps the route-count baseline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): gate the whole /api/v2 surface behind one flag; UI stays on v1 Every v2 route now runs exactly one check immediately after auth — v2ApiGateError — and answers 404 when the `v2-api` flag is off, so the surface is invisible until it is deliberately rolled out. The gate is keyed on userId only: a workspace/org-keyed check would have to read membership for a caller-supplied id before authorization runs, and its 404-vs-403 split would leak cohort membership (the trap the per-domain table gate worked around by running late). The two executions routes inherit it from the shared access resolver; the tables-specific gate is removed so no route checks twice. `tables-v2-api` stays, now gating only the internal predicate-grammar route /api/table/[tableId]/query — note v2 tables routes move to the unified flag, so enabling them is a `v2-api` decision now. Reverts the deploy modal, copilot handlers, and api_trigger example to the v1 execute endpoint: v1 works unchanged, and the UI must not advertise a surface most users would get a 404 from. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(cli): CLI contract for the v2 surface, incl. execution Adds `packages/sim-cli/src/contract` — the declarative definition of how the terminal maps onto the API — and folds in the v2 execution endpoints that just landed on improvement/v2-endpoints. ## The contract Read it as a diff against what is already derivable, not a listing. Method, path, path params, field types, enum values, defaults and required-ness all come from the generated operation table (which comes from the Zod contracts), and the command name derives from `<resource> [sub-resource] <verb>`. 23 of 47 operations therefore need no entry at all. The 24 that do carry only what a schema cannot express: - names, where REST overloads one path — `DELETE /rows` vs `DELETE /rows/[rowId]` becomes `batch-delete` vs `delete`, and `DELETE /deploy` becomes `undeploy` - flags, where a field's type misdescribes its meaning — `workflowIds` is `z.string()` that the route splits on commas; no generator can infer that - columns, which are editorial - confirm, for the 8 destructive operations ## Execution `executeWorkflow` / `getWorkflowExecution` / `cancelWorkflowExecution` derive badly (`/execute` and `/cancel` are verbs the deriver reads as nouns), so all three are named explicitly: `workflows run`, `workflows executions get|cancel`. `stream` is marked `omit`: it switches the response to SSE, which the JSON client would try to parse. Advertising a flag that breaks the response is worse than not offering it — a `--follow` command that renders the stream is separate and hand-written, like `files download`. ## Also - Drops `check:openapi-drift`. The branch landed `check:openapi`, which does the same path/method reconciliation plus a recursive field diff and validates doc examples against the real Zod schemas — mine was a strict subset. - Surfaces the new v2 rollout gate in the CLI: it answers 404 for callers outside the cohort, indistinguishable from a missing resource, so a 404 now carries that as a possibility rather than a diagnosis. - `executor/utils/errors.ts` widens instead of casting through `unknown`, which is both more honest (the value is an Error) and keeps the double-cast ratchet at 8. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * fix(executor): restore child-cost aggregation dropped by the staging merge Staging's custom-block rewrite deleted `aggregateChildCost` from workflow-handler.ts, and git merged that file cleanly — but this branch's workflow-tool-runner.ts, added for the v2 execute migration, still imports it. A silent semantic conflict: no marker, broken build. Taking staging's rewrite is correct, so the helper is defined locally in its one remaining consumer rather than resurrected in the file staging just rewrote. Same four lines over the still-exported `calculateCostSummary`, so a failed child workflow keeps billing the hosted-key spend it consumed instead of reporting $0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(cli): yaml and text output formats `--output` now takes table | json | yaml | text, settable per-command, via SIM_OUTPUT, or persisted per profile as before. `yaml` joins `json` in rendering the API's raw values rather than the table's formatted cells, so a duration stays `1500` instead of becoming `"1.5s"` — switching format changes the encoding, never the data. Line folding is disabled: valid YAML, but it breaks line-oriented greps and is miserable to read. `text` is tab-separated with no header and no colour — the shape `cut -f2` and `while IFS=$'\t' read` expect, so shell plumbing works on a box with no JSON tool. It uses the rendered cells rather than raw values, since it is a human-ish format for pipelines rather than something to parse. An absent value collapses to an empty field instead of the table's em-dash: `cut` returning a literal `—` would read as a value to every downstream emptiness test. A bad `--output` is now an error (commander `.choices`) rather than a silent fall back to `table`. The environment variable and the config file stay tolerant — those are ambient and set once, so a bad value should not break every command, but a flag just typed should not be quietly disregarded. Uses js-yaml 4.3.0, already a direct dependency of apps/sim, rather than adding a second YAML library to the monorepo. Also drops a stale README reference to check:openapi-drift, which the v2-endpoints merge superseded with the deeper check:openapi. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * refactor(tables): make lib/table/orchestration the single implementation (#6134) * refactor(orchestration): move the shared error contract out of lib/workflows OrchestrationErrorCode and statusForOrchestrationError are the contract every lib/[resource]/orchestration module returns against, but they lived inside the workflows module, so resource-neutral code (lib/folders) already had to import from a workflow path. Moved to lib/core/orchestration/types. Adds a 'locked' class mapping to 423. Both tables and workflows have a lock that forbids a mutation, and each caller was translating that to a status itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): make lib/table/orchestration the single implementation Column update was implemented four times — the UI route, v1, v2, and the copilot table tool — each calling the same column services but owning its own guards, error mapping, and audit. The copies had drifted, and the drift was the bug: v2 was missing both guards, only the copilot copy minted stable option ids, and only v1/v2 audited. performUpdateTableColumn, performDeleteTable, and performDeleteTableRow now own that logic; all ten call sites reduce to auth, parse, call, render. The guards are asserted once in lib/table/orchestration rather than four times against four routes. Behavior this consolidates, previously true on only some paths: - The typeChanging guard. updateColumnType early-returns on an unchanged type and drops any options sent with it, so restating the current type alongside new options silently discarded them. v2 had no guard at all and, since its contract shares v1's body schema, accepted options and ignored them. - The select-unique guard. Each write is its own locked transaction, so a rename or type change paired with a constraint write that is going to fail commits first and then throws, half-applying the schema change. - Stable select-option ids. Cells reference the option id, so an edit that re-sends an option by name has to reuse it or every cell holding it is orphaned. Only the copilot path did this; normalizeSelectOptionsInput moves to lib/table/select-options and now covers every caller. It preserves a supplied id, so it is a no-op for the fully-formed options the HTTP contracts accept. - required forwarded into the type and options writes, so a conversion validates against the constraint the same request is setting. - An audit on every successful update. The UI route and the copilot tool emitted none. - Single-row delete through the row service. v2 did a raw db.delete, skipping assertRowDelete and deleteOrderedRow, so a delete-locked table returned 200 and the row-count bookkeeping never ran. - The delete actor handed to deleteTable, which audits only when a row was actually archived. v1 and v2 omitted it and audited themselves outside that check, emitting TABLE_DELETED for a no-op delete of an archived table. Failure classes come back as OrchestrationErrorCode; v2 renders them through a new v2ErrorForOrchestration, mirroring statusForOrchestrationError on the v1 and UI surfaces, so a given failure maps to the same status everywhere. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(tables): bind the column-update tests to the orchestration function The base's route tests assert which column service each payload reaches — the behavior that now lives in performUpdateTableColumn. They mocked the `@/lib/table` barrel; the orchestration module imports the service directly, so they mock that too and keep asserting the same thing through the extracted implementation. The orchestration tests move onto the base's semantics: writes address the stable column id, a rename rides inside the write it accompanies rather than running first, and the currency guards replace the non-select options guard the service now owns. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(copilot): drop the column-type import the delegation made dead Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): move the audit log out of the table service `lib/table/service.ts` wrote its own audit rows, so whether an operation was audited depended on which function a caller reached for rather than on a user having performed it. That is what let v1 and v2 audit a no-op delete, and what made `deleteTable`'s optional `actingUserId` double as an audit opt-out flag. Worse, most sites fell back to `actingUserId ?? createdBy`, so an unattributed call was logged against the table's *creator*. The copilot `mv` path passed no actor at all: renaming someone else's table recorded them as the renamer. Audit now lives in the orchestration functions — performDeleteTable, performRenameTable, performMoveTableToFolder, performUpdateTableLocks — and the services just write. Internal callers (folder cascade, import rollback) keep calling the service and are silent by construction rather than by remembering to omit an argument. Two services now return what the audit needs: `deleteTable` reports whether it actually archived a row, so a repeat delete logs nothing; `updateTableLocks` returns the before/after locks, since only the locked write can observe the transition its description names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(tables): restore audit provenance and conflict status in orchestration Moving the audits into the orchestration functions dropped three things the routes had been carrying, and added one the orchestration now owns twice. - The v1 and v2 column-update routes passed `request` to `recordAudit`, so their audit rows recorded the caller's IP and user-agent. The orchestration function had no way to receive it. Every table orchestration function now takes an optional `OrchestrationRequestContext` and every HTTP route forwards it; the copilot and VFS callers, which have no request, omit it. - `classifyTableMutation` matched `TableConflictError` on "already exists" appearing in the message and reported it as `validation`, turning the UI route's 409 on a duplicate table rename into a 400. It now matches the type, the way `performRestoreTable` already did. - `captureServerEvent` ran on every delete while the audit was gated on a row actually being archived, so a repeat delete of an archived table still reported `table_deleted`. Both now hang off the same evidence. - The copilot delete path kept its own `captureServerEvent` from when the service did not emit one, double-counting every copilot table delete. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a * fix(tables): say which type a no-op column update restated A copilot `update_column` payload whose only content was the column's current type used to return success with the live schema, while the v1, v2, and UI routes rejected the same payload with "No updates specified". Delegating to `performUpdateTableColumn` unified them onto the routes' rejection — correct, but the message tells the caller its request was empty when it named a type. The orchestration function now reports the same thing `updateColumnType` reports when it loses this race concurrently: the column is already that type, re-issue without the type change. An empty payload still reads "No updates specified". Drops the copilot's `outcome.table ?? tableForUpdate` fallback with it — the comment described the no-op that can no longer reach that line, and a success always carries a table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a * refactor(tables): classify failures by type instead of by message text The table module decided HTTP statuses by searching error messages for phrases. `VALIDATION_MESSAGE_FRAGMENTS` and `ROW_WRITE_ERROR_PATTERNS` held 32 substrings between them, and fifteen more lists were inlined in routes — 83 matchers over 17 files, each its own copy of the guesswork and already drifted apart. It made message wording load-bearing: `TableRowLimitError`'s own doc comment noted that its text had to contain "row limit" for a route to answer 400, and adding "already exists" to a rename message silently demoted a 409 to a 400 (the bug fixed one commit ago, by adding another special case). Services now throw `OrchestrationError`, which carries the transport-neutral `OrchestrationErrorCode` the layers above already speak. Classification is one `instanceof` in `orchestrationErrorResponse` (UI + v1) and `v2CaughtOrchestrationError` (v2). Every pattern list is gone. Wording is free to change; an unclassified error still becomes a generic 500, which is what an unexpected fault should be. `asOrchestrationError` walks the `cause` chain rather than testing the caught value directly: drizzle wraps a throw raised inside a transaction callback in a `DrizzleQueryError` whose own message is the failed SQL, so a bare `instanceof` would drop every failure raised inside `withLockedTable`. That is the same reason `rootErrorMessage` had to dig for a root cause before. Three throws stay bare `Error` deliberately — `Table ID mismatch`, `Workspace ID mismatch`, and `Failed to build upsert conflict predicate` are internal invariants no consumer classified, and they keep falling through to a 500. `Insufficient capacity` was in the pattern list with no producer anywhere in the codebase. Status changes, all deliberate: - `'forbidden'` joins the code union so the table-row-limit ceiling keeps its 403; without it this refactor would have flattened it to 400. - import-async's table-limit rejection: 400 -> 403, matching the two other create routes it had drifted from. - Renaming a table to an invalid name: 500 -> 400. `validateTableName` messages don't contain "Invalid", so no matcher ever caught them. - Restoring a table that isn't archived, or into an archived workspace: 500 -> 400. - A duplicate *column* name stays `validation`/400 rather than becoming a 409 like a duplicate table name. Both v1 and the orchestration have always answered 400 for it; changing a published status is not this refactor's job. The twelve tests that changed were asserting the substring mechanism itself, constructing plain `Error`s with magic strings. They now assert the real contract, plus new cases pinning that identical wording carrying no classification stays internal and keeps its message off the wire. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * refactor(cli): output format is a profile setting, not a flag Drops `-o, --output`. Format is set once per profile with `sim configure --set-output <format>`, or overridden ambiently with SIM_OUTPUT for a one-off (`SIM_OUTPUT=json sim logs list | jq`) and for CI, which already runs file-less on env alone. Both remaining sources are ambient — set once, then read by every later command — so an unrecognized value falls back to `table` rather than breaking the CLI. There is no longer a strict tier, because there is no longer anything typed per-invocation to be strict about. Frees `-o` for `sim files download -o <path>`, which previously had to share the short flag with a global that meant something else entirely. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * feat(cli): runtime that builds every command from the contract Turns the CLI contract into working commands. 43 leaves across 7 groups, up from the 6 hand-written ones — every v2 operation the contract does not hide is now reachable, including `sim tables upsert`, `sim workflows run`, and the whole tables surface. ## What the generator now emits `V2_OPERATIONS` carries a field→slot map per operation: each query/body field's kind, whether it is required, its enum values, and its server-side default. Types alone could not drive this — the runtime has to *iterate* fields to build flags, and everything from argv arrives as a string, so it needs the kind to turn "50" into 50 and '{"a":1}' into an object. It also lifts each operation's one-line `summary` from the OpenAPI specs. The contracts carry validation, not prose, so `--help` had been showing raw URLs; the specs already hold a written summary per operation and `check:openapi` guarantees one exists, so this reuses documentation rather than inventing a second place to describe the same endpoint. ## The runtime `derive.ts` names a command `<resource> [sub-resource] <verb>` from the route, covering 41 of 47. `request.ts` assembles the call: path params from positional args, `workspaceId` injected from the profile into whichever slot declares it, everything else coerced and validated locally — so a bad enum, malformed JSON, missing required flag, or absent workspace fails before any network call. `build.ts` constructs the commander tree, auto-pages cursor lists up to `--limit` (0 for everything), and renders through the contract's columns or, for runtime-shaped rows, keys unioned across the page. Fixed while wiring: `new Command('upsert <tableId>')` makes the *whole string* the command name, so `sim tables upsert` never matched and fell through to the group's help. Arguments have to be declared with `.argument()`. ## What stays hand-written Two leaves, each for a reason generation cannot satisfy in principle: `files download` streams binary rather than the JSON envelope, and `tables rows list` discovers columns from user-defined row data nested under `data`. They attach onto the generated groups, so `sim files --help` lists them alongside the rest. The five previous command files are deleted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * fix(cli): review round 1 — flag lookup, terminal controls, download safety ## CLI flags silently dropped (Cursor, High) Commander camelCases every multi-word flag, so `--min-duration-ms` is stored as `minDurationMs`. `buildRequest` looked flags up by their own kebab name, found nothing, and dropped the field — no error, it just never reached the API. That was every multi-word flag on every generated command. The unit tests passed because they fed flag values already keyed by flag name, which is not what commander produces — they validated a fiction. Added `build.test.ts`, which parses real argv through the built commands; three of its assertions fail against the previous code. The old tests now use camelCase keys with a comment saying why. ## Terminal control sequences (Greptile, P1 security) `stripAnsi` matched only SGR (`ESC [ … m`), so a knowledge document, table cell, or workflow name could carry OSC, non-SGR CSI, or `ESC c` through to an interactive terminal — setting the window title, moving the cursor to overwrite what was already printed, or resetting the terminal. Replaced with a `sanitize` covering OSC (BEL- and ST-terminated), CSI, any ESC + printable, and the bare C0/C1 range, keeping tab and newline. Applied where API values become display text, so the colour the CLI adds afterwards still works. ## Downloads (Greptile, P1 ×2) `createWriteStream` truncated silently, and the destination name usually comes from the server's content-disposition rather than anything the caller typed — so a download could irreversibly replace an unrelated local file. Now opens `wx` and fails with a message naming `--force`, which was added for the deliberate overwrite. The stream's error listener was attached after the read loop finished, so an EEXIST/EACCES/ENOSPC during writing was an unhandled 'error' event that took down the process. It is now registered before the first write and raced against the pump. ## Personal-key caption (Cursor, Low) With "No workspace (personal key)" picked, the caption still promised a default workspace the approval does not send. It now distinguishes no-pick from picked-but-not-admin. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * fix(cli): review round 2 — body-cursor paging, timestamp sanitization ## `tables rows query` printed nothing (Cursor, High) `isCursorList` only looked for `cursor` on the query slot, but `queryRows` is a POST whose whole filter — cursor included — is in the body. It therefore took the single-request path, which handed an array of rows to `printRecord` and printed an empty record, and it never auto-paged past the first page. Replaced with `cursorSlot`, which checks both slots and tells the pager where to put the cursor back. Added a defensive branch so an array reaching the single-resource path renders as a list with inferred columns rather than silently printing nothing. ## Invalid timestamps bypassed sanitization (Greptile, P1 security) `timestamp()` echoes an unparseable value verbatim, and that value is still server-supplied — so the branch was a way past every other formatter for the control sequences round 1 closed. Now sanitized on that path too. Audited the remaining formatters: no other path returns a server value unsanitized. Both fixes have tests that fail against the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * fix(cli): review round 3 — poll retry, download flush errors Both findings are flaws in round 1's fixes rather than in the original code. ## A redeemable login was thrown away (Cursor, High) `pollForKey` treated every non-429 status as terminal. But the poll route releases its mint reservation on any mint failure — its own comment says "a later poll can retry" — so a transient 5xx or a same-second name conflict ended the login after the user had already approved in the browser, forcing a full restart for something the server had deliberately left recoverable. Retryable is now 409, 429, and 5xx. Everything else stays terminal: 400 means a malformed request id or verifier and 401/403/404 mean the server is refusing on purpose, so retrying those would just spin to the 15-minute timeout. ## A failed download reported success (Greptile, P1) `file.end(resolve)` passes the flush error to the callback as its argument, so the pump fulfilled *with* the error and the command printed "Saved" for a truncated file. Confirmed against node directly — `end`'s callback receives the errno. It now rejects on that argument, which is the path an ENOSPC actually takes, since the bytes may not reach disk until the final flush. Adds `device-flow.test.ts` (11 tests: the retry matrix, transport failure, terminal refusals, and that the poll secret never enters the browser URL) and `hand-written.test.ts` covering the download's overwrite guard and flush failure. The two retry tests fail against the previous code; the flush test needs `/dev/full` and so runs in CI rather than on macOS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * fix(cli): review round 4 — repeated flags encode per field kind `coerce` comma-joined every `list` flag, but that is only correct for the three fields whose wire type is a `string` the route splits (`workflowIds`, `folderIds`, `triggers`). The others genuinely want an array: - `rowIds` and `selectedOutputs` are `array`, so joining sent a string where the schema expects a list — `sim tables rows batch-delete --row a b` failed validation, and so did a single `--row a` - `knowledgeBaseIds` is a string-or-array union whose array branch is the right one; joining made `kb_1,kb_2` a single bogus id, so multi-`--kb` search silently searched nothing `list` now means only "accept the flag more than once" — the encoding follows the field's kind, which the generator already records. The two questions were conflated under one contract field and the `FlagSpec` doc now says so. Four tests, three of which fail against the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * fix(cli): review round 5 — header sanitization, auth ordering, stale suggestion ## Table headers stayed executable (Greptile, P1 security) Round 1 sanitized cell *values* but not the column *names*, and a table's columns are user-defined — so the same control sequences were still executable one row higher, in the header. Sanitizing is now done inside `renderTable` rather than at each call site, so a future column source cannot reopen it, with the two key-derived column builders covered as well. ## Fresh install was told the wrong first step (Cursor, Low) Generated commands read `profile.workspaceId` directly, bypassing `requireWorkspace()` — which checks the key first precisely so a new user is told to log in rather than to set a workspace they cannot use yet. That ordering was fixed for the hand-written commands earlier and reintroduced by the runtime. `sim tables list` on an empty profile now says "Not logged in" again. ## A stale suggestion shadowed the fallback (Cursor, Medium) The picker took `selected ?? suggestedWorkspaceId ?? lastActiveWorkspaceId`. The suggestion comes from a profile the CLI wrote earlier, so it can name a workspace the user has since left — and merely being truthy, it blocked the last-active fallback and left the card on "no workspace" with a perfectly good one available. It now counts only when it resolves against the loaded list. Two of the three have tests that fail against the previous code; the third is verified end-to-end (`sim tables list` on an empty profile). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials (#6150) * feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials * fix(api): correct credential role, skill permission bar, MCP url identity, and custom-tool conflict mapping * fix(api): align credential mutation gating, provider-outage status, and unique-violation conflicts * fix(api): close unique-violation, revival, orphan-write, and env-rename gaps * fix(api): treat every provider-outage code as unavailable on create and update * fix(credentials): use the shared outage predicate on the session update path * fix(contracts): anchor the predicate double-cast annotation to the cast `check:api-validation:strict` counted 9 unannotated double-casts against a baseline of 8, failing CI. The predicate leaf schema was annotated, but the annotation sat above the declaration while the checker anchors on the line carrying the cast — five lines below, at the close of the object literal. The scanner walks back at most three lines and stops at the first non-comment one, so it hit `value: z.unknown().optional(),` and never saw the reason. Splitting the object schema from the cast puts them adjacent, so the existing reason binds. No behavior change — the cast, the schema, and the reasoning are unchanged. Also lowers the rawJsonReads ratchet 6 -> 5 to match the current count, which had drifted down; leaving it high lets a removed raw read silently come back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(skills): point the orchestration error contract at its moved module #6150 branched before #6134, so skill-lifecycle.ts imports @/lib/workflows/orchestration/types — the module #6134 moved to @/lib/core/orchestration/types. Git merged a file deletion on one side with a new file referencing it on the other: no textual conflict, broken build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(cli): pick up the new v2 domains; discover modules instead of listing them Merges `v2-api-spec` (#6150 — v2 endpoints for MCP servers, skills, custom tools, folders, credentials) and the newer `improvement/v2-endpoints`. ## The generator was list-driven, so none of it would have appeared `DOMAINS` and `SPEC_FILES` were hardcoded. Five new contract modules and a new `openapi-v2-resources.json` had landed, and the generator would have skipped every one — silently, with `--check` still passing, because the generated file matched a generator that never looked. Both are now discovered from disk. That is the same silent-drop class the review rounds kept surfacing, and it is the property the whole pipeline rests on: a new v2 domain should reach the CLI by regenerating, not by remembering to edit a list. Result: 47 → 72 operations, 13 contract modules, and 25 new commands (`sim skills list`, `sim mcp-servers get`, `sim folders delete`, …) with no CLI change beyond the discovery fix. Summaries for the new domains now resolve too, so their `--help` reads properly instead of falling back to `METHOD /path`. ## Confirmation gates for the new destructive operations Five new DELETEs arrived ungated. `deleteFolder` is the sharpest — the route archives the folder *and cascades to its contents* — so its message says so rather than reading like a single-item removal. Added a test asserting every DELETE carries a confirmation, with `undeployWorkflow` the one documented exception (reversible by redeploying). It fails against this commit's own starting state, so the next domain to arrive cannot land ungated the way these did. ## One fix outside the CLI `lib/skills/orchestration/skill-lifecycle.ts`, added by #6150, imports `OrchestrationErrorCode` from `@/lib/workflows/orchestration/types`, which does not exist — the type lives in `@/lib/core/orchestration/types`, where every other consumer reads it. The branch does not type-check without this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EsThUPqZXjwuuyRjBbmVkj * fix(cli): render single-key resource envelopes, and column the new domains `sim mcp-servers create` created the server, exited 0, and printed nothing. The v2 route answers `{ data: { mcpServer: {...} } }`, and the record renderer keeps only scalar fields — one key holding an object left it with none. Unwrap a lone object-valued key before rendering; a payload with siblings (`{ row, operation }` from upsert) is a real result and is left alone. The five domains that arrived with the last generation had no contract columns, so `mcp-servers list` inferred 20 including `hasOauthClientSecret`. Give each a column set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * refactor(knowledge): make lib/knowledge/orchestration the single implementation (#6154) * refactor(knowledge): make lib/knowledge/orchestration the single implementation Knowledge base create was implemented four times — the internal route, v1, v2, and the copilot tool — and the orchestration around the shared write had drifted. Extract it the same way lib/table/orchestration was: services write, orchestration decides which writes run, guards them, audits them, and returns a transport-neutral failure. Behavior converged, not preserved: - One chunking default (DEFAULT_CHUNKING_CONFIG). The agent defaulted minSize to 1 against the API's 100, so identical input produced differently-chunked knowledge bases depending on who created it. The agent path now chunks at 100. - Every successful mutation is audited inside the orchestration function. The copilot tool called recordAudit zero times, so agent-created knowledge bases, document uploads, updates and deletes left no audit trail at all. - Failures classify by class, not by message text. The knowledge service errors are OrchestrationError subclasses and storage-quota rejections throw a shared StorageLimitExceededError, replacing four separate message greps for "already exists" / "does not have permission" / "storage limit". delete_connector reported the opposite of what happened. It reached the route through an internal HTTP self-call that sent no query string, so the route's keep-documents default always applied while the agent told the user the documents had been removed. The self-call is gone — all four connector operations run in-process — and the orchestration returns the real counts. Also: - OrchestrationErrorCode gains 'payload_too_large' (413 / PAYLOAD_TOO_LARGE). Without it, dropping the storage-limit message match would have regressed the documented 413 on knowledge base create and document upload to a 500. - messageForOrchestrationError renders a route's own wording for an unclassified fault, so a driver's message no longer reaches the client on a 500. - v1 and v2 knowledge base update now forward actorUserId, which the service requires for a workspace move; both omitted it. - The connector DELETE route reads deleteDocuments through parseRequest. Its contract declared z.boolean(), which would have rejected the string a query param actually is. - Drop the 409 from POST /api/v2/knowledge/{id}/documents in the OpenAPI spec. Nothing on the upload path throws a conflict; it was only ever reachable by the message match this change removes. Behavior change worth noting: a v1/v2 PUT carrying only the workspaceId scope field and no actual updates now returns 400 rather than 200 with the unchanged knowledge base. Deliberately deferred: document update remains internal-only. Extracting performUpdateKnowledgeDocument makes exposing it on v1/v2 a contract and a route away, but that is a new public surface rather than part of this consolidation. * fix(knowledge): make connector create atomic and stop flattening failures Review round 1 on #6154. - Resolve the billing payer before the connector is committed, not after. A malformed attribution header rejected post-commit left a live connector behind a 500, and a retry created a duplicate plus duplicate sync work. Manual sync resolves before writing its audit for the same reason. - Let the source-config validator carry its own failure class. Collapsing every rejection to `validation` flattened the connector PATCH route's 401 (stale stored credential) and 409 (missing workspace context) into a 400. - Add `unauthorized` to OrchestrationErrorCode. It is the class that 401 was already expressing on this route, and the v2 vocabulary already had UNAUTHORIZED; only the shared union was missing it. - Report a knowledge base that exists but failed to archive as failed, with the reason, rather than as not found. The copilot delete loop folded every non-not-found failure into `notFound`, telling the user it was never there. - Route copilot failures through the same message helper the HTTP surfaces use, so an unclassified fault's raw text (a driver's failed SQL) no longer reaches the agent verbatim while the UI and public APIs get the generic wording. * fix(cli): stop dropping nested fields, and emit exports as documents `sim workflows export <id>` printed `version` and `exportedAt` and nothing else. The record builder kept only scalar fields, so `workflow` and `state` — the entire export — were discarded with nothing to say they had been. Same for `workflows get`, which silently dropped `variables` and `inputs`. Record views now render every field. Nested values serialize to one line and are cut at 160 chars: visibly partial beats silently absent, and json/yaml output still prints them whole. Export is a document, not a record — it exists to be redirected to a file and fed back to `import`, and table/text flatten and truncate, so neither can round-trip it. `document: true` in the contract makes those formats fall back to JSON; yaml is honoured because it round-trips. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * feat(cli): JSON flags accept @file and @- alongside inline JSON A workflow export is hundreds of lines, and `--workflow` only took it inline. The shell makes that miserable: unquoted `$(cat wf.json)` word-splits into broken JSON, and nothing in the help said passing a file was an option. Every JSON flag now reads `@path`, or `@-` for stdin, so the round trip is `sim workflows export <id> > wf.json` then `import --workflow @wf.json` — or one pipe. `@` cannot collide with a real value because JSON only ever starts with `{ [ " -`, a digit, or t/f/n. Stdin drains with a readSync loop rather than readFileSync(0): a pipe is opened non-blocking, so the single-read form returned EAGAIN and died with a raw stack trace exactly when the upstream process had not written yet. Parse failures that look like a filename now say so — naming @path, or the file itself when the bare value turns out to exist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * feat(api): expand the public v2 files surface (#6160) * feat(api): expand the public v2 files surface Adds folder support, rename/restore, move, bulk archive, share, and content replace to /api/v2/files, so managing files by API no longer stops at upload + download + archive-one. Routes are thin: auth -> parse -> perform* -> serialize. Share and content replace get their orchestration extracted first so the session routes and the public ones cannot diverge on the effective-authType resolution, the EE public-sharing gate, or the storage-quota classification. Presigned upload stays session-only: presign does an advisory quota check and the real debit happens in the separate register step, so a caller that never registers leaves unaccounted bytes with no reaper. The buffered multipart path debits inside uploadWorkspaceFile's own transaction. * fix(files): classify folder and content failures instead of 500ing them Bugbot round 1. The v2 routes map errorCode straight to a status, so every manager failure that arrived unclassified became a 500 for what is really a caller-fixable 400 or 404. - Folder manager throws OrchestrationError: missing target/folder -> not_found, reparent cycle / self-parent / restore-into-archived-workspace -> validation. - File manager does the same for the in-transaction 'File not found' paths that the earlier pass missed. - updateWorkspaceFileContent's outer catch re-wrapped everything in a bare Error, which stripped the class off StorageLimitExceededError and the new not_found alike. It now rethrows a classified failure untouched and attaches cause to the generic wrap, so asOrchestrationError can still walk the chain. - Every remaining perform* gained the asOrchestrationError branch. - renameWorkspaceFile returned the pre-update read, so the v2 PATCH reported a stale updatedAt; it now returns the timestamp it actually wrote. Docs: upload auto-suffixes a duplicate name rather than rejecting it, matching the in-app uploader. The description claimed 409 and was simply wrong. * fix(files): surface a failed upload read-back as the real error getWorkspaceFile swallows a query failure and returns null unless throwOnError is set, so a transient blip on the post-upload read reported as 'file could not be read back'. Distinguish the two: a real null after a just-committed write is an invariant break, a query failure is itself. * revert(api): drop the dedicated v2 file-folder routes File folders already live in the shared folder table as resourceType 'file' (#6045 cut them over, #6051 dropped workspace_file_folders), and the remaining file-specific folder machinery is being folded into the generic folder engine. Publishing /api/v2/files/folders/** would pin that transitional split into a public contract we'd then have to keep or break. Files stay folder-aware — folderId/folderPath on the projection, folderId on upload, and the move route — because a folder id is a folder.id and survives the unification untouched. Folder management belongs on /api/v2/folders once that surface serves resourceType 'file'; until then there is no v2 way to enumerate file folders, which is the deliberate gap. The orchestration classification fixes stay: the internal routes and the copilot file-folder tools still call those perform* functions. * fix(files): classify upload failures instead of matching their wording Bugbot round 2. uploadWorkspaceFile had the same outer-catch rewrap that updateWorkspaceFileContent did, so a blown storage quota reached the route as a bare Error and the v2 handler recovered the status by substring-matching the message. Any rewording silently demoted a 413 to a 500. - uploadWorkspaceFile rethrows a classified failure untouched and attaches cause to the generic wrap. - FileConflictError is now an OrchestrationError('conflict'), so a duplicate name classifies like every other conflict. Its 'FILE_EXISTS' discriminator had no readers and is gone; the instanceof checks elsewhere still hold. - The v2 upload handler uses v2CaughtOrchestrationError, dropping all three string matches. Also documents that bulk-archive is best-effort: unknown or already-archived ids are skipped rather than failing the call, and deletedItems is what actually happened. That asymmetry with the single-id DELETE was undocumented. * feat(cli): wire the expanded v2 files surface Regeneration picked up seven new operations (72 → 79), every one of which derived badly. `/files/move` and `/files/bulk-archive` put a verb where the deriver expects a sub-resource, so each became a group holding a lone `create`; `GET /files/[id]/share` fetches one share and was read as a collection and named `list`; and `PATCH /files/[id]` derived to `files update` while its own summary said "Rename File". Named them: batch-archive (matching tables rows batch-delete), move, rename, restore, set-content, share get, share set. Bulk archive is gated behind --yes like the other batch destructives. `files list` gained --scope active|archived, and its rows now carry folderPath — added as a column, since which folder a file sits in is what distinguishes two rows sharing a name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * feat(cli): sim files upload The counterpart to `files download`, and hand-written for the same reason: POST /api/v2/files is multipart, which the generated flag surface cannot express, so `uploadFile` has been hidden since the start. Reads the file with openAsBlob so it stays on disk while the request is written, rather than buffering the whole upload in memory. Size is checked against the route's own 100MB ceiling before anything is sent. Content type comes from the extension, since the stored type decides whether the workspace later renders a file or offers it for download. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * fix(cli): make tables rows query show the rows Three things stacked up so the command appeared to do nothing. A row's cells live under `data`, and column inference skips object-valued fields — so the table came back listing an id and two timestamps per row and none of the content the query was run for. `expand` names the wrapper whose keys become columns, unioned across the page like the top-level ones. A cell key that shadows a top-level field is shown by its full path, so two different values never share a header. A cell containing a newline pushed the rest of its row onto the next line and every column after it lost alignment; in text mode a tab invented a field that `cut -f` reads as real. Display cells are now flattened to one line. `sanitize` still keeps \t and \n — json and yaml must round-trip them, and this is applied only to finished cells. A single cell holding an LLM response set the column width for the whole table and pushed everything after it off-screen, so table cells clamp at 60 columns. text/json/yaml are untouched: those exist for the whole value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * fix(cli): make boolean flags able to say false `--is-active false` turned sharing ON and reported success. Booleans were declared presence-only, so the flag meant `true` and commander dropped the `false` as an argument the command had no use for — silently, because excess arguments are ignored by default. A required boolean now takes its value (`--is-active <true|false>`): it is a state to set, not a switch to flip on, and as a presence flag it could only ever send one of the two values it needs to express. Optional booleans stay presence-flags — `--deployed-only` reads better than `--deployed-only true` — but each also gets `--no-<name>`. Omitting one means "leave it alone", which is not the same as setting it false; without the negation there was no way to disable an MCP server or unlock a folder. Excess arguments are now an error on every generated command, so a value attached to the wrong flag stops rather than being silently discarded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * feat(api): add search, filtering, and sorting to the v2 list endpoints (#6189) * feat(api): add search, filtering, and sorting to the v2 list endpoints One convention across every v2 list, documented on lib/api/contracts/v2/shared.ts: `search` (case-insensitive substring on the resource's natural name field), `sortBy` + `sortOrder` (per-resource enum, never a free string), and enumerated resource-specific filters. Reuses the sortBy/sortOrder pair v2 logs and v2 knowledge-documents already ship rather than inventing a third dialect alongside the Logs filters and the Tables predicate grammar. Every filter and sort is pushed into SQL. GET /api/v2/files previously read the whole scope and sorted/sliced it in JS; it now goes through a new queryWorkspaceFiles that filters, orders, and bounds the page in one query. Cursors are stamped with the sort they were minted under, so replaying one under a different sort is a 400 instead of silently duplicated or skipped rows. * fix(api): validate v2 cursor key values and compare timestamps at ms precision Two review findings, fixed at the root by making a keyset key own its cursor codec instead of hand-writing a decoder per sort. Cursor key values are caller-controlled, and matching the sort stamp and key count was not enough: an unparseable timestamp or a non-numeric size reached the query as an Invalid Date or NaN and surfaced as a 500. Each key now type- checks its own value and rejects a cursor it cannot hold, which both routes render as the documented 400. Timestamp keys now order and compare on date_trunc('milliseconds', col). Postgres keeps microseconds and defaultNow() populates them, but a cursor value round-trips through a millisecond-only JS Date — comparing the raw column against the truncated value re-admitted the page's own last row, duplicating it and stalling pagination outright at a page size of one. Reachable today via workspace_files.updated_at, which insertFileMetadata leaves to defaultNow(). * feat(api): complete the v2 workflows resource with versions and CRUD (#6184) * feat(api): complete the v2 workflows resource with versions and CRUD Adds version listing/detail plus create, update, and delete to the v2 workflows surface, which previously covered only execution and deployment. - GET /api/v2/workflows/[id]/versions — cursor-paginated, newest first - GET /api/v2/workflows/[id]/versions/[version] — version + pinned state - POST /api/v2/workflows, PATCH and DELETE /api/v2/workflows/[id] All six delegate to the existing orchestration and persistence helpers; no new domain logic. * fix(api): check folder containment before lock state; reject malformed version cursors assertFolderMutable walks a folder's ancestor chain without filtering on workspace, so inspecting it before containment let a caller tell a locked folder in someone else's workspace (423) from a nonexistent one (400). Create and update now assert containment first, matching the ordering import-workflow.ts already uses. A version cursor that decodes to JSON without a numeric version filtered every row out and returned an empty page with nextCursor null, which reads as a clean end-of-list. Malformed cursors are now a 400. * refactor(api): page workflow versions in the persistence helper listWorkflowVersions read every version row and the route filtered and sliced the result in memory, so the response was bounded but the query was not. It now takes optional limit/afterVersion, turning the cursor into a real keyset query; the route asks for limit + 1 and only trims the has-more probe. Both params are optional, so the internal, v1 admin, and copilot callers are unchanged. Also restores the untouched GET handler in [id]/route.ts to its original formatting — collapsing its signature had re-indented the whole body and buried the actual additions in whitespace churn. * feat(api): expand v2 tables with stateless multipart transfers (#6188) * feat(api): expand the public v2 tables surface Adds 16 operations so a v2 caller can do what the internal surface can: rename/move/lock a table, restore it, manage saved views, run enrichment columns, look up rows, and import/export with observable job control. Extracts lib/table/orchestration/import.ts (performTableCsvImport, performCreateTableFromCsv) and lib/table/export-stream.ts from the first-party routes, then repoints those routes at them, so v1 and v2 cannot drift on what an import or export actually does. events/stream, metadata and dispatches stay internal — they are editor state, not public API. * fix(api): make v2 table PATCH all-or-nothing and name the lock in every 423 Greptile P1: PATCH applied locks, rename and move as three sequential transactions, so a folder rejected mid-request left the earlier writes persisted while the response reported failure — and the schema-changed signal was skipped, leaving open clients on stale state. Every rejectable condition now runs before the first write, and the signal fires whenever anything did land. Cursor: v2TableLockError dropped the lock kind, so async import, column run, enrichment and table mutations returned a bare LOCKED. A table has four independent locks, so the caller could not tell which to clear. * fix(api): report the lock kind on classified 423s too, not just thrown ones The previous commit named the lock only where the rejection was thrown and caught at the route boundary. Where it instead arrives as a classified `errorCode: 'locked'` outcome — delete table, delete row, update column, and the table mutations — the kind was dropped, so those 423s stayed unactionable while their neighbours improved. The orchestration results now carry `lock`, and a shared `v2TableOrchestrationError` renders both arrival paths into the same `{ code, message, details: { lock } }` body. `details` is omitted rather than sent null when the kind is unknown, so a caller branching on it sees absence instead of a phantom value. * fix(api): make async table imports observable, not just startable `POST /import-async` pointed callers at `GET /api/v2/tables/jobs` to track progress, but that endpoint filters to `type = 'export'` — imports are derived onto the table itself, one write job at a time, and exports get a separate list precisely because they are excluded from that derivation. The public Table shape omitted those derived fields, so an async import could be started and cancelled but never observed to completion, failure, or progress. That is the gap the import/export/job-control set was meant to close. Table now carries `job` — id, type, status, rowsProcessed, error, or null when idle — and the import-async docs point at the table rather than the export list. * feat(api): make v2 table PATCH state which operations landed on failure Greptile held the PR at 4/5 on the residual non-atomicity and named two acceptable resolutions: make PATCH atomic, or have the contract adopt and expose partial-success explicitly. Atomicity would mean threading one transaction through renameTable, moveTableToFolder and updateTableLocks — three shared service functions with four non-test callers including the first-party route and two copilot tools — and deferring their per-operation audits to commit time. That is a refactor of shared write paths well outside this PR. So the contract states it instead. Every rejectable condition is already pre-validated, so a failure here is a genuine fault; when one follows a successful operation the error now carries `details.applied` listing what is live. Absent when nothing applied, so its presence always means "these changes took effect despite the error". Documented on the operation. `v2ErrorForOrchestration` gained the optional `details` this needs. * fix(api): make table lock flags read-only on the public v2 surface The new PATCH /api/v2/tables/[tableId] accepted a `locks` object, gated on workspace admin plus the table-locks feature. That still lets an API key clear the guard placed there to stop it: `write` is the floor for the endpoint, and admin keys are ordinary API keys, so a lock is no longer a boundary the key cannot cross. Locks stay readable on the table resource and enforcement is unchanged (a locked verb still returns 423). Changing one is now a first-party admin action only. The v2 body is declared here rather than reusing the first-party updateTableBodySchema, which keeps its `locks` field so the UI can still toggle them. It is .strict(), so a request carrying `locks` is rejected with a 400 naming the field instead of silently succeeding without applying it. * fix(api): keep reporting applied operations when the PATCH re-read fails The composite table PATCH promises that `error.details.applied` names the operations that are live despite an error, but `applied` was scoped inside the try. A rename or move that committed and was then followed by a throw in the final re-read — or a re-read finding the table archived — returned a bare 500/404 with no details, telling the caller nothing had landed. It would then retry into a duplicate-name conflict or repeat the move. `applied` is now function-scoped so every post-write exit carries it: the 404 on a missing re-read, a thrown lock error, a classified orchestration error, and the generic 500. `v2TableLockError` gains the same `extraDetails` parameter `v2TableOrchestrationError` already had. * feat(api): add workflow group writes to the v2 tables surface v2 exposed GET /groups but none of the writes, so the public API could run an enrichment or workflow column and read its binding, but never create one. A caller could add a plain data column and trigger the machine; wiring the two together still required the UI. Adds POST/PATCH/DELETE on /api/v2/tables/[tableId]/groups. The group is the unit that fills columns — one group feeds several — so creating one creates its output columns in the same call, matching the first-party shape rather than inverting it onto the column endpoint. Four departures from the first-party body, all public-surface concerns: - group.id is optional and server-generated. The UI mints an id to render optimistically; a public caller has no such need and a client-chosen id is a collision waiting to happen. - outputColumns[].workflowGroupId is dropped from the body and stamped from the resolved group, so it cannot disagree with it. - autoRun defaults to false. First-party defaults true so a UI add fills cells immediately; here it would make one POST fan out a metered run across every existing row. - A group naming neither a workflowId (type manual) nor an enrichmentId (type enrichment) is a 400 rather than a half-specified group the route has to guess about. Also rejects an outputColumns entry no group output feeds — the two arrays are joined by column name, and the first-party client builds both from one picker so it cannot desync, but a public caller can. Workspace containment on workflowId is asserted before it is persisted, on create and on any update that re-points the group; without it a table becomes a way to invoke workflows the key cannot otherwise reach. * improvement(api): make v2 table import and export async-only Drops the three synchronous entry points: POST /tables/[tableId]/import, POST /tables/import-csv, and GET /tables/[tableId]/export. Sync import tied a write to the lifetime of an HTTP request. The body *was* the data, so it carried a 10 MB cap that Next silently truncates past — a partial import reporting success. It also had no job, so a timeout mid-write left rows in place with nothing to poll and nothing to cancel. The async path reads the file from storage instead: upload via POST /api/v2/files for a key, start with POST /import-async, watch GET /tables/[tableId] -> job, stop with POST /job/cancel. Sync export carried no such hazard, but one shape per operation beats two: with both removed the surface has exactly one way to move a table in or out, and the CLI wraps the extra calls. This also removes the last multipart handling in v2 tables. Those were the only routes bypassing parseRequest — form fields were parsed by hand against separate form schemas, outside the contract system every other v2 write goes through. Create-a-table-from-CSV is now two calls: POST /tables, then /import-async with createColumns. csvImportModeSchema is append|replace, so there is no single-call create. Route baseline 1064 -> 1061. * docs(api): correct the import-async note about upload size limits The docstring claimed there is no synchronous upload endpoint and so no request-body size cliff. Both are wrong: POST /api/v2/files is a synchronous multipart upload with a 100 MB cap, and it is the only v2 upload path (presigned is deliberately absent). What async-only actually bought: the cap went 10 MB -> 100 MB, it fails on an explicit size check and a bounded body read rather than a proxy cap that silently truncates, authorization completes before any body is buffered, and the table write is a job that can be watched and cancelled. * feat(api): unify file and table transfers * improvement(api): make multipart transfers stateless * fix(api): make table import completion retries idempotent * feat(cli): pick up v2 workflow CRUD, table transfers, and list search 79 → 111 operations across three merged PRs. The generator could not read the new contracts at all: a table view's filter is a recursive predicate, so Zod lifts it into `$defs` and refers to it, and `toTypeScript` threw on the first `$ref`. Those definitions are now hoisted into named aliases — recursion TypeScript resolves without complaint — named after the type that owns them so two operations lifting their own `__schema0` cannot collide. Uploading is no longer one multipart POST. `POST /api/v2/files` is gone, replaced by a presigned handshake, so `files upload` was left calling a route that no longer exists. It now creates the upload, signs part URLs in batches of 100 (each is short-lived, so signing all of them up front would expire the last ones), PUTs each part straight to storage, and completes with the ETags — aborting the upload if any step fails, since a half-finished one holds storage. Parts are read through `Blob.slice`, so only the part in flight is in memory. Verified byte-identical on a 24MB round trip. The rest is naming. `/cancel-runs`, `/rows/find`, `/restore`, `/columns/run` and the enrichment path each put a verb where a sub-resource was expected, so each had become a group holding a lone `create`. Transfer steps keep names that say what they are, since no single command drives a table import yet. Three new DELETEs needed gates, which the existing guard test caught. Aborting an upload and cancelling an import or export stop something in flight rather than destroying something kept, so those are exempt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * feat(cli): sim tables import Imports a CSV into a new or existing table, driving the same presigned handshake `files upload` uses — the two are the same protocol against different paths, so they now share one implementation. What made this more than a wrapper is that the import carries decisions the handshake does not: the source is a local file or one already in the workspace, the target is a new table or an existing one to append to or replace, and mapping/createColumns are rejected unless the target is existing. Both choices are required rather than inferred — defaulting to a new table would turn a forgotten --to-table into a silent second copy of the data — and the conditional flags are checked here so the error names the flag instead of arriving as a complaint about the request body. The transfer only queues the work; rows are parsed afterwards, so returning at `complete` would report success for an import that goes on to fail on a bad row. It polls to a settled status and reports the rows written, with progress on a terminal only. --no-wait opts out. The handshake steps are hidden now that a command drives them; `imports get` and `imports cancel` stay, being useful against an import already running. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * feat(cli): default tables import to a new table named after the file `sim tables import people.csv` now does the obvious thing rather than demanding a target. Requiring one guarded the wrong direction: a forgotten flag creating a new table is visible and easily undone, while the outcome worth protecting — writing into an existing table — is the one that now has to be asked for by name. --to-table becomes --table-id, and --mode/--mapping/--create-columns apply only alongside it. Passing one without it is an error rather than a no-op: silently ignoring `--mode replace` would let it read as honoured while a new table was created beside the one it was meant to overwrite. The reverse is also refused, since --table-id already names the destination. The derived name is sanitized, because table names are identifiers: the obvious basename would reject most real files, so `2026-quarterly sales.csv` imports as `_2026_quarterly_sales` instead of failing. --name overrides it, and is required for --file-id, where there is no file name to take one from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * feat(v2-tables): paginate the table list `GET /api/v2/tables` returned every table in the workspace in one response — it used the cursor envelope but hardcoded `nextCursor: null`, and had no `limit`. That was defensible when tables were only created through the UI; `POST /api/v2/tables` is public now, so a script can create them in bulk and the list has no way to ask for less. Adds `queryTables` alongside `listTables` rather than changing it, so the internal callers that genuinely want the whole scope are untouched — the same split `queryWorkspaceFiles` / `listWorkspaceFiles` already uses. Filter, order and slice all run in the query, so a `search` never costs a full-workspace read. A cursor whose values don't bind raises a validation error instead of being coerced to "no filter", which would have silently served page 1 under a resumed cursor. The keyset closes on `id` so a page boundary inside a run of equal names or timestamps stays stable. The shared `LimitQuery` doc component said "Maximum rows to return"; it now serves the table list too, so the wording is resource-neutral. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(cli): regenerate for the paginated table list `listTables` gained `limit` and `cursor`, so the CLI's auto-pager now drives it like every other paginated list — no CLI change, which is the point of generating this file. Also picks up `isCurrent` on workflow versions and a new `voice-output` enum member. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * fix(cli): make tables rows create and tables columns run usable Both were dead on arrival, for opposite reasons. `createTableRows` takes `z.union([batch, single])`. A union has no flat field list, so the generator emitted no body slot — and slot absence reads the same as "this operation has no body", so the command offered nothing and sent nothing. The generator's own comment claimed the runtime fell back to taking the body as JSON; nothing did. Unions are now marked, the fields every branch shares are still emitted (both require `workspaceId`, which comes from the profile), and `--body <json|@file>` carries the rest, merged over them so the caller still wins on any key it sets. Dropping that merge was my first attempt and it failed on the missing workspace. `runTableColumn` takes `limit: { type, max }`. The pager claimed the *name* `limit` regardless of type, so it became `--limit <n>` with a default of 100 and sent a number the route rejected on every call, whether or not the flag was passed. The special case now applies only where `limit` is numeric; elsewhere it is an ordinary field and gets the JSON flag its type calls for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0148a65fkPhP4N8tgGPYtRTU * feat(api): add multipart knowledge document uploads * fix(api): keep usage admission at knowledge upload session creation * feat(knowledge): wire knowledge base uploads to multipart sessions * fix(knowledge): refuse to abort an upload once a document is bound * fix(uploads): prevent multipart cleanup races * fix(cli): improve command usability and structure * feat(cli): support knowledge document uploads * Unify file creation and signed upload sessions (#6264) * feat(uploads): unify signed upload sessions * fix(uploads): preserve attachment storage semantics * feat(files): add authored file creation * fix(uploads): omit hoisted S3 metadata headers * fix(cli): support unified upload sessions * feat(api): add file metadata endpoint * feat(cli): accept simple list inputs * improvement(api): scope folders to resource paths (#6284) * improvement(api): scope folders to resource paths * fix(files): serialize folder resolution with uploads * fix(files): release folder lock before upload setup * feat(cli): add path-based resource directories * feat(cli): add resource mkdir commands * fix(cli): accept positional folder paths * fix(api): normalize folder paths and unblock resource mutations * fix(api): make resource cleanup and metadata consistent * improvement(uploads): persist multipart sessions in postgres * fix(db): store table row trigger timestamps in UTC * improvement(api): default folder deletion to non-recursive * feat(cli): streamline common resource workflows * feat(cli): standardize resource command syntax * fix(billing): unify chat usage source * improvement(logs): expose trace spans on log detail * feat(cli): sync unified chat billing source * feat(cli): expose log detail trace spans * fix(logs): parse list trace spans * improvement(api): replace workflow jobs with execution resources (#6294) * improvement(api): replace workflow jobs with execution resources * fix(api): preserve legacy jobs while preferring v2 executions * fix(api): make execution polling resume-aware * fix(ui): hide async examples for public workflows * fix(api): bridge resume queue visibility lag * feat(api): add v2 workflow resume endpoint * fix(api): project pending resume attempts * fix(api): prefer terminal logs over stale resumes * improvement(api): unify v2 resource query layers (#6319) * improvement(api): unify v2 resource query layers * fix(api): address v2 review findings * fix(api): preserve cancelled queue status * fix(api): guard cancelled job transitions * fix(api): close v2 resume and log gaps * feat(cli): improve v2 command workflows * feat(api): rename v2 executions to runs * feat(api): split credentials and secrets * feat(api): add workspace metadata and email attribution * improvement(api): consolidate public v2 route handling * feat(cli): sync v2 API and personal login defaults * fix(cli): use profile workspace for workspace get * fix(cli): use profile workspace for member listing * improvement(files): centralize operations across APIs and Copilot (#6392) * improvement(files): unify rename authorization * chore(skills): add file operation migration guide * improvement(files): consolidate file operation authorization * improvement(files): extract shared operation foundation * improvement(api): simplify internal route declarations * improvement(files): centralize application authorization * refactor(api): share workspace file name validation * refactor(files): centralize copilot application calls * docs(skills): generalize application operation migration * fix(cli): restore nested knowledge document commands * feat(cli): add interactive Sim chat * improvement(api): centralize remaining v2 resource operations (#6412) * improvement(api): centralize v2 resource operations * fix(api): preserve custom tool conflict errors * improvement(api): migrate policy-sensitive v2 reads (#6410) * improvement(workflows): centralize v2 application operations (#6411) * refactor(api): migrate v2 knowledge operations (#6413) * refactor(api): migrate v2 knowledge operations * fix(knowledge): fail upload completion on dispatch errors * fix(knowledge): preserve upload retry and VFS errors * feat(cli): refine chat and desktop updates * improvement(tables): centralize v2 application operations (#6414) * improvement(tables): centralize v2 application operations * fix(tables): preserve run validation and signals * style(desktop): refine macOS installer layout * fix(api): type timestamp cursor parameters * feat(cli): add saved chat commands * chore(desktop): trigger updated prerelease * feat(cli): publish @simai/cli release channels * fix(ci): include auth package in app prune * feat(cli): separate file descriptions from content * feat(cli): add resumable async Sim Chat * fix(cli): guard file unsharing * feat(auth): add scoped internal executor delegation (#6459) * feat(auth): add scoped internal executor delegation * fix(auth): derive delegation lifetime from one timestamp * Include share status in file metadata * feat(auth): centralize delegated identity policy (#6462) * fix(cli): stream file content to stdout by default * improvement(copilot): consolidate application adapters (#6450) * improvement(api): harden application route boundaries (#6451) * improvement(api): harden application route boundaries * fix(folders): reject creates at workspace cap * fix(knowledge): enforce trusted workspace scope (#6452) * fix(knowledge): enforce trusted workspace scope * refactor(knowledge): declare v2 body lifecycle * finish knowledge application migration * refactor(knowledge): compose copilot batch commands * fix(knowledge): parse connector query flags * fix(knowledge): finalize partial batch effects * fix(knowledge): align merged application boundaries * fix(knowledge): close application boundary review gaps * style(knowledge): satisfy branch biome checks * fix(knowledge): page connector documents in editor * refactor: enforce Copilot table application boundary (#6453) * refactor: enforce copilot table application boundary * fix(tables): finish application boundary migration * fix(tables): restore scoped copilot imports * fix(tables): compose copilot commands atomically * fix(tables): preserve workflow group scheduling * fix(tables): complete fixed copilot composition * fix(tables): reject enrichment output mutation * fix(tables): complete authorized application boundary * fix(workflows): migrate Copilot application boundary (#6455) * fix(workflows): migrate Copilot application boundary * fix(workflows): finish delegated application migration * fix(workflows): encode VFS folder aliases * fix(workflows): close application composition gaps * fix(workflows): preserve VFS validation errors * fix(workflows): complete application boundary migration * test(workflows): format canonical binding coverage * fix(workflows): scope executor metadata reads * fix(workflows): bind executor metadata targets * fix(copilot): recover tool arguments lost when a call is checkpointed mid-generation Tool arguments reach Sim two ways: whole on a frame's `arguments`, or in pieces as `args_delta` chunks that accumulate into `streamingArgs`. Only the first populated `params`, so a call checkpointed before any frame carried `arguments` executed with `{}` and failed its own schema on every required property. The file subagent's `workspace_file` calls arrive exactly that way, which left the agent retrying and then routing around the tool entirely. - executor: hydrate `params` from the streamed deltas before dispatch, covering both normal dispatch and the never-dispatched resume path. - handlers: record the subagent channel at registration rather than only on a finalized frame, so the workspace_file -> edit_content intent handoff can find its intent instead of reporting "No workspace_file context found". - preview adapter: pass the frame's tool call id into file delegation, which derives its audit id from it. Without it every preview threw and no file content streamed at all. - run: a checkpointed call with no recorded result now reports a failed result instead of throwing, which ended the whole turn and cost the user the entire response. Each fix has a regression test verified to fail without it. * feat(credentials): add v2 OAuth connection APIs * fix(credentials): preserve active OAuth connection links * fix(credentials): bind OAuth links to connection intent * feat(credentials): complete v2 credential lifecycle * fix(credentials): make disconnect idempotent * fix(credentials): stabilize oauth draft retries * fix(credentials): bind oauth callbacks to drafts * fix(credentials): fail closed on oauth completion * fix(credentials): bind shopify completion to oauth state * revert(chat): remove Sim Chat and mothership changes Reverts this branch's Sim Chat surface and its mothership-view edits. d4bdb87d04 ("feat(cli): add interactive Sim chat") could not be reverted: it is a 175-file commit that also introduced the ExecutionContext refactor and the v2 workspaces API, which 33 files under lib/copilot now depend on. Reverting it produced 42 content conflicts, 24 of them in the refactor rather than in chat code. Its chat contribution is removed forward instead. Reverted: b25e7ba0c3 fix(copilot): recover tool arguments lost when checkpointed d4c74648b0 feat(cli): add resumable async Sim Chat 3f0c1fcce0 feat(cli): add saved chat commands de263204fd feat(cli): refine chat and desktop updates -- mothership-view only; desktop updater, terminal themes, and bridge are kept Removed forward: the 17 CLI chat modules, /api/v2/chat, /api/v2/chats, the v2 chat contracts, and lib/copilot/headless. lib/copilot/chat/turn-persistence.ts is kept because the web chat's post.ts imports it. Left alone: upstream Copilot application-boundary work (#6450-#6462), the sim-chat billing source, and the v2 workspaces API. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JNFjchn6dcM7xevh34PKHE * fix(credentials): align custom oauth reconnects * fix(credentials): centralize application authorization * fix(credentials): keep OAuth draft intent immutable * chore(cli): regenerate v2 API for the staging sweep Additive only, both picked up automatically by the derived command surface: - cancelWorkflowRun `reason` gains already_cancelled / already_completed / already_failed (#6702) - getFile gains `scope` (active | archived), so `sim files describe` grows a --scope flag defaulting to active Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JNFjchn6dcM7xevh34PKHE * fix(credentials): allow renamed reconnect targets * fix(credentials): close OAuth draft edge cases * fix(ci): green the repo audits after the staging merge check:import-specifiers — 79 violations, all packages/sim-cli. That package is "moduleResolution": "nodenext" while the rest of the repo is "bundler": Node's ESM resolver takes the specifier literally, so `./ini.js` is required for a file that is `./ini.ts` on disk. Following the audit's advice to drop the extension would break the CLI at runtime. The checker now reads each workspace's tsconfig and, for a NodeNext package, resolves a `.js` specifier back to its source instead of flagging it — still catching genuinely missing files. check:utils — device-flow.ts polled with `new Promise(setTimeout)`. It cannot import sleep() from @sim/utils: that package is private, so a published @simai/cli would resolve it in the monorepo and fail from npm. Added a local helpers.ts and allowlisted it, matching the existing packages/cli entry. check:tool-registry-boundary — knowledge/page.tsx measured +43 against a +42 allowance. tools/registry.ts is not a gateway on any route, so the boundary the audit exists to protect is intact; the growth is this branch's own v2 work. Re-recorded per the script's own instruction. helm — restored staging's networkpolicy_test.yaml. An earlier integration merge had dropped its trailing newline, which was the only helm delta against staging and was tripping the chart-version-bump check. Not addressed: the Security audit step reports high advisories (brace-expansion, undici, Socket.IO, OpenTelemetry, fast-uri). Every one is transitive and present in staging's lockfile too; the step is continue-on-error and did not fail the job. bun audit reads a live advisory feed, so staging's green run predates them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JNFjchn6dcM7xevh34PKHE * fix(cli): read share fields from the v2 share object The file-share columns pointed at a `sharing` wrapper that v2 does not return. The share travels under `share` on file metadata (null when unshared) and as the unwrapped body on the share endpoints, and its flag is `isActive`, not `enabled`. Every one of those columns was therefore rendering an em-dash on `files describe`, `files share get`, and `files share set`. A missing field path renders blank instead of failing, so nothing caught this. Added a rendering test over both surfaces; it fails if a path stops resolving. `hasPassword` is surfaced on the two share commands while they are being fixed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JNFjchn6dcM7xevh34PKHE * fix(credentials): fail closed without breaking auth * improvement(cli): bundle and publish as @sim/cli * fix(credentials): preserve migrated route behavior * fix(cli): confirm before overwriting login profile * fix(cli): adapt service-account credential fields * feat(cli): prompt for secret values * fix(cli): harden login and credentialless chat * fix(cli): skip existing package releases * fix(cli): avoid exposing API key fragments * fix(cli): cancel failed download streams * fix(cli): sanitize authentication metadata * fix(cli): rename authentication output metadata * fix(cli): publish downloads atomically * fix(cli): harden download destinations and labels * fix(cli): support dangling download symlinks * fix(cli): keep new downloads atomic * refactor(cli): remove unrelated Copilot changes * fix(cli): publish as sim package * fix(cli): use staging npm tag * fix(cli): align vitest lock resolution --------- Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Siddharth Ganesan <siddharthganesan@gmail.com>	2026-08-15 04:51:30 -04:00
Theodore Li	6006870f02	feat(credentials): add v2 credential lifecycle APIs (#6664) ... * feat(credentials): add v2 OAuth connection APIs * fix(credentials): preserve active OAuth connection links * fix(credentials): bind OAuth links to connection intent * feat(credentials): complete v2 credential lifecycle * fix(credentials): make disconnect idempotent * fix(credentials): stabilize oauth draft retries * fix(credentials): bind oauth callbacks to drafts * fix(credentials): fail closed on oauth completion * fix(credentials): bind shopify completion to oauth state * fix(credentials): align custom oauth reconnects * fix(credentials): centralize application authorization * fix(credentials): keep OAuth draft intent immutable * fix(credentials): allow renamed reconnect targets * fix(credentials): close OAuth draft edge cases * fix(credentials): fail closed without breaking auth * fix(credentials): preserve migrated route behavior * feat(credentials): add provider search * fix(credentials): prevent stale secrets and drafts	2026-08-15 00:47:06 -04:00
Theodore Li	ee1fc379a4	fix(tables): allow unbounded v1 row queries (#6713) ... * fix(tables): allow unbounded v1 row queries * fix(tables): drain under-budget queries fully * fix(tables): bound expanded query metadata * fix(tables): always return query totals	2026-08-14 21:08:36 -04:00
Waleed	3848f97b4c	fix(grafana): validate against the API docs, add data source querying and contact-point CRUD (#6712) ... * fix(azure-data-explorer): correct the tags ingestion-property example The example rendered as tags="[''daily'']" — doubled single quotes from an escaping slip, which is not valid Kusto. The reference writes a tags list as tags='["TagA","TagB"]': single outer quotes with the JSON array's own double quotes inside. The clause builder already handled that form; only the example text was wrong. A template literal avoids the escaping entirely, since the metadata generator reads the source verbatim and would otherwise carry the backslashes into the description the model sees. Adds a test asserting the reference's exact multi-property clause round-trips, including the comma inside the quoted array. * fix(grafana): correct response contracts, required alert fields, and outbound request hardening Validated against Grafana's HTTP API reference and, where the docs contradict themselves, against the Go wire structs. Response shapes the tools got wrong: - update_annotation declared an `id` that was always 0; a patch returns only a message, so the request's annotation id is echoed and labelled as such - delete_folder discarded the numeric id Grafana returns and presented an input-echoed uid as if it came from the API - delete_dashboard fabricated `id: 0` / `title: ''` via `||` on absent fields - the contact-point `provenance` description was inverted: "api" means API-managed, empty means it stayed UI-editable Requests that could not succeed: - create_alert_rule left noDataState and execErrState unset and invisible to the model, but Grafana's validator rejects an empty value outright, so every model-driven create failed. Both are now sent with Grafana's own defaults, and skipped for recording rules, which take a different validator - get_data_source routed a numeric input at /api/datasources/:id, which exists only behind an off-by-default feature toggle. UID only now - list_annotations did not trim the dashboard UID, so a padded value matched nothing Outbound hardening on the three proxy routes: - the service-account token was re-sent to redirect targets; the shared fetch only drops it when asked, so stripAuthOnRedirect is now set - no timeout was passed, leaving two sequential hops at the 5-minute default - upstream error bodies were interpolated whole into the tool result, putting up to 10MB of HTML into logs and traces; now truncated - UID path segments are URL-encoded so they cannot re-target the request - update_folder sent both `version` and `overwrite: true`, which Grafana treats as alternatives, making the freshly fetched version decorative and silently clobbering a concurrent rename - replaced the `any` casts with narrowed types Block surface: - 25 outputs the tools emit were undeclared and so unreferenceable downstream; get_data_source had 13 of its 18 unreachable - `version` was typed string though the dashboard, folder, and data-source producers all emit a number - the dashboard title field was shown only for create, so a dashboard could never be renamed through Update Dashboard - six list outputs were typed json rather than array * fix(grafana): let the health check report ill-health, and disambiguate block outputs The data source health check could only ever report health. Grafana answers an unhealthy source with HTTP 400 carrying the same {status, message} payload as a healthy one, and the tool framework converts any non-2xx into an opaque tool error — so the diagnostic the caller actually wants was unreachable. The check now goes through an internal route that reads the verdict off either status and reports it as a successful check, while a failure carrying no verdict (missing data source, bad token, plugin with no health endpoint) stays a real error. The plugin's `details` payload is surfaced too. Also on that route, matching the other three: an outbound timeout, redirect auth stripping, a truncated upstream error, and a URL-encoded UID. Block output descriptions: ten keys are emitted by several tools with different meanings and were described for only one producer — `database` meant both a data source name and a health status, `annotations` both an annotation list and an alert rule's summary map. Eleven `json` outputs were opaque although the tools already document their inner fields. All rewritten to name every producer. Smaller alignment fixes: - the same EmbeddedContactPoint.settings field was typed `object` in list and `json` in create - list_contact_points mapped non-nullable uid/name/type through `?? null`; Grafana returns an empty string, which is what create already assumed - create_alert_rule sent `orgID`, which Grafana overwrites from the authenticated context, and `Number()` on a non-numeric value put NaN -> null in the body - the three update routes declared `output` as required though the auth short-circuit omits it, and did not declare the `details` they emit on a validation error * feat(grafana): complete contact-point CRUD, and add folder move and rule-group read Four operations the integration was missing, taking it to 29. update_contact_point / delete_contact_point close a real gap: contact points could be listed and created but never corrected or removed. Two things worth recording, because the published docs get both wrong: - both verbs answer 202 with only a message, not the object. The rendered docs claim delete returns 204; the current spec and handler both say 202. So the UID is echoed from the request, the way delete_folder and update_annotation already do - update is a full replace with no PATCH counterpart, so name, type, and settings are all required and the description says so. Omitting disableResolveMessage resets it X-Disable-Provenance is exposed on update only. Its polarity is the opposite of the alert-rule case: omitting it always succeeds, while sending it against an API-provisioned contact point is rejected — with 403, not the 409 rules use. It is not exposed on delete at all, because that handler never reads stored provenance and the endpoint takes no such parameter. move_folder reuses get_folder's mapping verbatim — same DTO. It always sends the parentUid key, since Grafana reads an empty value as "move to the root", which a conditionally-omitted field could not express. get_alert_rule_group surfaces the group evaluation interval, the one alerting knob the per-rule operations cannot reach. It reuses the shared mapAlertRule for the nested rules, and the interval is documented as an integer of seconds. * feat(grafana): add data source querying, and ground the skill and templates in real tools query_data_source closes the largest gap in the integration: 29 tools could read dashboards, folders, and alert configuration, but none could read a metric value. It posts to /api/ds/query and returns both the raw response and the frames flattened into rows. The flattening is derived from the documented layout rather than any data source's field names: a frame carries schema.fields[] alongside data.values[], where values[i] is the whole column for fields[i], so zipping them by position works for Prometheus, SQL, or anything else with a backend. A failed query is a 400 by Grafana's own status table, so it stays a tool error — unlike the health check, where the failure status carries the answer. That also lets four templates and the review-firing-alerts skill stop promising things the integration could not do. Three templates assumed a metric-query tool, which now exists. The fourth, and the skill, assumed live alert instance state, which the provisioning API never returns — they now derive firing rules from alert-state annotations, which are documented to carry newState and prevState, and say so explicitly rather than implying a live snapshot. Deliberately not added: a tool over /api/prometheus/grafana/api/v1/rules for live instance state. That endpoint appears on no Grafana HTTP API doc page, its response is only readable from Go internals and test assertions, and the instance-level state casing differs from the rule level with no documented contract. Not something to build an output schema on. * fix(grafana): declare the two block outputs the earlier fixes introduced Renaming update_annotation's phantom `id` to `annotationId` and adding `details` to the health check both created outputs the block never declared, so neither was referenceable downstream. Caught by re-running the output-coverage check over both integrations; the block now covers all 64 keys the 30 tools emit. * fix(grafana): make Update Contact Point actually usable from the block The new replace operation could never succeed. contactPointType and contactPointSettings were widened to cover it, but contactPointNameNew was left create-only — and the update maps `name` from that field, so the required parameter was never supplied. disableResolveMessage had the same gap, and it matters more than it looks: the update is a full replace, so a block-driven update was silently clearing resolve suppression on every contact point it touched. Both fields are now shown, and required where the API requires them. Also states a reason on each intentionally-unconstrained response field — Zod issue objects, alert query stages, notification settings, recording-rule config, and data-source health detail are all genuinely opaque, but that was left implicit.	2026-08-14 15:58:17 -07:00
Waleed	d5701d5b2b	fix(icons): align table block icon and 12-unit icon stroke with the emcn family (#6708) ... The table/table_v2 blocks and the table trigger used a local lucide-shaped TableIcon (stroke 2.0, full-bleed 24 viewBox, 3x3 grid) while every other table surface used emcn's Table. Consolidate onto the emcn icon and drop the local copy. Nested tool-call rows in Chat applied no color class, so a non-brand block icon inherited body text instead of --text-icon. redo/undo/zoom-in/zoom-out draw 0.85 stroke on a 12-unit viewBox, rendering 0.992px at a 14px box against the family's 0.904px. 0.775 restores parity.	2026-08-14 14:49:24 -07:00
mzxchandra	5a88ce22d1	feat(ashby): incremental job sync, custom field writes, and application lifecycle ops (#6703) ... * feat(tools): add incremental job sync and draft postings to Ashby reads list_jobs accepts Ashby's syncToken and returns it as nextSyncCursor, so a scheduled sync costs O(changed reqs) instead of rescanning every req. Ashby only returns the token once the last page is drained, which the param description states. The output is named as a cursor deliberately. It is an opaque resumption marker, not a credential, so it belongs with nextCursor - and a field literally named syncToken matches the /^.*token$/i deny-list in redaction and renders as [REDACTED], which makes an incremental sync unusable since the operator cannot read the value the next run needs. The wire name stays syncToken. list_job_postings gains includeUnpublishedJobPostings, plus the posting status field - without status a caller cannot tell a returned draft from a published posting, which makes the flag useless. Also widens the custom field valueLabel type, which MultiValueSelect returns as an array, for the write operations that follow. * fix(tools): render Ashby object-shaped API errors readably Ashby documents two error shapes and uses both. The `errors` array form carries `{ message, parameter }` objects, which stringified to '[object Object]' and hid the real cause - including the 403 a key gets when it lacks a module permission. Also adds the shared pieces the new write operations need: one definition of the custom field value shape for the read and write paths to agree on, and a normalizer for Ashby's case-sensitive objectType enum so a model emitting 'candidate' fails here with the allowed values rather than at the API. * feat(tools): add Ashby custom field writes, delete, source, and anonymize customField.setValue/setValues are the only way to annotate a job or req, since Ashby has no job notes and no job tags. Writing null clears a value, so the annotation is reversible. Because null clears, every one of these operations requires explicit intent before it can destroy data. The block's required markers do not cover the agent path - a model calls the tool directly, so tools.config.params never runs and validateRequiredParametersAfterMerge skips a param marked not-required: - set_custom_field_value rejects an absent or blank fieldValue; an explicit null still clears - change_application_source requires unsetSource to clear, and rejects a source id and an unset request together, since preferring either one silently discards the other. Ashby has no 'leave unchanged' mode, so setting and clearing are the only two intents and exactly one must be expressed - set_custom_field_values rejects an empty array locally rather than relying on Ashby to reject it application.delete needs candidatesDelete, a module permission separate from candidatesWrite. candidate.anonymize strips PII but leaves the record; Ashby exposes no candidate deletion endpoint. * test(tools): cover the new Ashby request and response shapes Includes a gated live harness (ASHBY_LIVE=1) alongside the mocked tests. vitest.setup.ts stubs global fetch for every file in the app, so the live file restores the real implementation and asserts the restore worked - without that guard the whole suite silently passes against a mock. * feat(blocks): expose the new Ashby operations in the block fieldValue is polymorphic (boolean, number, string, array, object, null), so it decodes structured input and otherwise passes text through. The decoding is deliberately narrow rather than a blanket JSON.parse, which corrupts real text: 1e999 becomes Infinity and serializes back out as null, which CLEARS the field; a long numeric id loses precision past 2^53; and prose starting with { turns into an object. Only the literal keywords, {, [ or " prefixes, and exactly round-tripping numbers decode. fieldValue carries no wand generationType: json-object forces braces and json-array forces brackets, and both would wrap a value that must stay bare. fieldValues, whose contract really is an array, uses json-array. Setting and clearing an application source are mutually exclusive, so the Source ID field is conditioned off while the clear switch is on and the params mapping sends only the intent the switch selects. A value typed before the switch was flipped cannot reach the tool and surface as an error with no visible cause. * docs(ashby): document the new operations, permissions, and limitations Ashby scopes permissions per module and they fail at runtime, not build time, so the block docs now carry the permission table. Also records the hard API limits worth designing around: no note or tag on a job, no pagination on jobPosting.list, and no delete for jobs, candidates, or custom field definitions. * fix(blocks): stop a stale create-path source id leaking into a source change The executor merges { ...inputs, ...transformedParams }, so any key the params mapping leaves unset inherits whatever inputs held. The shared create-path sourceId subblock reaches inputs even on change_application_source: it is mode 'advanced', and the serializer includes an advanced subblock whenever its value is non-empty without ever evaluating its condition (serializer/index.ts). So a source id typed while on Create Application survived into a source change. With both fields blank it silently attributed a source nobody asked for, and with the clear switch on it collided with the unset request and failed with no visible cause, because the field producing it is hidden in that state. sourceId is now always assigned for this operation rather than conditionally, so it can never inherit. The regression test asserts the merged result rather than the mapping alone, since the gap between them is where the bug lived.	2026-08-14 14:37:23 -07:00
Waleed	5bb59f08ee	feat(connectors): add 9 knowledge base connectors (#6699) ... * feat(connectors): add 9 knowledge base connectors Box, Zoho Desk, PagerDuty, Trello, Microsoft Excel, Google Slides, Google Vault, Mintlify, and SFTP. Selected by intersecting the published connector catalogs of Glean, Onyx, Dust, Vectara, Writer, Guru, Elastic, Microsoft 365 Copilot, Notion AI, Unstructured, and Airbyte against services that already ship a Sim block, so OAuth providers, credentials, and icons are reused. Box was the largest gap, appearing in 7-8 of ~10 catalogs. Every connector was validated against live provider documentation twice, the second pass treating the first pass's conclusions as unproven. Notable correctness work that came out of that: Listing truncation. The sync engine hard-deletes documents past a cap that is not flagged with `listingCapped`, and five connectors had a path there — an empty Mintlify discovery, Zoho Desk's exact-multiple default caps, Trello's archived lists and 1000-card ceiling, a Google Vault cursor bailout, and a PagerDuty stalled page. The engine also gained a backstop: an empty or collapsed listing blocks deletion reconciliation until the same observation repeats on a consecutive sync, reconstructed from existing sync-log counters so no migration is needed. API alignment. `desk.zoho.ca` does not resolve (Canada is `desk.zohocloud.ca`, and Singapore and UAE were missing); `modifiedTime` is absent from Zoho's ticket list projection, so every ticket re-embedded on every sync; Trello's `dateLastActivity` is documented to miss some edits; PagerDuty's 10,000-record ceiling bounds `offset + limit`, not offset; Excel indexed dates as raw serial numbers while Google Sheets renders them; Google Vault truncated at roughly 249 matters. Security. SFTP followed symlinks in `getDocument` and composed unchecked server-supplied filenames into paths; it now also supports optional host-key fingerprint verification, which runs during key exchange before any password is sent. Trello interpolated user-supplied board ids into URL paths. Google Vault is narrowed to `ediscovery.readonly`. `getDataverseBaseUrl` accepted any host while attaching a bearer token, and is pinned to Microsoft's Dataverse domains — pre-existing shipped code, fixed here. Also adds `ConnectorAuthConfig.optional` so a public source can be configured without inventing an API key, and teaches the scope check that a granted read-write scope satisfies a required `.readonly` sibling. Microsoft Dataverse was built and then removed: its OAuth cannot complete consent. Dataverse requires a per-environment resource URI, the provider declares a static `https://dynamics.microsoft.com/user_impersonation` that is not an Entra Application ID URI, and the environment URL is only collected after the credential exists. That predates this change and also affects the 12 shipped Dataverse tools. * fix(dataverse): strip the bearer token when a request redirects The host allowlist added alongside the connector work only constrains the initial destination. `secureFetchWithPinnedIP` follows redirects and keeps the `Authorization` header unless a tool opts out, so a redirect away from an allowed Dataverse origin would forward the caller's OAuth token to whatever host answers. Dataverse redirects in normal operation — file downloads hand back a signed storage URL, and environment hosts move between regional origins — so this is reachable without a compromised environment URL. Sets `stripAuthOnRedirect` on all 18 Dataverse tools, matching the existing GitHub job-logs and Windchill precedent. * fix(connectors): address review findings on listing and hashing - microsoft-excel: `fetchWorksheets` read only the first Graph page and never followed `@odata.nextLink`. A workbook with more sheets than fit in one page dropped the remainder from the listing without setting `listingCapped`, so the sync engine reconciled those documents away as deleted. The walk now pages, bounded by MAX_WORKSHEETS, and only follows a nextLink that stays on the Graph origin, since the link is server-supplied and carries the token. - google-slides: the listing `contentHash` covered only the file id and modified time, so toggling the speaker-notes option left every stored hash matching and no presentation was ever re-hydrated with the new scope. The setting is now part of the hash, in the single shared stub builder so the list and hydrate paths stay identical. - mintlify: `pathPrefix` filtered with a bare `startsWith`, so a prefix of `/guides` also matched a sibling like `/guides-archive`. It now shares the `/`-boundary rule `withinBasePath` already used, extracted as `isUnderPath`. * fix(connectors): list newest first in zoho desk, accept a trailing slash prefix - zoho-desk: `sortBy: 'createdTime'` is ascending — Zoho denotes descending with a `-` prefix — so the default 500-record caps kept the oldest tickets and articles and recent ones were never listed. Because the cap sets listingCapped, that stale tail could not reconcile away either. Now sorts `-createdTime`. Still ordering on createdTime rather than modifiedTime, so rows do not reshuffle mid-walk. - mintlify: `resolvePathPrefix` kept a trailing slash while `isUnderPath` accepts an exact match or `prefix + '/'`, so `/guides/` matched neither `/guides` nor `/guides/intro` and the source synced nothing. A regression from the previous round, which replaced a bare `startsWith`. The prefix is now normalized before comparison. * fix(dataverse): strip the bearer token on the upload route's own redirect `upload_file` posts to an internal route rather than calling Dataverse directly, so the tool-level `stripAuthOnRedirect` added in 903c94e9 only covers the same-origin hop into that route. The route's own outbound PATCH carries the caller's OAuth token and left redirect stripping at its default, so a redirect to a signed storage host — which is exactly how Dataverse serves file operations — would have handed that host a reusable credential. The other 17 tools build the Dataverse URL directly, so the tool-level flag already covers them.	2026-08-14 14:14:43 -07:00
Waleed	cf78946529	fix(v2): close seven correctness and honesty gaps found sweeping the API (#6702) ... * fix(v2): close seven correctness and honesty gaps found sweeping the API A ten-slice sweep of the live v2 surface turned up no regression from the recent cancellation work, but did surface a set of pre-existing defects where an endpoint either lost data, hid a failure, or reported something that was not true. Each is fixed at the layer that owns the behavior. Terminal execution logs. The two force-fail boundaries wrote `status: 'failed'` without `ended_at` or `total_duration_ms`, so a force-failed run dropped out of every duration-filtered log query — the same defect class already closed for cancellation, still open on its sibling. The cancellation payload factory is generalized to take the status; the cancellation call sites are untouched and still emit a byte-identical row. Custom tools. One malformed row failed the whole page, and because the list is keyset-paginated that row made every page containing it permanently unreachable. The projection now validates against the same contract schema the route builder applies, repairing only what can be repaired without inventing information — a stringified schema, and a missing `type` whose contract admits exactly one value — and omitting with a warning what cannot. Both rows observed in production are recovered rather than discarded. Table filters. `eq`/`ne`/`in`/`nin` compiled a wrongly-typed operand into a containment test that silently matched nothing, so a filter written against the value the write path had stored returned an empty page instead of its rows. The operand is now read through the same column-type registry the write used, and rejected only where that registry refuses it. Range operators already behaved this way; `null` and the cleared-cell sentinel still pass through untouched. Error messages. A custom `error` on a string schema also replaced the wrong-type wording, so supplying a number for a name reported that the name was missing. Messages now distinguish an omitted field from a mistyped one, `topK` names its own bounds, the knowledge search refine reports against a field rather than the whole body, and a workspace id is bounded before it reaches a lookup. Archived file metadata. A soft-deleted file was listed but unreadable, leaving no way to check share state before restoring it. The read takes the same `scope` selector the list already exposes; the default is unchanged, and the parameter relaxes only the `deleted_at` predicate, never the authorization. Cancellation reporting. Cancelling an already-terminal run reported a durable write that never happened. The service now distinguishes the no-op and names the state it observed, and both surfaces present one vocabulary instead of the internal route deriving its own. No claim predicate or write changed. Protocol. A 401 carries a challenge naming the header the API actually reads, and a body that failed to parse is reported as an unsupported media type only when the caller positively declared a non-JSON one — after the read has already failed, so nothing that succeeds today can begin to fail. * fix(v2): correct three regressions this branch introduced, and harden its tests Adversarial review of the previous commit found that three of its "behavior preserving" claims were wrong. Each is corrected here at the layer that owns it. Table filters no longer coerce a `date` operand, and no longer throw. `date` is the one column type whose registry `coerce` is not idempotent — it drops sub-second precision — and the leaf that compiles a filter also builds the unique-constraint and upsert-conflict probes, so re-reading an already-coerced operand could stop it matching the row it was written from and admit a duplicate inside the write transaction with no error. Throwing was the second mistake: the v2 predicate grammar type-checks structure but not operand values, so a rejected operand no longer failed at submission but inside the delete, update, dispatch and cancel runners, where a filter that cannot compile means the cells it started can no longer be cancelled. Coercion is now total — it rewrites what the registry accepts and passes everything else through unchanged, exactly as before. Reviving a force-failed run no longer inherits its terminal duration. Writing `ended_at` and `total_duration_ms` on the force-fail boundary was correct in isolation, but a partial resume flips that row back to `pending` and those columns survived. The preserved value is meant to be the pause checkpoint — the run's active time — and it had become wall clock measured at the failed resume, which the checkpoint rule then faithfully carried into the next terminal write. The revival clears them only for a row that was terminal, so an ordinary paused row keeps the checkpoint it is supposed to keep. Cancelling reports the terminal state it actually observed. Reclassification now requires that nothing else went wrong, so a genuine paused-reconciliation failure survives instead of being rewritten as an already-terminal no-op, and the claim's own row count — not a snapshot read before it — decides whether this cancel terminalized the run or lost a race to something else. The status the snapshot needed rides along on the ownership query that already reads the row, rather than the second read that query's own contract warns against. A custom tool that cannot be projected now answers the same way everywhere: the list omits it, and reading or patching it by id reports it as absent rather than as a server fault. Analytics stops reporting a cancellation for a request that cancelled nothing. The tests around all of this were audited by mutating each fix and checking the suite noticed. Where it did not, the assertion is stronger now: the absent content-type branch is genuinely exercised rather than relying on a header the client library supplies, the duration encoder is pinned to the column it must measure from, execution ownership is pinned to both ids it must match, and the archived-file concealment test proves it conceals the archived read specifically. Two tests that asserted a paused branch they could not observe are gone; the rendered-SQL test that can decide it already covers them. * fix(execution): report a workflow-group cancellation as the write it performed Cancelling a workflow-group run whose log had already been cancelled, but whose cell sidecar still needed reconciliation, durably cancelled that sidecar and then reported `already_cancelled` with `durablyRecorded: false` — because the terminal-status shortcut answered from the entry snapshot alone and never asked what this request had written. The analytics event, which now gates on that field, stopped firing for a cancellation that really happened. The outcome a cancel reports is the same question whichever path answers it, so there is now one vocabulary for it rather than one the direct claim tracked and one the group transition did not. Every group result maps to that outcome through a total map, so a new group result cannot compile without deciding what it wrote, and the reclassification leads with whether this request wrote at all. A group transition that reports itself already cancelled is deliberately mapped as unknown rather than as a no-op: it leaves the sidecar alone but still terminalizes a log that was active, and the result does not say which happened. That costs nothing today, because the only snapshot that would reclassify proves the log was already terminal. * fix(execution): have a workflow-group cancellation report the writes it made Three review findings landed on the same reporting logic, each a different face of one cause: the caller could not see what the group transaction had written, so it inferred. It inferred from an entry snapshot, then from the returned kind, and the remaining blind spot was the kind that covers two different transactions — a repair that terminalizes an active log, and a genuine no-op — which left a cancel that wrote nothing still claiming a durable write when it lost a race. The transaction now reports both writes it can make, each read from that statement's own returning row and recorded immediately before the throw that already depended on it, so the report cannot drift from the write. The caller derives its outcome from those rather than from the kind, and the kind is back to naming the situation instead of standing in for the work. The group path can now always answer whether it wrote. The only remaining unknown is the direct claim when its update throws or is never attempted, which genuinely has no row count to report.	2026-08-14 14:14:30 -07:00
Vikhyath MondretiandClaude Opus 5	237f973a11	fix(condition): stop a secret value from breaking or forging a condition (#6705) ... Condition expressions pasted every environment variable value into the expression as source. Block references in the same expression go through a proper escape and get quoted; env vars went through neither. That left three defects: - A bare string placeholder was a SyntaxError. `{{NAME}} === 'alice'` resolved to `alice === 'alice'`, so the form the Function block docs recommend could not be used here at all. - Ordinary data broke the block. An apostrophe (`O'Brien`) or a newline in a legitimate value produced unparseable source and failed the run. - The quoted form was injectable. A value of `x' || true || '` turned `'{{NAME}}' === 'bob'` into `'x' || true || '' === 'bob'`, forging a true branch out of a comparison that should be false. Inline only structurally inert literals — numbers, booleans, and null, with optional space/tab padding. Every other value keeps its `{{NAME}}` placeholder and is bound as a string by the execution-boundary compiler, the same one Function blocks and Custom Tools already use. Legacy outcomes are preserved. `{{COUNT}} === 3` and `{{ENABLED}} === true` still compare as literals, and an embedded `"Bearer {{API_KEY}}"` still compares equal — now via compiled concatenation rather than a pasted value. Padding is admitted rather than trimmed so the inlined text stays byte-identical to the stored value, which is what keeps a padded number correct both bare and quoted. A resolved secret also no longer travels to the execution boundary inside the condition source. The one deliberate behavior change: a value whose text is itself a quoted JS literal (a secret stored as `'foo'`, a plausible workaround for the bare-string SyntaxError) now compares as the 5-character string rather than as source. That form is the injectable one, so it cannot be kept. Docs: state the placeholder type contract, which was described mechanically but never in terms of what a reader gets. `{{KEY}}` in Function and Custom Tool code always evaluates to a string, so a bare `if ({{FLAG}})` is always true and a list has to be stored as JSON. This is what a customer hit after the resolver lift in #6247 moved Function blocks off source inlining. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-14 13:50:56 -07:00
Waleed	a7115e87ee	feat(integrations): add Azure Data Explorer (#6701) ... * feat(integrations): add Azure Data Explorer Add a 14-operation Azure Data Explorer (Kusto) integration covering KQL queries, schema and metadata discovery, table management, inline and query-sourced ingestion, ingestion-failure triage, and arbitrary management commands. Authentication uses a Microsoft Entra service principal through an internal proxy route, since the Kusto token audience is per-cluster and cannot be expressed as a static-scope OAuth provider. * fix(azure-data-explorer): only read partial-failure status from the QueryStatus table Scanning every returned table for Severity and StatusDescription columns misread an ordinary query as a failed request whenever the user's own result selected columns of those names — a common shape for a log table. Failure detection now consults only the table the response's table of contents names as QueryStatus, and primary-result selection reuses the same index instead of re-reading it. * fix(azure-data-explorer): keep the Show Operations and Show Table Details cards from painting empty check:canvas-sentences flagged the Show Operations sentence: it anchored `core` on operationId, which is an advanced-mode optional field, so an untouched card resolved to nothing and painted empty. Show Table Details had the same shape in milder form — table is optional there, since omitting it describes every table, leaving a dangling preposition. Both now lead with literal copy and treat their field as an optional refinement. Also simplifies the primary-table condition to a single `!= null` check. * fix(azure-data-explorer): authenticate sovereign clusters against their own Entra authority The cluster allowlist accepted Azure China and US Government hosts, but every token request went to login.microsoftonline.com. Those clouds are isolated instances with their own Entra endpoints, so a sovereign cluster passed URI validation and then could never obtain a token. Each Kusto service domain is now declared alongside the authority that issues tokens for it, so the two cannot drift apart, and the authority is part of the token cache key. * improvement(azure-data-explorer): warn that ingest-from-query matches columns by position Kusto aligns an ingested query result to the target table on column type and order, never on column name, so a query projecting the right columns in the wrong order lands data in the wrong columns without erroring. Surfaces that in the tool description and param the model reads, in the wand prompt that generates the query, in the rollup skill's steps, and in the docs. Also verifies the target schema first rather than after. * chore(azure-data-explorer): drop the unsourced kustomfa host from the cluster allowlist Every other entry traces to a Microsoft reference — the Kusto connection-string doc, the national-cloud endpoint tables, and the Fabric KQL-database REST reference. kustomfa.windows.net does not, and the connection-string doc states the trust boundary as hostnames ending in kusto.windows.net. An allowlist should only hold hosts we can justify, so this drops it and records the sourcing standard for anything added later. * fix(azure-data-explorer): handle commas inside quoted properties and empty extent IDs Two defects in the shared command helpers: buildWithClause split the property list on every comma before validating, so a value that legally contains one — a docstring sentence, or a tags array with more than one entry — was torn in half and rejected. Splitting is now quote-aware, and an unterminated quote is rejected outright rather than swallowing the rest of the clause. transformColumnListResponse dropped empty strings, but `.ingest inline` reports "no data shards were generated" as a single record carrying an empty extent ID. A no-op load therefore looked like a missing column instead of an empty result. Only non-strings are skipped now.	2026-08-14 12:27:13 -07:00
Waleed	1d342722ad	feat(rabbitmq): add RabbitMQ integration (#6700) ... * feat(rabbitmq): add RabbitMQ integration * fix(rabbitmq): strip auth on redirect, require https, and bound the retrieval response * fix(rabbitmq): reserve message metadata in the retrieval response budget	2026-08-14 12:05:01 -07:00
Waleed	7f64d5e600	perf(tables): stop a table write refetching every loaded page in the tab that made it (#6698)	2026-08-14 11:13:53 -07:00
Vikhyath Mondreti	fa394e5e07	fix(execution): resolve secrets against the acting principal, not the workflow owner (#6690) ... * fix(execution): resolve secrets against the acting principal, not the workflow owner * fix(execution): resolve anonymous public-API runs as the workspace billing account * fix(execution): propagate run identity across dispatch paths and scope public runs to workspace secrets	2026-08-13 18:50:15 -07:00
Waleed	9fc65863cd	fix(v2): hold create to the rules update enforces, and bind the last two cursors (#6684) ... Three defects, one shape: a rule applied to one path and not its sibling. Two were found by probing the live surface after the previous fixes deployed, and the third by reading for the pattern. Creating a workflow group through the public surface validated almost nothing the update path validates. An enrichment group could name an enrichment the registry does not define, or an output the enrichment does not have, or carry no output id at all — each a 201 storing a column no run can ever write, discovered only when the caller later tried to edit the group and got the 400 create should have given. The workflow half was the same: a fabricated block-and-path coordinate was stored on create and refused on update. Create now runs the same two registry helpers and the same workflow-output check the update path uses. The discriminator there is the backing workflow id, not the declared type. The workflow sidebar creates enrichment-template groups labelled `enrichment` while backed by a real workflow and carrying no enrichment id, so keying on the label would have refused the first-party create path outright. A group's producer type could also be relabelled after the fact into a state creation refuses. Nothing rejected it and nothing could repair it, since the update body carries no enrichment id to supply. Relabelling an enrichment group as workflow-backed is the harmful direction: it keeps the enrichment id while moving the group onto the workflow branch with an empty workflow id, so every cell run fails. An update may now only restate the type the group already has. The workflow-version and workspace-member lists were the last two paged reads minting cursors with no route identity, so a token from one parent resumed another at a position that silently skips rows — the defect the previous change closed everywhere else. Both now wrap their domain token with the same scope binding, and the pagination guardrail gained a declaration of every nested list's parent path param, because the old one recorded only query filters and so could not tell an unfiltered list from a forgotten parent.	2026-08-13 17:01:18 -07:00
Waleed	ab8a64fcae	fix(v2): close the defects live probing found (#6681) ... * fix(v2): close the defects live probing found Staging finally deployed the merged release, so the surface could be exercised for real. Every fix already shipped held up. These are the defects only live traffic surfaced, plus the ones a static sweep had found and left. A cursor named a position in a sequence without naming the sequence. `cursorScopeKey` hashed only the caller's filters, so any two lists filtering on nothing but `workspaceId` produced one fingerprint and accepted each other's tokens: a tables cursor replayed against the knowledge list answered 200 and silently skipped a row. Table rows never reached that check at all, so a cursor from one table paged another. Identity now comes from the route's own contract — method plus resolved path — because a hand-written name is the step an author forgets, and forgetting it is invisible. An unresolved path placeholder throws rather than fingerprinting the template, so a misconfigured route fails on every request instead of an unlucky one. Every token minted before this is refused with an accurate message; they are single-walk and unpersisted. Knowledge search and the document list answered different questions. Search grouped same-tag filters by slot and joined them with OR while the list conjoined every filter, so `gte 9` and `lte 2` on one tag returned nothing from the list and a full billed page from search. Search now conjoins. The OR grouping replaced an explicit `|OR|` mechanism that was deleted outright, was never documented in any contract, and cost the ability to express a range on a single tag; the union it gave is still reachable as separate searches. The search body also accepted an unbounded query that was billed and then silently truncated to the embedding model's window, and ignored the tag-filter cap the list enforces. A body over ten mebibytes was reported as malformed JSON. Next's proxy truncates there, well under this app's fifty-megabyte ceiling, so the parse failed on a body the caller sent whole and the size branch was unreachable. The ceiling is now clamped to what the proxy will pass. Also: a group's output columns accepted a `workflowGroupId` and discarded it; an enrichment group could never gain an output, because a new output coordinate demanded workflow metadata a group with no workflow cannot have; `newOutputColumns` alone reported success and created nothing; a saved view stored layout references to columns that do not exist while refusing the same name in a filter; an MCP server stored `retries: 0` as three and overrode an explicit auth type; a disabled server answered tool discovery with an unclassified fault; rotating a header server's headers left it reading connected; and a run whose workflow was deleted reported the root folder path while also reporting the workflow deleted. Where the honest fix was out of reach, the contract was corrected instead of half-fixing the code: the polled run resource rebuilds `error.code` by matching the persisted message, so it can never report the two codes that need block attribution, and now says so. `OUTPUT_TOO_LARGE` is removed — no path ever emitted it. `triggers` was left alone deliberately. It reads as a closed enum but production holds 43 distinct values, because a webhook run stores its provider id; pinning the enum would refuse legitimate history a log search exists to find. The description now says the vocabulary is open. * fix(v2): clamp explicit body caps to the proxy ceiling too The previous commit clamped the default JSON body cap but left explicit per-route overrides alone, so a route declaring a larger `maxBodyBytes` still fell into the truncation it was meant to report: the four inline workspace-file routes at 70 MB and the deployed-chat route at 220 MB. Next attaches `proxyClientMaxBodySize` to every request and clones the body unconditionally for any non-GET method on a matched path, pushing EOF at ten mebibytes with only a warning, so the handler reads a truncated prefix. Those routes therefore already fail above that size — as a malformed-JSON 400. Clamping the effective limit inside the two body readers makes the same request fail as payload-too-large, quoting the limit actually in force. One existing test asserted the unreachable case, allowing a sixty-mebibyte base64 body; it now asserts what the proxy will forward intact. The inline-file path still advertises fifty mebibytes and cannot exceed the proxy ceiling until that ceiling is raised, which changes buffering for every route and belongs in its own change. * fix(v2): close the two holes the first review round found Both are places where a fix in this branch shut one door and left a smaller one open in the same wall. Letting an enrichment group gain an output meant skipping workflow resolution — but that resolution was the only thing validating a new output, so a PATCH began storing coordinates the runner can never fill. It fills a cell from `result[out.outputId]` and skips an output with no `outputId` at all, while the writer diffs on that same id and the sidebar reads and writes by it; the contract leaves it optional. The regression test added with that fix was itself asserting such a dead coordinate. Create's registry checks are now two shared helpers both paths call, and on update an output is exempt only when an identical binding already existed, so renaming a group whose enrichment has since changed still works while anything added or repointed must name a real output. `mappingUpdates` on an enrichment group now says it is inexpressible rather than resolving an empty workflow id into a missing workflow. The layout-reference check was handed the tolerant column set, so a placeholder minted to keep a dangling filter writable also whitelisted a brand-new layout reference — storing an entry the next read discards, which is the inconsistency the check was added to remove. Layout now resolves against the live columns, which is exactly what pruning keeps, while filters and sorts keep the exemption they need.	2026-08-13 16:12:40 -07:00
4 people	1424809614	feat(netsuite): add Oracle NetSuite integration (#6476) ... * revise netsuite integration * fix(netsuite): align selector route with snowflake * test(netsuite): remove selector route coverage * test(netsuite): align coverage with snowflake * fix(netsuite): complete integration validation * refactor(netsuite): align integration with codebase patterns * test(netsuite): correct async job citation * fix(netsuite): address final audit findings * fix(netsuite): surface upsert/transform Location, relax task link check Oracle documents the Location response header for create and update, and both tools already require it. Upsert and transform also produce a record but Oracle documents no response headers for either, so they dropped the header entirely and the new record's ID was unreachable. Add a `resource-optional` location mode that captures Location when NetSuite sends it and never fails when it is absent, and wire it to upsert and transform along with their tool and block outputs. Async task discovery rejected the whole response if any task link carried a rel other than `self`, collapsing the picker into a 502. Oracle documents a `self` link per task but never guarantees it is the only one, so skip other relationships and fail only when no self link exists. Also use the shared `truncate` helper in the error sanitizer per the repo convention instead of an inline slice. * fix(netsuite): validate SuiteQL pages against their documented shape The shared collection-page validator required links, items, count, hasMore, offset, and totalResults on every 200, and a missing field turns a successful call into a reported failure. Oracle documents all six for record collections and SuiteAnalytics dataset pages, but its SuiteQL reference lists only links, count, offset, totalResults, and items. A documented SuiteQL response that omits hasMore would therefore have been rejected. Split out a suiteql-page validator that requires the five documented SuiteQL fields and type-checks hasMore only when the account returns it. Record collections and dataset pages keep requiring all six. * chore(netsuite): regenerate tool metadata after rebase on staging The rebase conflicted only in the generated tool-id, tool-metadata, and tool-output artifacts, which NetSuite and the newly landed LogRocket integration both extend. Regenerated from the merged registries: the result is staging's catalog plus the 27 NetSuite tools, with LogRocket's entries intact and no other tool changed. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Bill Leoutsakos <billleoutsakos@Mac.localdomain> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-08-13 15:47:03 -07:00
Waleed	58b5ee9337	feat(logrocket): add LogRocket integration (#6678) ... * feat(logrocket): add LogRocket integration * improvement(logrocket): fail loudly on a non-numeric identify timestamp and cover releases in the catalog copy * fix(logrocket): require a highlights identity and treat pagination cursors as opaque strings * fix(logrocket): trim request fields so whitespace-only input fails validation * fix(logrocket): declare the release version in the block inputs map	2026-08-13 15:01:22 -07:00
Waleed	e2b7335644	fix(v2): stop telling callers something the server did not do (#6676) ... A works-as-advertised sweep of the v2 surface found one defect class in five places: input is validated for shape, then its meaning is re-derived independently by each consumer — so a filter compiles differently than it validated, or a write commits and the response then reports failure. Knowledge tag filters were validated once and re-parsed three times. The document list read `Number()`, search read `parseFloat()`; the list matched booleans case-insensitively, search compared against the literal `'true'`; the list escaped LIKE metacharacters, search did not; and the date pattern was tested against the untrimmed string the validator had already trimmed. Two of those paths dropped the predicate entirely and answered 200 with the whole knowledge base — on a billed endpoint. Values are now coerced once, where the resolved field type is known, and both builders consume the result. A builder that cannot compile an already-validated filter now raises instead of silently widening the result set. `PUT /api/v2/secrets/{name}` with `scope: personal` committed the secret and then answered 500, because a user-global write was reported through a workspace-scoped mirror lookup, and an org admin's inherited access has no `permissions` row for the fan-out to find. The personal path no longer decides success from a per-workspace mirror. The `workspaceId` descriptions said a personal secret lives in one workspace; it does not, and they now say so. A custom tool could be stored with a schema the read path cannot serialize — `POST /workflows/import` and Copilot both wrote through name-only checks — so one row made the whole workspace list 500, and a title-only PATCH committed, audited, then reported failure. Every write now passes the same guard the response schema is derived from. `POST /api/v2/tables` accepted `workflowGroupId` on an initial column. Nothing can populate it legitimately, and it made every later column-add and group-add fail with no way to clear it. The key is refused at the boundary, and `createTable` now runs the invariant every later mutation already runs, closing the internal and v1 ingresses too. Those invariants moved to a leaf module: reaching them through `workflow-columns` pulled the executable tool registry into the tables page graph, taking it from 1,767 modules to 6,999. An out-of-range upload part number answered 500 rather than the 400 its published contract promises, because the throw happened above the route's try/catch and was not an `HttpError`.	2026-08-13 14:19:41 -07:00