mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
025ea4d2bd071100b33f1b2d4132bb7881310bd7
892
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Waleed
|
025ea4d2bd |
fix(docs): serve JSON-LD in the HTML, fix sidebar spacing, and tighten the CLI guides (#6763)
* docs(cli): use -g for the install, and cut the prose that was not pulling weight
`--global` is valid but `-g` is what every comparable CLI documents, and the
long form only came from the package README. Also drops the yarn tab: it read
`yarn global add sim`, which works on Yarn 1 only — Yarn 2 removed global
installs, so that command fails for anyone on a modern Yarn. Adds `npx sim` for
running without installing.
The guides had accumulated design rationale that belongs in code comments rather
than user docs — why the filter grammar is JSON, why the config section naming
is asymmetric, why an unexpected error keeps its stack trace. Surveying how gh,
Vercel, Turborepo, Deno, Bun and Supabase write theirs, none carry that kind of
justification, and callouts are reserved for content whose absence produces a
wrong result rather than for general asides.
So: 1016 lines to 763, and 12 callouts to 3. The three that remain are the
pairing-code check, that `sim logout` does not revoke the key, and the
`--limit 100` default on `batch-delete`/`batch-update`, which silently truncates
a larger match. Troubleshooting drops the entries whose error message already
contained its own fix and keeps the seven whose cause is not obvious.
* fix(docs): render JSON-LD as native script tags so it reaches the HTML
All four structured-data blocks — WebSite, TechArticle, BreadcrumbList,
SoftwareApplication — were rendered with `next/script`, which never emitted a
script tag. Measured on a production build, `/api-reference/getting-started`
contained zero `<script type="application/ld+json">` elements; the payload
existed only in the `__next_s` client-injection queue and the RSC flight data,
so anything reading the served HTML saw no structured data at all. React was
also logging "Encountered a script tag while rendering React component" on every
page.
`next/script` is for loading and executing JavaScript. JSON-LD is data, and
Next's own guidance is a native `<script>` in the component. `serializeJsonLd`
already escapes the `<` character to its unicode form, which is the
sanitization that guidance calls for, so only the element changes.
Same build, after: three valid tags per page with `WebSite` in `<head>`, and the
injection queue gone entirely.
* fix(docs): scope the flush-separator rule to a container's first separator
`[data-separator]:not([data-separator] ~ [data-separator])` was meant to keep the
first sidebar group flush against the top padding, but `~` only reaches siblings,
so it also matched the first separator inside every expanded folder. Under
Self-Hosting, "Install" lost its top margin and crowded the "Architecture" link
above it — 25px of gap where "Configure" and "Operate" below it had 40px.
`:first-child` expresses the intent directly. Only the four sidebar roots open
with a separator; every nested folder starts with a page, so the intended case
still goes flush and nothing else changes.
* fix(docs): move the flush-separator rule onto the separator component
Keeps the styling with the component that owns it, per the repo standard, and
lets the global rule be deleted outright rather than corrected — `global.css`
now only loses a rule in this PR. Tailwind's `first:` variant compiles to the
same `:first-child` selector, so behavior is unchanged: the build emits
`.first\:mt-0:first-child{margin-top:0}` and the prerendered HTML carries the
class on the separator.
|
||
|
Waleed
|
257029a60c |
feat(microsoft_ad): licensing, security, audit, role, and device operations (#6742)
* feat(microsoft_ad): licensing, security, audit, role, and device operations
Deepens the Microsoft Entra ID block from 12 to 36 tools against the Microsoft
Graph v1.0 reference: license assignment and tenant SKUs, password set/reset,
sign-in session revocation, authentication methods, sign-in and directory audit
logs, app role and directory role assignments, service principals, device reads,
and conditional access policy reads.
Device write (device-update, device-delete) is deliberately excluded. Both
document Directory.AccessAsUser.All as their only delegated scope, with the
higher-privileged read documented as unavailable, so supporting them would mean
requesting tenant-wide act-as-the-user directory access for two operations that
additionally require the caller to hold Intune Administrator.
Also drops an undocumented ?$select= from create_user that was silently nulling
department and accountEnabled in the response.
* fix(microsoft_ad): resolve OData filter and search by owning operation
The params mapper assigned result.filter from each filter subBlock in turn, so
the last non-empty one won regardless of the selected operation. Because a
subBlock keeps its value after the operation changes, a filter written for one
endpoint was sent to every other collection operation — invalid OData against a
different Graph resource, or a silently wrong page.
Resolves the filter and search terms from an explicit operation-to-field map
instead, so each operation reads only the field it owns.
* fix(microsoft_ad): clear non-owning filter and search on the merged inputs
The executor merges { ...inputs, ...transformedParams }, so declining to copy a
stale filter is not enough — the serialized value survives the merge and still
reaches the tool. Advanced-mode subBlocks are serialized on non-emptiness alone
and never have their condition evaluated, so the value is present even when the
field is hidden.
Write filter and search on every operation, as undefined when the operation owns
neither, so the merge clears them.
* fix(microsoft_ad): clear the MFA flag and let paged user operations continue without a User ID
The set_password MFA dropdown only wrote its key when non-empty, so the "No Change"
empty string survived `{ ...inputs, ...transformedParams }` and reached Graph in place
of a boolean. Assign it explicitly, including as `undefined`, the same way `filter` and
`search` are handled.
`list_user_app_role_assignments` and `list_user_devices` page by `@odata.nextLink`, and
both tools already treat `userId` as optional once a continuation URL is supplied. Drop
them from the required set when Next Page is filled in so pagination-only runs pass block
validation.
Also note on the reset_password output that a generated password reaches workflow outputs,
run history, and the model, matching how other tools that return secrets document exposure.
* fix(microsoft_ad): require the service principal ID only on the first page
Every other single-resource ID field pairs its condition with a matching required
rule; servicePrincipalId had none, so a first-page run could pass block validation
with an empty ID and fail inside the tool instead. Require it unless a continuation
URL is supplied, matching the paged per-user operations.
* fix(microsoft_ad): reject a continuation URL from a different collection
Every paged operation reads the one shared Next Page field, and a subBlock keeps
its value after the operation changes. Paging /users and then switching the block
to /devices short-circuited back to the user page, silently returning the previous
collection instead of the selected one.
Assert the continuation URL's terminal path segment against the collection the tool
actually reads, which also rejects a nextLink pasted from an unrelated response.
|
||
|
Waleed
|
fed891f69d |
docs(cli): add a CLI docs section generated from the command tree (#6762)
* docs(cli): add a CLI section, generated from the command tree
The `sim` CLI shipped with no coverage in the docs site. Adds a fourth
top-level tab for it, and moves Academy last.
The command reference is generated. `sim` exposes 147 leaf commands across
33 groups, most of them derived at runtime from the v2 route contracts, so a
hand-written reference would be wrong the week after it was written. The
generator walks the command tree `buildProgram()` hands to commander — the
same tree the terminal parses — rather than re-deriving it from the contract,
which would be a second implementation free to describe commands nobody can
invoke. `check:cli-docs` is a zero-arg `check:*` script, so the existing audit
runner picks it up and stale pages fail CI.
Generating against the real tree surfaced a collision it had been hiding:
`bulkUpdateKnowledgeDocuments` and `updateKnowledgeDocument` both derived to
`sim knowledge documents update`. Commander resolves a duplicate to the first
match, so the bulk form shadowed the single-document one and its flags were
unreachable while still appearing in `--help`. The bulk form is now
`batch-update`, matching how `tables rows batch-delete`/`batch-update` already
handle the same REST overload, and the generator fails on any duplicate path
so the next one cannot land silently.
Five hand-written guides cover install, auth, configuration, output formats,
and scripting. Also corrects two commands in the package README that do not
exist as documented (`tables columns <tableId>`, and `--sort score:desc`,
which is JSON).
* docs(cli): document every flag from the contracts, add troubleshooting and a single-page reference
The command reference was structurally complete but said almost nothing: 223 of
377 flags rendered as "Set sort by" because the CLI only ever read flag help
from its own contract overrides, and fell back to restating the flag name.
The prose already existed. The v2 route contracts carry 931 `.describe()` calls
and the OpenAPI specs publish all of them — 327 parameters and 282 body
properties, 100% coverage — but the generated operation table dropped every one,
carrying only a per-operation summary. It now carries the field descriptions,
the path-parameter descriptions, and positional help, so `--help` and the docs
explain a flag the same way the API reference does. Placeholder descriptions are
now zero, and 147/147 commands, 377/377 flags and 130/130 arguments are
documented.
`check:cli-docs` fails on a request field with no `.describe()` rather than
letting it render as documentation that says nothing.
Also in this pass:
- Commands are root-level sidebar entries under a Commands heading rather than
a folder, and headings are the command's description, so the table of
contents distinguishes entries at the first word instead of repeating
"sim knowledge documents …" fourteen times. A guard fails the build if two
descriptions on a page collide, since they would share an anchor.
- A single-page `Complete reference` carrying all 147 commands, for in-page
search and for agents fetching `/cli/reference.mdx`. It keys on exact command
paths because descriptions are only unique within a group.
- A troubleshooting page, with every message copied from the source.
- Table columns are sized by a local component; the flag column was starved
while descriptions kept most of the row empty.
- The prerelease install channels are dropped from the docs and the package
README, which is what npm renders.
* fix(docs): match the CLI tab by path segment, and escape backslashes before pipes
`pathname.includes('/cli')` also matches `/integrations/clickup` and
`/integrations/clickhouse`, so both existing integration pages lit the CLI tab
and unlit Documentation. Matching is now per path segment. Anchoring to the
start would not work either — a non-default locale prefixes the path, as in
`/ja/cli` — so the segment is matched wherever it sits.
Table cells now double a backslash before escaping pipes. A value ending in one
turned `a\` + `|` into `a\\|`, which the table parser reads as an escaped
backslash followed by an unescaped pipe, splitting the cell early. Nothing in
the command surface contains a backslash today, so this was latent rather than
visible.
The reference page's global options table is two-column and was being wrapped in
`CommandTable`, which sizes the second column for the `Required` cell of the
three-column tables and crushed the description into 5.5rem. It now matches the
overview page, which leaves that table unsized.
|
||
|
Waleed
|
6a29a9e2f4 |
feat(mssql): add Microsoft SQL Server integration (#6739)
* feat(mssql): add Microsoft SQL Server integration
Add a Microsoft SQL Server block backed by six tools (query, execute,
insert, update, delete, introspect), mirroring the existing PostgreSQL
and MySQL integrations.
Connections go through the `mssql` (Tedious) driver: `connectionTimeout`
is top-level while `encrypt`, `trustServerCertificate`, and
`instanceName` live under `options`, and `port` is omitted when a named
instance is used. Values are bound as `@paramN` via `request.input()`;
no user value is interpolated into SQL. Identifiers are bracket-quoted
after validation and WHERE clauses run through the shared injection
guard.
Introspection reads INFORMATION_SCHEMA plus the `sys.indexes` catalog
views for tables, columns, primary keys, foreign keys, and indexes.
The icon is a placeholder database cylinder drawn with `currentColor`
until the real brand mark lands.
Requires `bun install` for the new `mssql` / `@types/mssql` deps.
* feat(mssql): use the SQL Server brand mark on a white tile
* fix(mssql): pin the validated IP and correct the introspection catalog reads
Tedious exposes `options.connector`, a hook that replaces its own
resolve-and-connect path, so the connection can be pinned to the address
`validateDatabaseHost` already approved instead of re-resolving the
hostname. `server` stays the hostname because tedious derives the TLS
`servername` from it independently of the connector, so SNI and
certificate validation survive the pin. This brings MSSQL in line with
the PostgreSQL and MySQL tools.
Named instances are dropped: tedious resolves them with a UDP SQL Server
Browser lookup issued outside the connector, and node-mssql deletes
`port` whenever `instanceName` is set, so no configuration leaves a
named instance pinned. A named instance is reachable through its static
TCP port.
Introspection fixes:
- index key columns now filter on `key_ordinal > 0`; INCLUDEd columns
and partitioning columns both report `0` and were being returned as
key columns, ordered ahead of the real ones
- foreign keys resolve through `sys.foreign_keys` /
`sys.foreign_key_columns` rather than
`INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS`, whose join to
`TABLE_CONSTRAINTS` has no row when a key references a unique index
and so dropped the key entirely
- `is_unique` is a `bit`, which tedious maps to a boolean, so it is
coerced rather than compared
- schemas come from `sys.schemas`, which needs only `public` and carries
no metadata-visibility caveat
The WHERE-clause guard also covers `WAITFOR TIME`, `OPENQUERY`,
`OPENXML`, the legacy `master..sys*` compatibility views, and extended
and OLE-automation procedures beyond `xp_cmdshell`.
Regenerates the docs and catalog artifacts the icon change left stale.
* chore(mssql): commit the lockfile entries for mssql and its tedious dependency tree
* fix(mssql): make the Query operation genuinely read-only
The block label, tool description, and docs all present Query as SELECT-only
while the route ran whatever T-SQL it was given, so an agent picking
mssql_query because "it is only a SELECT" could delete rows. Screen the
statement for mutating keywords with string literals stripped, which also
catches the WITH ... DELETE form that a leading-token check would miss.
Also switch the tool barrel to absolute imports per the repo convention.
* fix(mssql): compose the shared WHERE guard and close the semicolon-less batch gap
The local validateWhereClause re-derived an older copy of the shared patterns
and scanned raw text, so it missed a bare 1=1 and false-positived on prose in a
quoted value. Delegate to validateSqlWhereClause, which masks string literals
first, and keep only the SQL Server surfaces it has no reason to know about.
T-SQL needs no statement terminator, so every semicolon-anchored stacked-query
check reads straight past `id = 1 DROP TABLE dbo.users`. Screen for a bare
statement-introducing keyword to close that; word boundaries leave ordinary
column names like updated_at and deleted_at untouched.
Export maskSqlStringLiterals so the dialect layer masks the same way the shared
guard does rather than carrying a weaker single-quote-only copy.
* fix(mssql): screen administrative T-SQL and reject batches in the read-only path
The previous round left two keyword lists maintained separately, and both were
short: DBCC, KILL, CHECKPOINT, USE, and DENY were in neither, so
`SELECT 1; DBCC SHRINKDATABASE(...)` and `id = 1 DBCC SHRINKDATABASE(...)`
both got through. Collapse them into one MSSQL_STATEMENT_KEYWORDS shared by the
query and WHERE screens so a keyword cannot be covered in one place and missed
in the other, and add the administrative commands.
Also reject any second statement after a semicolon in the Query path outright.
That closes SELECT 1; <anything> structurally instead of by naming the anything,
so the blacklist no longer has to be exhaustive to hold.
* fix(mssql): reject SQL comments in the read-only query path
A block comment placed inside a keyword splits it as far as a lexical scan is
concerned, so keyword coverage cannot settle whether the server rejoins the
halves. Refuse comments in the Query path instead of modelling the tokenizer.
A SELECT sent through this operation has no need for one, and Execute Raw SQL
still accepts them. Masking leaves comment markers intact, so a literal
containing -- still passes.
* fix(mssql): close the masker-desync bypasses and correct the catalog reads
Every T-SQL screen runs over the shared literal masker, which was written for
the MySQL dialect. Three ways to desynchronise it let real SQL hide inside what
the masker believes is a string, two of which survived the existing even-quote
check:
- a backslash before a quote. T-SQL has no backslash escape, so the server
closes the literal where the masker swallowed the quote and runs the rest as
code. `a='x\' DELETE FROM dbo.t WHERE b='y'` holds four quotes and masks the
DELETE out of the keyword screen entirely, so the read-only Query operation
would run it.
- a double quote inside a bracketed identifier, which the bracket rule missed
because it only looked for single quotes.
- any unbalanced double quote or backtick, which the parity check did not cover.
All three now fail closed. Introspection also filters hypothetical and disabled
indexes, which were reported as if they were live, and resolves the referenced
side of a foreign key through sys.schemas so a cross-schema reference is no
longer an ambiguous bare table name. Values bound through request.input are
serialized when they are nested JSON, which the driver otherwise rejects with a
bare "Invalid string.".
* test(mssql): cover the block param merge and the operation-to-tool map
Asserts on the merged `{ ...inputs, ...buildParams(inputs) }` the generic tool
handler forwards rather than the mapper's return, since a key the mapper omits
keeps its raw subBlock value through that merge. Pins the TLS toggles to their
string form end to end — a switch subBlock would serialize `'false'`, which is
truthy, and the route contract would coerce the user's off into on — and checks
that duplicate subBlock ids agree on their seeded default.
* fix(mssql): release a pool whose connect failed, and allow a keyword with no trailing space
Only a pool handed back to the route reaches its `finally`, so a pool whose
connect rejected leaked its tarn resources — one per attempt when a bad
credential is retried. It now closes itself, and a failure to close cannot mask
the connect error the caller needs.
The read-only screen also anchored on `\s` after the opening keyword, which
refused valid reads like `SELECT*FROM dbo.users` and `SELECT(1)`. A word
boundary accepts those while still refusing `SELECTX`, and cannot loosen the
screen — the keyword and batch checks run over the whole statement regardless.
* chore(mssql): regenerate tool metadata and the integration catalog after rebase
Artifacts rebuilt with the generators rather than hand-merged, so they carry
both the mssql entries and the tools that landed on staging in parallel.
* fix(mssql): reject a parenthesised or negated constant tautology in a WHERE clause
The shared guard recognises `OR 1` but not `OR (1)`, `OR ((1))`, `OR NOT 0`, or
`OR NOT (FALSE)` — a parenthesis or a NOT between the operator and the constant
hides it. Both patterns require the constant to be the whole parenthesised term,
so a real disjunct such as `OR (1 = priority)` is untouched.
This narrows the gap rather than closing it, and is not meant to close it: an
always-true expression is not lexically decidable in general, which is why the
WHERE screen stays documented as defense-in-depth rather than a boundary.
* chore(mssql): regenerate tool metadata after rebase onto staging
Rebuilt with the generators so the artifacts carry the servicenow and
crowdstrike tools that landed on staging alongside the mssql entries.
* fix(mssql): screen trigger and state statements, and drop the space anchor on Execute
DISABLE and ENABLE were missing from the shared statement list, so
`SELECT 1 DISABLE TRIGGER dbo.audit ON dbo.users` passed the read-only screen as
a semicolon-less batch and turned auditing off. SET, BEGIN, COMMIT, and ROLLBACK
are added with them, since session and transaction state are reachable the same
way. FETCH is deliberately left out: OFFSET ... FETCH NEXT is the standard paging
clause, and screening it would reject the ordinary paged SELECT.
Execute Raw SQL anchored its allowlist on `\s`, which refused `EXEC(@sql)` —
the ordinary form of dynamic SQL, on the one operation meant to run it. It now
uses `\b`, matching the read-only screen.
|
||
|
Justin Blumencranz
|
7f936dc02a |
feat(tooling): enforce docs freshness and modernize agent skills (#6756)
* feat(docs): fail CI when generated integration docs are stale * fix(docs): don't flag delete-then-recreated trigger pages in check mode * docs(skills): require docs:check in the integration authoring skills * chore(skills): migrate agent commands to native skills * fix(skills): clean orphaned Claude projections |
||
|
Waleed
|
8a44621382 |
feat(cloudflare): add WAF rulesets, rate limiting, Zero Trust Access, R2, Workers, and Tunnels (#6740)
* feat(cloudflare): add WAF rulesets, rate limiting, Zero Trust Access, R2, Workers, and Tunnels
Extends the Cloudflare integration past DNS/zones/cache with the security and
Zero Trust surface:
- Rulesets engine (zone-scoped): list rulesets, get a ruleset, read a phase
entry point, and create/update/delete rules. WAF managed-rule overrides are
surfaced through the http_request_firewall_managed entry point, since
Cloudflare has no dedicated overrides endpoint.
- Rate limiting (zone-scoped) via the current Rulesets-based http_ratelimit
phase, not the deprecated rate_limits endpoint.
- Cloudflare Access (account-scoped): applications, application policies,
groups, identity providers, and service tokens.
- R2 buckets, Workers scripts/routes, and cloudflared Tunnels.
Destructive operations (delete application, delete policy, revoke service
token, delete rule, delete bucket) spell out their blast radius, and every
tool branches on the envelope's success flag rather than the HTTP status.
Security events are intentionally omitted: Cloudflare exposes them only
through the GraphQL firewallEventsAdaptive dataset, whose field list is not
documented outside schema introspection.
* fix(cloudflare): correct docs drift and remove any from the tool layer
Validation pass over all 47 Cloudflare tools against developers.cloudflare.com.
- Two tool descriptions still escaped a quote as \'. That reaches the model
verbatim and truncates the generated MDX cell — the get_zone_settings
`value` output row was missing from the published docs entirely. Both are
now template literals, and the row is back.
- list_rulesets ignored pagination. The endpoint pages by cursor via
result_info.cursors.after (not page/per_page), so a zone with many rulesets
silently truncated with no way to page. Expose per_page + cursor and return
the next cursor.
- The managed-ruleset override description claimed action and enabled were
the overridable properties. They are the ones the Rulesets engine documents
at every level, but individual managed rulesets add more: an OWASP Core
Ruleset rule override also takes score_threshold. Corrected in both the
tool output description and the block's action-parameters wand prompt.
(sensitivity_level is a DDoS override, not a WAF one — deliberately absent.)
- list_tunnels/get_tunnel dropped the documented `metadata` field.
- list_r2_buckets appended order=name whenever any filter was set. `order`
only qualifies `direction`, and `name` is its sole documented value.
- Path-interpolated IDs are trimmed, so a pasted ID with trailing whitespace
no longer 404s.
- Replaced every `any` in the integration with checked types: a shared
CloudflareEnvelope plus per-resource raw payload interfaces, read through
readCloudflareResponse. The mappers in utils.ts were the widest hole —
typing them caught four real output-shape mismatches (identity provider
read_only, service token enabled, DNS record meta/priority, certificate
geo_restrictions) that `any` had been hiding.
- BlockMeta only described DNS and zone work. Added templates and skills for
the WAF, rate limiting, and Zero Trust Access surfaces the block now has.
Confirmed against the docs and left unchanged: rulesets/rate limiting are
zone-scoped and Access/R2/Workers scripts/Tunnels are account-scoped while
Workers routes are zone-scoped; tunnels live under /accounts/{id}/cfd_tunnel;
the ratelimit object is a sibling of action/expression, not nested in
action_parameters; every rate limiting period and mitigation_timeout option
matches the documented set; R2 delete returns an empty result so echoing the
requested bucket name is correct; app-nested Access policy endpoints are
current, not deprecated; and every tool fails on a 200 carrying success:false.
* fix(cloudflare): stop per-operation subblock defaults colliding on a shared id
Subblock initial values are seeded into block state keyed by subblock id —
both stores/workflows/utils.ts and lib/workflows/defaults.ts assign
`subBlocks[subBlock.id]` in a plain forEach — so two controls sharing an id
leave one stored value and the last definition in file order wins. Four ids
were duplicated with differing defaults:
- `type` was defined four times. The Access "Application Type" control is
last, so every new block seeded `type = 'self_hosted'` and the three DNS
record controls inherited it — Create DNS Record sent a Zero Trust
application type as its record type. The subblock added on this branch
broke a default on tools that shipped long before it.
- `status` was defined three times. The empty tunnel filter is last, so
List Certificates lost its `all` default.
- `proxied` was defined three times. An empty filter is last, so Create DNS
Record lost its explicit `false`.
- `action` was defined twice. The rate limiting dropdown is last, so the
ruleset-rule action input was seeded `block`, quietly making "block live
traffic" the default for a WAF custom rule the user never configured.
Give the colliding controls their own ids and map them back to the tool
params per operation, ahead of the coercions that read them, so each
operation keeps its own default. The other 17 duplicated ids agree on their
value and are left shared.
Adds tests covering each separated default plus a sweep asserting no id
carries two different seeded values, so a future duplicate goes red.
* fix(cloudflare): generate array include rules and allow bootstrapping a phase ruleset
The Access policy include wand asked for a JSON object while the tool parses
the field with parseJsonArrayParam, so generated rules failed validation.
Switch it to json-array, whose prompt reinforcement omits the object braces.
Rate limiting and WAF custom rules could only be appended to an existing
ruleset, but a zone that has never had a rule in a phase has no entry point
ruleset and returns 404, leaving no way to add the first rule. Add
cloudflare_create_ruleset for the documented POST /zones/{id}/rulesets
bootstrap, seeded with optional initial rules.
* fix(cloudflare): correct verified API defects and stop filters leaking into writes
Independent re-validation of all 48 tools against developers.cloudflare.com
turned up defects that the shipped tools would have hit on their happy path.
Delete DNS record reported every success as a failure. That endpoint is the
one Cloudflare v4 response with no envelope — its documented body is
`{"result":{"id":...}}` with no `success` — so `!data.success` was always
true. Branch on an explicit `=== false` instead.
The two replace-semantics PATCH endpoints could silently destroy live config.
Update rate limit rule defaulted a missing action to `block`, converting an
existing `log` or challenge rule into a hard block on real traffic; update
ruleset rule left action and expression optional and had no `ratelimit` or
`logging` passthrough, so updating a rate limiting rule stopped it rate
limiting. Both now require the fields the replacement needs, and the ruleset
rule carries the two nested objects through.
Access applications were unbuildable for most types: `domain` was required,
but it does not exist on the saas, app_launcher, warp, biso, dash_sso,
infrastructure, mcp, mcp_portal, or proxy_endpoint request variants. The
application type enum was also six values behind. Access group `is_default`
is an array of rule objects, not a boolean.
Purge cache merged every supplied target into one body, but the purge body is
a one-of over the five target kinds; it now names the conflict instead.
The remaining fixes are documentation drift: the priority field is MX and URI
only (an SRV record carries priority inside its content), the certificate
status filter documents only "all", the Worker tag filter takes tag:allowed
pairs, and the managed-rule override list conflated the DDoS-only
sensitivity_level with the WAF rule-level set.
Separately, controls that share a subBlock id share one stored value, and
`shouldSerializeSubBlock` short-circuits on `mode: 'advanced'` before it
evaluates `condition` — so a hidden list filter was reaching a write. A
`list_dns_records` content filter could overwrite a record's content, cache
tags could be written onto a DNS record, and the zone status enum could reach
the tunnel list, whose enum is disjoint. Filters that differ from the value
they collided with now carry their own id, remapped through one table before
any coercion. Sharings that mean the same thing everywhere are unchanged.
Aliases are cleared by explicit assignment rather than destructuring, because
the executor merges the mapper's output over the raw inputs and a merely
omitted key survives as its raw subBlock string. The tests assert on that
merged result, and three mechanical invariants now go red on a new collision:
no id spans a read filter and a written value, no dropdown id carries two
option sets, and no hidden advanced control feeds an operation that cannot
render it. That last one found the name filter reaching three list operations.
* docs(cloudflare): point self_hosted_domains at its replacement
Cloudflare deprecated the field in favour of destinations, which the tools
already surface. The output stays — Cloudflare still returns it — but the
description now says which one to read.
* refactor(cloudflare): drop a dead exception from the empty-type guard
create_dns_record now takes its record type from the recordType control,
whose dropdown has no empty option, so the operation can never reach this
guard with an empty type. Clear it unconditionally.
* fix(cloudflare): point the canvas sentences at the renamed filter controls
The list filters that were split off their write-side twin kept their old ids
in canvasPresentation, so seven clauses referenced a control that is no longer
visible for that operation — check:canvas-sentences catches exactly this, and
a broken clause fails silently on the card rather than throwing.
* fix(cloudflare): stop the rate limiting action defaulting on a replacing update
Making action required on update_rate_limit_rule was only half the fix: the
Action dropdown still seeded block for the update operation too, so an update
that edited only the threshold kept sending block and converted a live log or
challenge rule into a hard block — exactly the harm the required flag was
meant to prevent. The update now has its own control with no seeded value, so
the action is something the caller states rather than inherits.
The certificate status filter also still offered Active and Pending, which
Cloudflare does not document for that endpoint; the only documented value is
all, and omitting it returns active packs.
* fix(cloudflare): stop the Access replacements seeding a type and a decision
Same class as the rate limiting action: both Access updates are full
replacements, and the shared controls seeded self_hosted and allow for the
update operations too. Editing only a policy's include rules would silently
convert a live deny, bypass, or non_identity policy to allow — widening who
gets in — and editing an application would rewrite what it IS.
Each update now has its own required control with no seeded value, so the
type and the decision are stated rather than inherited. Regression tests
cover both, and the canvas sentence follows the renamed decision control.
|
||
|
Waleed
|
cabd2e2fc1 |
feat(datadog): extend to 40 tools and align every operation with the published OpenAPI specs (#6745)
* feat(datadog): add incidents, SLOs, dashboards, synthetics, Cloud SIEM, and APM tools
Extends the Datadog block from 12 to 39 operations, all verified against
Datadog's published OpenAPI specs:
- Incidents (v2, public beta): list, get, create, update, add todo
- SLOs (v1): list, get, create, update, delete, history
- Dashboards (v1): list, get, create, delete
- Synthetics (v1): list tests, get test, latest results, trigger, pause/resume
- Cloud SIEM (v2): search signals, get signal, update triage state, assign,
list detection rules
- APM: search spans (v2), list Service Catalog definitions (v2)
Adds tools/datadog/utils.ts so every tool builds its URL from the configured
site/region and shares the JSON:API-aware error extraction, and handles the
v1 flat vs v2 envelope shapes and cursor pagination per endpoint.
* fix(datadog): align every operation with the published OpenAPI specs
Validated all 39 shipped operations (plus the 12 pre-existing ones that had
never been spec-checked) against the DataDog v1 and v2 OpenAPI schemas.
- `POST /api/v2/downtime` requires `monitor_identifier`, so a downtime created
without a monitor id was rejected. Default to the `*` monitor tag.
- A one-time downtime schedule declares `additionalProperties: false` and
accepts only `start`/`end`; the timezone moves to `display_timezone`.
- `GET /api/v2/downtime` has no `monitor_id` filter, and the response carries
no `disabled` attribute. Downtime ids are UUID strings, not numbers.
- Drop scaffold types for operations that do not exist (metric metadata, event
query, monitor update/delete/unmute, host listing) along with their fields.
- Note that monitor mute is no longer published in the v1 specification.
- Add browser Synthetic test results, which the browser-specific endpoint
returns with its own camelCase step-count shape.
- Replace every `any` with a spec-derived interface, keeping the polymorphic
service-definition schema opaque.
* fix(datadog): remove remaining any types and declare every returned output field
Replace the six surviving `Record<string, any>` request-body and response-cast
sites with concrete spec-derived shapes, and declare the output fields that
transformResponse already returned but outputs omitted:
- create_downtime / list_downtimes: timezone, created, modified
- create_monitor / get_monitor: options, creator
- list_monitors: message, priority, options, created, modified, creator
- query_logs: content.attributes, content.tags
- update_security_signal_state / _assignee: type; assignee also gained the
archiveReason/archiveComment pair its sibling already declared
- query_timeseries: series gained the items shape it never described
* fix(datadog): stop dropping downtime targeting inputs in the block mapping
create_downtime accepts monitorTags, timezone and muteFirstRecoveryNotification,
but the block exposed no inputs for them and never forwarded them. Monitor-tag
targeting silently fell back to the `*` tag, so a downtime meant for one team's
monitors muted every monitor in scope. Adds the three advanced sub-blocks and
wires them through.
Also routes list_downtimes' currentOnly through toSwitchBoolean. A switch yields
the strings 'true'/'false', and 'false' is truthy, so turning the toggle off
still sent current_only=true. Every other switch in the block already used the
helper; this was the last raw one.
* fix(datadog): correct metric type codes, stop SLO update data loss, drop unpublished mute
Independent re-validation of all 39 operations against the DataDog/datadog-api-client-go
generator specs (v1 and v2 openapi.yaml) rather than the client-rendered docs site.
Correctness:
- submit_metrics sent inverted MetricIntakeType codes (gauge as 0/unspecified, rate as 1/count,
count as 2/rate), silently changing how Datadog aggregated every submitted series. The spec
enum is 0 unspecified, 1 count, 2 rate, 3 gauge; an unrecognized type is now omitted so
Datadog infers it. Also stops stamping an invented `resources: [{name:'host'}]` default and
now forwards `interval`, which Datadog requires for count and rate metrics.
- update_slo replaced the whole SLO with only the fields the caller filled in, so editing one
field erased description, tags, query, monitor_ids, groups, thresholds, and timeframe.
PUT /api/v1/slo/{slo_id} is a full replacement, so the stored SLO is now read first and the
supplied edits are overlaid onto it, with the read-only fields stripped.
- update_incident admitted empty strings, so a blank input could blank a stored incident title
or fail as an invalid date-time.
- query_timeseries reported a failed query as success: Datadog returns 200 with a non-ok
`status` and the reason in `error`.
- create_monitor swallowed malformed options JSON and created a monitor with no thresholds.
- send_logs rebuilt each entry from a fixed field list, discarding the custom attributes
Datadog accepts as additionalProperties, and padded absent optional fields with empty strings.
Removed:
- mute_monitor. /api/v1/monitor/{monitor_id}/mute is absent from the v1 spec entirely, there is
no unmute counterpart to reverse it, and downtimes are the supported mechanism.
Contract accuracy:
- Security signal search advertised relative times ("now-1h"); the spec types filter.from/to as
format: date-time. Descriptions, placeholders, and wand prompts now produce ISO-8601.
- list_incidents advertised an `include` value ("integrations") that is not in the spec enum,
and neither incident tool trimmed the comma-separated list, so "users, attachments" 400d.
- Invalid "ok" group state dropped from both monitor descriptions.
- time_slice removed from SLO create input, which cannot build one without an SLI specification.
- DatadogSite gains ap2, uk1, and us2.ddog-gov.com.
Pagination and errors:
- list_downtimes silently truncated at Datadog's default 30 with no way to page; adds
page[limit]/page[offset] and surfaces totalCount.
- query_logs returned a cursor it had no way to accept back.
- Error extraction consolidated onto datadogErrorMessage, which now also reads the
dictionary-shaped errors of the SLO delete conflict. Ten tools were reading `.detail` off
plain strings or the raw entry off objects, degrading every failure to a bare status line.
- Debug logging removed from list_monitors.
Adds 29 regression tests, each verified to fail when its fix is reverted.
* fix(datadog): add SEV-0, document page-size caps, drop unsourced output defaults
- The severity dropdown omitted SEV-0, which IncidentSeverity allows and both incident
tool descriptions already advertised.
- Page-size descriptions now state Datadog's documented default of 10 and cap of 100
instead of an arbitrary example, so an agent does not request an out-of-range page.
- trigger_synthetics_tests emitted an explicit null for a string-typed optional output,
and update_synthetics_status reported 'live' on the error path regardless of what the
caller actually requested.
* fix(datadog): keep mute_monitor and add the missing unmute counterpart
Reverses the removal in the previous commit. Absence from the datadog-api-client-go
generator spec showed the endpoint is unpublished there, not that it is retired:
Datadog's official Python client still implements it on master as
`Monitor.mute(id, scope=, end=)` and `Monitor.unmute(id, scope=, all_scopes=)`
(datadogpy datadog/api/monitors.py), which `_trigger_class_action` resolves to
`POST /api/v1/monitor/{id}/mute` and `/unmute` with exactly those body fields.
mute_monitor has also been in the block since #2175 in December, so dropping it would
have broken existing workflows for an endpoint that two independent sources agree is live.
The genuine defect was that muting was a one-way trapdoor: Sim could mute a monitor but
had no way to reverse it. Adds datadog_unmute_monitor, sharing the monitor ID and scope
inputs with mute, so the operation is recoverable from the same block.
Also: mute no longer discards the response body (it now reports the monitor id, name, and
state), routes errors through datadogErrorMessage, encodes the monitor ID in the path, and
stops dropping an explicit `end` of 0.
* fix(datadog): make downtime targeting explicit and reach downtime pagination from the block
Addresses the review findings on the previous round.
- create_downtime accepted both a monitor ID and monitor tags but `monitor_identifier` is a
oneOf, so it silently kept the ID and dropped the tags, muting a different set of monitors
than the caller asked for. It now rejects the ambiguous combination.
- create_downtime ran Number.parseInt on the monitor ID with no validation, so a non-numeric
value became NaN and serialized as null inside monitor_identifier. It now uses the same
parseMonitorIds guard the SLO path already had, naming the offending value.
- list_downtimes gained limit/offset in the tool but the block exposed neither, so no
block-driven call could page past Datadog's default. Adds the two sub-blocks and wires them
through the params mapper.
- The block did not declare the totalCount the tool now returns, so nothing downstream could
bind to it.
* fix(datadog): tolerate non-string list inputs and keep the shipped mute subblock ids
Both defects were introduced by this branch.
- splitCommaList called .split on its argument, so routing create_downtime's monitorId
through it turned a legitimate numeric input into a TypeError before the request was
built. A <Block.output> reference to get_monitor or list_monitors resolves to a number,
and an LLM tool call can pass a number or an array, so the helper now normalizes all
three shapes. The previous Number.parseInt path had accepted a number by coercion.
- Adding the unmute operation renamed the mute subblock ids scope/end to muteScope/muteEnd.
Workflow state is persisted by subblock id, so every existing Mute Monitor block would
have kept the old keys and silently lost its scope and end time. Restored the shipped ids;
both are still unique block-wide and no operation reads another operation's value.
* fix(datadog): compare downtime targets after parsing, not before
A whitespace-only Monitor ID is truthy as a raw string but parses to no monitor, so
the oneOf conflict guard rejected a valid tag-targeted downtime whenever the untouched
Monitor ID field carried blank text. Both sides are now compared after parsing.
|
||
|
Waleed
|
9e67655b23 |
feat(servicenow): semantic incident, change, catalog, approval, CMDB, and knowledge tools (#6747)
* feat(servicenow): add semantic incident, change, catalog, approval, CMDB, knowledge, and directory tools
The ServiceNow block only exposed generic Table API CRUD, so every real task
started with "which table is that on?". This adds 27 semantic tools that wrap
the same Table API plumbing under the names customers actually use.
- Incidents: create, get by number or sys_id, search, update, resolve, close,
and append a work note or customer-visible comment.
- Change: create, get, list, update, move state, and list change tasks through
the documented Change Management API.
- Service catalog: browse items, order one via the Service Catalog API
order_now endpoint, and list or get requested items.
- Approvals: list pending approvals for an approver, approve, and reject.
- CMDB: search CIs on any class, read a CI with its inbound and outbound
relations through the CMDB Instance API, and list cmdb_rel_ci rows.
- Knowledge: search and read articles through the Knowledge Management API.
- Directory: find a user by email or user name and list group members, which
is what fills assigned_to and assignment_group.
Reference fields are the usual source of confusion, so every semantic read
defaults to sysparm_display_value=all — a reference comes back as both its
sys_id and its label — and every semantic write exposes
sysparm_input_display_value so a display name can be written instead of a
sys_id. Coded state values are exposed as labelled dropdowns built from one
constants module rather than raw integers.
The shared instance-URL, Basic Auth, sysparm, envelope, and error handling now
live in tools/servicenow/utils.ts, and the existing eight generic tools were
moved onto it rather than keeping their own copies.
* fix(servicenow): stop per-operation subblock defaults colliding on a shared id
Subblock initial values are seeded into block state keyed by subblock id, so
two subblocks sharing an id leave one stored value and the last definition
wins. Three ids were duplicated with differing defaults:
- `displayValue` was defined twice, unset for the generic Table API tools and
`all` for the semantic ones. The semantic definition won, so a new block set
to Read Records or Aggregate Records sent `sysparm_display_value=all` — a
wire change to two already-shipped tools.
- `state` was defined four times. The Approval State definition won, so every
new block carried `state=requested`, which Create Incident wrote to the
incident and Move Change State used instead of its own `-5` default.
Give the colliding controls their own ids and map them back to the tool params
per operation, so the generic tools keep their original request shape and each
semantic operation keeps its own default.
Also correct descriptions that overstated what the API does: the LIKE operator
is not documented as case-sensitive, List Requested Items has no requester
filter, and the Change Management API task shape differs from the Table API.
Adds tool tests covering the refactor invariants for the eight pre-existing
Table API tools and the display-value separation.
* feat(servicenow): read a change request's real next states from the instance
The change tools describe state transitions using the base-system codes, which
only hold on an instance that has not customized its change model. ServiceNow
publishes an endpoint that answers the question directly for the record in
hand, so use it rather than keep assuming.
GET /api/sn_chg_rest/change/{sys_id}/nextstates returns the states reachable
from the change request, the instance's own state-value-to-label map, and, for
model-driven changes, each transition with the conditions it has and has not
met. The tool flattens the per-target-state grouping ServiceNow returns (each
transition already carries from_state and to_state, so nothing is lost) and
derives the states whose conditions currently pass.
Also record the sourcing for the coded values in constants.ts: the change
states and close codes are published as a table, but the incident state codes
are not — only 6 (Resolved) appears in the docs — so mark the rest as defaults
rather than guarantees. Note that sysparm_input_display_value also reinterprets
date and time values in the caller's timezone instead of GMT, which matters for
the change start and end dates.
* docs(servicenow): stop asserting undocumented coded values in placeholders
The additional-fields examples used hold_reason with a coded value of "1".
ServiceNow documents the On hold reason choices by label only — Awaiting
Caller, Awaiting Change, Awaiting Problem, Awaiting Vendor — and publishes
neither the column name nor the codes, so the example was asserting something
unsourced. Use a field whose value is caller-supplied instead, and record the
On Hold requirement on the incident state control using the labels the docs
actually give, including that Awaiting Caller makes Additional Comments
mandatory.
* fix(servicenow): drop phantom parent fields from the catalog order output
order_catalog_item read parent_id and parent_table off the order_now response.
Those fields belong to submit_producer, a different Service Catalog endpoint;
the documented order_now result is sys_id, number, request_number, request_id,
and table. Both outputs were therefore always null.
* fix(servicenow): correct what knowledge search returns as an article id
Search results carry a table-prefixed identifier — "kb_knowledge:9e528db1..."
— not a bare sys_id, while GET /knowledge/articles/{id} accepts only a bare
sys_id or a KB number. The output described it as a sys_id and the tool
description told callers it was what they needed to fetch the article, so
chaining the two tools on that field would fail. Point callers at the KB
number instead. Relevancy score is documented as a number, not a string.
* docs(servicenow): cite the page that actually documents approval statuses
The approval state constants pointed at the classic-approvals landing page,
which does not list the statuses. Approval status is documented separately and
names four — Requested, Approved, Rejected, and Not Requested.
* fix(servicenow): stop constant interpolation leaking into tool descriptions
The docs generator and the client-facing integration catalog read tool
descriptions from source rather than from the evaluated module, so a
template literal like `state ${INCIDENT_STATE.RESOLVED}` shipped to users
verbatim: `apps/sim/lib/integrations/integrations.json` and the published
ServiceNow integration page both rendered `${INCIDENT_STATE.RESOLVED}`
instead of `6`. Inline the base-system coded values in the description
text; the constants stay in use everywhere behavior depends on them.
Also drops an escaped `\'` in the `inputDisplayValue` description for the
same reason, and adds a standing guard test asserting no subBlock id
carries two different seeded defaults — the invariant behind the
per-operation defaulting bug, now checked structurally rather than only
through the four per-operation cases.
* refactor(servicenow): type the shared response boundary instead of any
`parseServiceNowResponse` returned `any`, so every tool reading `data.result`
did unchecked property access — a shape change on the instance side would have
produced a wrong-typed output silently rather than a type error.
Introduces `ServiceNowEnvelope` (`result?: unknown`) as the parser's return
type and narrows the record index signatures from `any` to `unknown`. Adds
`toRecordObject`, `readString`, and `readNestedNumber` so the tools that read
individual fields narrow deliberately at the point of use.
This surfaced five genuinely unchecked reads: Order Catalog Item, Get Knowledge
Article, and Search Knowledge were declaring `string | null` / `number | null`
outputs while emitting whatever the instance sent, and Get Change Next States
assigned an unvalidated object to `Record<string, string>`. Each now coerces or
drops a non-matching value rather than passing it through.
* fix(servicenow): publish the shared tool params and stop offering inert controls
The docs generator reads tool source rather than importing it, so the shared
`params.ts` consts the semantic tools spread were dropped from every published
Input table — 27 of 35 ServiceNow tools listed no instance URL, username, or
password at all. Follow a spread into the module it is imported from so those
rows are published; ten other integrations gain the rows they were missing for
the same reason.
Two controls were dead on arrival: Additional Fields was offered on Move Change
State and Add Incident Comment, and neither tool read it. Wire it through the
change transition, which needs it, and drop it from the comment tool, whose body
is exactly one journal field.
Every coded-value control was a select-only dropdown, so a customized instance's
state or close code was unreachable — sharpest on Move Change State, whose
target state is required and whose real codes come from Get Change Next States.
Make them comboboxes.
Also correct two doc claims ServiceNow does not publish (the incident state
citation pointed at a page that does not exist and compares the legacy
incident_state field; closing an incident is not documented as requiring
itil_admin), replace Record<string, any> with checked narrowing that surfaced
two unsound widenings, and document that List Change Tasks returns a fixed
{value, display_value} shape under `tasks` rather than `records`.
* fix(servicenow): stop one subblock id from carrying two value spaces
Subblock values are stored per block keyed by id, so an id reused across
operations keeps its value when the operation changes. Incident and change
shared `state`, and `closeCode`, `closeNotes`, `comments`, and the knowledge
search phrase were each reused for a different value space — so an incident
state could be written onto a change request, an incident close code sent as a
change close code, or an encoded query searched as knowledge text.
Give each value space its own subblock and republish it to the tool param from
the operation that owns it, the way targetState and approvalState already work.
The generic Table API ids stay exactly as they are, since renaming one would
orphan the stored value of every workflow already using those shipped tools.
The previous guard only compared seeded defaults, which is why this class stayed
hidden; the new one asserts against the merged params a tool actually receives.
* fix(servicenow): point the canvas sentences at the renamed subblocks
The split of the colliding subblock ids left the operation sentences anchored on
ids that no longer exist, so those clauses would silently drop from the card.
* fix(servicenow): validate collection members and split the fields projection
toRecordArray cast every member of a successful response, so a null or scalar in
a collection was handed to the next block as a record while the tool reported
success and its declared output said that could not happen. Members that are not
plain objects are now dropped, and knowledge articles and change transitions get
the same narrowing. The two response types that described an unverified inner
shape now say what is actually checked.
The 'fields' subblock also carried two value spaces: a JSON body on Create and
Update Record, a comma-separated projection everywhere else. Operations added
since read a separate returnFields control, so a body can no longer arrive as a
projection or the reverse. The shipped ids are untouched, since renaming one
orphans the stored value of every workflow already using those tools.
|
||
|
Waleed
|
4bc89c9256 |
feat(crowdstrike): add alerts, host response, IOC, Spotlight, RTR, and case tools (#6746)
* feat(crowdstrike): add alerts, host response, IOC, Spotlight, RTR, and case tools CrowdStrike Falcon shipped only three read-only Identity Protection sensor tools. This adds 20 tools across the response and investigation surface SecOps teams actually automate against. Alerts (current Alerts API): query, get details, update status/assignment/ tags/comment/visibility. Hosts: contain, lift containment, hide, unhide. Host groups: query, get details, add/remove hosts. IOC Management: query, get, create, update, delete. Spotlight: query vulnerabilities, get vulnerability details. Real Time Response: init session, execute a read-only command, poll command status, delete session. Case Management: query cases, get case details. Every endpoint, request field, and response field is taken from CrowdStrike's published surface (developer.crowdstrike.com API reference, FalconPy endpoint definitions, and the swagger-generated gofalcon models). Required API scope is documented in each tool description. Deliberately not implemented: - Detects API: decommissioned 2025-09-30, superseded by Alerts. - CrowdScore Incidents API and behaviors: decommissioned 2026-03-09 and removed from the developer center entirely. Case Management is CrowdStrike's replacement, so its two documented read operations are implemented instead. - Case create/update/merge: the swagger types case `status` and `severity_info.level` as bare strings with no enum, so a correct write cannot be built without guessing. CrowdStrike answers 200 with a populated `errors` array for partial failures. Responses now surface those per-item errors, and an empty result set carrying errors is reported as a failure rather than silently succeeding. The route's shared Falcon client, response normalizers, and operation dispatch move into colocated modules so the handler stays readable at 23 operations. * fix(crowdstrike): correct the RTR read-tier commands and stop dropping the IOC delete filter Validation pass over all 23 tools against CrowdStrike's swagger-generated SDKs (gofalcon falcon/models + falcon/client, FalconPy _endpoint/*.py) turned up four real defects. The Execute RTR Command dropdown offered `csrutil` and a bare `reg`. Neither is a read-tier base command: CrowdStrike's own swagger description for RTR_ExecuteCommand enumerates cat, cd, clear, env, eventlog, filehash, getsid, help, history, ipconfig, ls, mount, netstat, ps, and "reg query". `csrutil` appears nowhere in CrowdStrike's published surface, and `reg` alone is not a base command — the registry variants are "reg query" (read) and "reg set"/"reg delete" (Active Responder). Both entries are corrected everywhere they were repeated: dropdown, tool description, and param description. Delete Indicators showed a Filter input, declared the param, accepted it in the contract, and implemented CrowdStrike's documented filter-takes-precedence rule in the route — but the block never mapped the field into the tool call, so the filter was silently discarded and a filter-only delete failed validation. The `filter` case is now mapped alongside the ID list. A 200 carrying only envelope errors was reported as HTTP 200 with success:false, which reads as a success to anything inspecting status. Failures now adopt the per-item error code the envelope supplies, falling back to 502. Alert updates gain a first-class Remove Tags By Prefix field. The spelling was previously unresolvable, so it was left to the raw action-parameter escape hatch; CrowdStrike's swagger settles it as `remove_tags_by_prefix` in both the PatchEntitiesAlertsV2 and PatchEntitiesAlertsV3 descriptions. Case Management and Spotlight scopes now name the OAuth scope string (case-templates:read, spotlight-vulnerabilities:read) alongside the label the Falcon API client UI shows, so either rendering is findable. * fix(crowdstrike): stop blank sensor filters reaching Falcon and expose the RTR outputs The falcon.ts/normalize.ts/operations.ts split routed query_sensors through the shared buildUrl helper, which skips only undefined. An empty filter or sort string therefore emitted `?filter=` / `?sort=` where the pre-split route omitted the param, sending Falcon an empty FQL expression. Reject blank values in the contract instead, matching the newer operations. Also surface the ten RTR fields the tools already return but the block never declared, and broaden the block metadata past the original sensor-only surface. * fix(crowdstrike): fail query operations on error-only envelopes and guard IOC pagination Falcon can answer 200 with an errors array and no resources. The detail operations already treated that as a failure, but the five query branches returned an empty successful result, so a failed alert query read as a valid no-match to the calling workflow. Blank FQL rejection now covers the alert, host-group, indicator, vulnerability, and case contracts too, not just sensors, and Query Indicators rejects offset combined with after instead of forwarding a pagination pair CrowdStrike refuses. * fix(crowdstrike): stop blank inputs reaching Falcon and restore the dropped output docs The executor merges `tools.config.params` over the raw block inputs, so a key the mapper omitted kept its raw subBlock value — and an untouched subBlock is stored as `null`, which the route contract rejects. Query Alerts with an empty Filter, Update Alerts without every optional field, and Delete Indicators without an audit comment all 400'd before reaching CrowdStrike. Seed every optional key as `undefined` so omission is authoritative, which also stops a value left over from another operation riding along. Shared output consts in `outputs.ts` were silently dropped from the generated docs: the generator scans tool source and resolves consts only from `types.ts`, so `errors`, `affected`, and `pagination` rows vanished from 17 tool pages and every nested property row with them. Inline the literals. Against CrowdStrike's own generated SDKs and developer portal: - add csrutil, ifconfig, users, and the eventlog subcommand forms to the read-tier RTR base commands, matching PSFalcon's ValidateSet - add detection_suppress/detection_unsuppress and cap host actions at the documented 100 ids - cap the IOC search limit at the documented 500, not 2000 - correct the Cases scope to "Cases: Read"; case-templates guards a different collection - type the IOC payload so a blank string cannot clear a stored field on PATCH - send `MsaRangeSpec` bounds capitalized, as the spec serializes them - fail the sensor and RTR-session-close paths on a 200 whose envelope carries only errors, and surface partial sensor errors - give Delete Indicators its own filter so a stale alert query cannot widen it - drop the pre-selected network-isolating host action * fix(crowdstrike): correct the RTR command tier, IOC update contract, and US-3 region Independent re-validation against gofalcon's swagger-generated models and CrowdStrike's developer center turned up several wire-level errors. - Real Time Response advertised "eventlog backup"/"export"/"list", "reg query", ifconfig, and users as base commands. base_command names a command family and subcommands belong in command_string; the eventlog write variants are Active Responder commands that would fail on scope under this Read-scoped tool, and ifconfig/users appear in neither authoritative list. The block now offers the 16 documented read-tier families and the contract enforces them. - Indicator updates accepted an entry with no id, which cannot name a record, and accepted type/value, which the update model does not expose. Creates accepted an entry with no type, value, or applied_globally -- the one property CrowdStrike marks required, and the one that decides fleet-wide scope. - CrowdStrike documents that PATCH overwrites any omitted field with a blank value. The contract can only catch blanks, so the update tool now tells the caller to read the indicator first and resend its full field set. - Added the US-3 commercial region, which was missing from every cloud list. - Aggregate queries silently dropped percents and filters_spec. - Deleted the response-envelope body unwrap: no endpoint this integration calls returns that shape, and getFalconErrorMessage never honored it anyway. - Softened the Detects and Incidents claims to what the sources actually state. A tool description longer than the docs generator's 600-character id-search window silently publishes as an empty string; three descriptions had crossed it. Shortened them and added a test that fails before the catalog goes blank. * docs(crowdstrike): name the endpoint and Identity Protection scope on the sensor tools The three sensor tools were the only ones in the family that named neither their endpoint nor their OAuth2 scope, and none of them said these are the domain controllers Falcon Identity Protection monitors rather than Falcon endpoint sensors -- a distinction an agent choosing between them and the Hosts tools has no other way to make. Identity Protection Entities: Read is also a separate product entitlement from Hosts and Alerts. * refactor(crowdstrike): say which ID caps are CrowdStrike's and which are Sim's Every bulk-ID limit claimed CrowdStrike as its source, but only the sensor (5000), host action (100), indicator batch (200), and Spotlight (400) caps are published. The alert, host group, indicator, and case caps are Sim's own bound on request size, and the validation message now says so instead of attributing a limit CrowdStrike does not document. |
||
|
Waleed
|
cbbcca970c |
fix(okta): stop partial updates erasing stored profile data (#6751)
* fix(okta): stop partial updates erasing stored profile data Post-merge audit of the Okta integration (follows #6741), verified against the OpenAPI spec bundled in okta-sdk-golang/.generator. Two updates could silently destroy data: - `update_group` targets `PUT /api/v1/groups/{groupId}`, which Okta documents as `replaceGroup` — it swaps the profile wholesale. Sending only the two fields the tool exposes erased the stored description on every rename, and dropped every org-defined custom attribute along with it. The tool now reads the group and overlays the supplied fields before replacing, matching the read-modify- write `salesforce_update_custom_field` already uses for the same hazard. - `update_user` gated its profile fields on `!== undefined`, so an empty string reached Okta and blanked the stored value. The block strips blanks before they get there, but the tool is `user-or-llm` and a model routinely emits `""` for a field it has nothing to say about, so the guard belongs on the tool. Also corrected: - `forgetDevices` defaults to true at Okta, so the unseeded switch rendered off while remembered factors were in fact being cleared. - Group rules take a plain keyword on `search`, not the SCIM-style expression the shared Search field's wand generates, so they get their own field. - `get_logs` dropped `limit=0`, which the spec documents as valid. - `get_user` emitted an activation timestamp under `activated`, which the block declares as the lifecycle boolean; the timestamp is now `activatedAt`. - Descriptions that overstated what an endpoint does: `list_users` omits DEPROVISIONED users, `delete_user` deactivates before it deletes, `delete_group_rule` answers 202, and `excludedGroupIds` is always empty because Okta does not support group exclusions. * fix(okta): forward the abort signal through the group read-modify-write * test(okta): rename the shared body-builder helper * fix(okta): key the send-email and search mappings off the operation * docs(okta): use TSDoc for the new block annotations |
||
|
Waleed
|
852906ec91 |
feat(splunk): add Splunk Enterprise and Cloud integration (#6743)
Adds a Splunk block with 12 REST operations: run search (oneshot), create/get/cancel search job, get search results, list/get/dispatch saved searches, list/get fired alerts, list indexes, and list apps. Bearer-token or basic auth, with optional /servicesNS namespace scoping.
Every tool was validated against the Splunk REST reference. Results use search/v2/jobs/{sid}/results because the v1 endpoint is deprecated and disabled from Splunk Enterprise 9.0.1. A half-specified namespace fills the missing node with the documented - wildcard rather than nobody/search, which would have hidden user-private objects. Dispatching endpoints fail loudly instead of reporting success with a null sid, and Create Search Job rejects exec_mode=oneshot since that mode returns results rather than a search ID. The results and control endpoints tolerate an empty body. saved/searches sends the f field filter the reference prescribes for it.
|
||
|
Waleed
|
d45dad7e8b |
feat(okta): add System Log, MFA, sessions, apps, roles, and group rules (#6741)
Expands the Okta block from 18 to 44 operations, covering the System Log, MFA factors, sessions, applications, administrator roles, and group rules. Adds shared helpers for the SSWS auth header, Okta error parsing, and the Link-header `after` cursor, and routes every tool through them so there is one auth and error path. All eight list operations now return `nextCursor` and `hasMore`. Makes the block's param transform authoritative over the serialized inputs: the executor merges it on top of them, so a key the transform omits keeps the raw subBlock string. Assigning `undefined` is what actually drops it, which is what keeps a non-numeric `limit` from reaching Okta verbatim and stops a blank field in a partial `update_user` from overwriting the stored value with an empty string. |
||
|
|
337a53f12c |
feat(cli): Sim CLI with AWS-style profiles and a platform key exchange (#6147)
* improvement(api): pull in the v2 external endpoint surface Cherry-picks improvement/v2-endpoints ( |
||
|
Theodore Li
|
6006870f02 |
feat(credentials): add v2 credential lifecycle APIs (#6664)
* feat(credentials): add v2 OAuth connection APIs * fix(credentials): preserve active OAuth connection links * fix(credentials): bind OAuth links to connection intent * feat(credentials): complete v2 credential lifecycle * fix(credentials): make disconnect idempotent * fix(credentials): stabilize oauth draft retries * fix(credentials): bind oauth callbacks to drafts * fix(credentials): fail closed on oauth completion * fix(credentials): bind shopify completion to oauth state * fix(credentials): align custom oauth reconnects * fix(credentials): centralize application authorization * fix(credentials): keep OAuth draft intent immutable * fix(credentials): allow renamed reconnect targets * fix(credentials): close OAuth draft edge cases * fix(credentials): fail closed without breaking auth * fix(credentials): preserve migrated route behavior * feat(credentials): add provider search * fix(credentials): prevent stale secrets and drafts |
||
|
Theodore Li
|
ee1fc379a4 |
fix(tables): allow unbounded v1 row queries (#6713)
* fix(tables): allow unbounded v1 row queries * fix(tables): drain under-budget queries fully * fix(tables): bound expanded query metadata * fix(tables): always return query totals |
||
|
Waleed
|
3848f97b4c |
fix(grafana): validate against the API docs, add data source querying and contact-point CRUD (#6712)
* fix(azure-data-explorer): correct the tags ingestion-property example
The example rendered as tags="[''daily'']" — doubled single quotes from an
escaping slip, which is not valid Kusto. The reference writes a tags list
as tags='["TagA","TagB"]': single outer quotes with the JSON array's own
double quotes inside.
The clause builder already handled that form; only the example text was
wrong. A template literal avoids the escaping entirely, since the metadata
generator reads the source verbatim and would otherwise carry the
backslashes into the description the model sees.
Adds a test asserting the reference's exact multi-property clause
round-trips, including the comma inside the quoted array.
* fix(grafana): correct response contracts, required alert fields, and outbound request hardening
Validated against Grafana's HTTP API reference and, where the docs
contradict themselves, against the Go wire structs.
Response shapes the tools got wrong:
- update_annotation declared an `id` that was always 0; a patch returns only
a message, so the request's annotation id is echoed and labelled as such
- delete_folder discarded the numeric id Grafana returns and presented an
input-echoed uid as if it came from the API
- delete_dashboard fabricated `id: 0` / `title: ''` via `||` on absent fields
- the contact-point `provenance` description was inverted: "api" means
API-managed, empty means it stayed UI-editable
Requests that could not succeed:
- create_alert_rule left noDataState and execErrState unset and invisible to
the model, but Grafana's validator rejects an empty value outright, so every
model-driven create failed. Both are now sent with Grafana's own defaults,
and skipped for recording rules, which take a different validator
- get_data_source routed a numeric input at /api/datasources/:id, which exists
only behind an off-by-default feature toggle. UID only now
- list_annotations did not trim the dashboard UID, so a padded value matched
nothing
Outbound hardening on the three proxy routes:
- the service-account token was re-sent to redirect targets; the shared fetch
only drops it when asked, so stripAuthOnRedirect is now set
- no timeout was passed, leaving two sequential hops at the 5-minute default
- upstream error bodies were interpolated whole into the tool result, putting
up to 10MB of HTML into logs and traces; now truncated
- UID path segments are URL-encoded so they cannot re-target the request
- update_folder sent both `version` and `overwrite: true`, which Grafana treats
as alternatives, making the freshly fetched version decorative and silently
clobbering a concurrent rename
- replaced the `any` casts with narrowed types
Block surface:
- 25 outputs the tools emit were undeclared and so unreferenceable downstream;
get_data_source had 13 of its 18 unreachable
- `version` was typed string though the dashboard, folder, and data-source
producers all emit a number
- the dashboard title field was shown only for create, so a dashboard could
never be renamed through Update Dashboard
- six list outputs were typed json rather than array
* fix(grafana): let the health check report ill-health, and disambiguate block outputs
The data source health check could only ever report health. Grafana answers an
unhealthy source with HTTP 400 carrying the same {status, message} payload as a
healthy one, and the tool framework converts any non-2xx into an opaque tool
error — so the diagnostic the caller actually wants was unreachable. The check
now goes through an internal route that reads the verdict off either status and
reports it as a successful check, while a failure carrying no verdict (missing
data source, bad token, plugin with no health endpoint) stays a real error. The
plugin's `details` payload is surfaced too.
Also on that route, matching the other three: an outbound timeout, redirect
auth stripping, a truncated upstream error, and a URL-encoded UID.
Block output descriptions: ten keys are emitted by several tools with different
meanings and were described for only one producer — `database` meant both a
data source name and a health status, `annotations` both an annotation list and
an alert rule's summary map. Eleven `json` outputs were opaque although the
tools already document their inner fields. All rewritten to name every producer.
Smaller alignment fixes:
- the same EmbeddedContactPoint.settings field was typed `object` in list and
`json` in create
- list_contact_points mapped non-nullable uid/name/type through `?? null`;
Grafana returns an empty string, which is what create already assumed
- create_alert_rule sent `orgID`, which Grafana overwrites from the
authenticated context, and `Number()` on a non-numeric value put NaN -> null
in the body
- the three update routes declared `output` as required though the auth
short-circuit omits it, and did not declare the `details` they emit on a
validation error
* feat(grafana): complete contact-point CRUD, and add folder move and rule-group read
Four operations the integration was missing, taking it to 29.
update_contact_point / delete_contact_point close a real gap: contact points
could be listed and created but never corrected or removed. Two things worth
recording, because the published docs get both wrong:
- both verbs answer 202 with only a message, not the object. The rendered docs
claim delete returns 204; the current spec and handler both say 202. So the
UID is echoed from the request, the way delete_folder and update_annotation
already do
- update is a full replace with no PATCH counterpart, so name, type, and
settings are all required and the description says so. Omitting
disableResolveMessage resets it
X-Disable-Provenance is exposed on update only. Its polarity is the opposite of
the alert-rule case: omitting it always succeeds, while sending it against an
API-provisioned contact point is rejected — with 403, not the 409 rules use. It
is not exposed on delete at all, because that handler never reads stored
provenance and the endpoint takes no such parameter.
move_folder reuses get_folder's mapping verbatim — same DTO. It always sends
the parentUid key, since Grafana reads an empty value as "move to the root",
which a conditionally-omitted field could not express.
get_alert_rule_group surfaces the group evaluation interval, the one alerting
knob the per-rule operations cannot reach. It reuses the shared mapAlertRule for
the nested rules, and the interval is documented as an integer of seconds.
* feat(grafana): add data source querying, and ground the skill and templates in real tools
query_data_source closes the largest gap in the integration: 29 tools could
read dashboards, folders, and alert configuration, but none could read a metric
value. It posts to /api/ds/query and returns both the raw response and the
frames flattened into rows.
The flattening is derived from the documented layout rather than any data
source's field names: a frame carries schema.fields[] alongside data.values[],
where values[i] is the whole column for fields[i], so zipping them by position
works for Prometheus, SQL, or anything else with a backend.
A failed query is a 400 by Grafana's own status table, so it stays a tool
error — unlike the health check, where the failure status carries the answer.
That also lets four templates and the review-firing-alerts skill stop promising
things the integration could not do. Three templates assumed a metric-query
tool, which now exists. The fourth, and the skill, assumed live alert instance
state, which the provisioning API never returns — they now derive firing rules
from alert-state annotations, which are documented to carry newState and
prevState, and say so explicitly rather than implying a live snapshot.
Deliberately not added: a tool over /api/prometheus/grafana/api/v1/rules for
live instance state. That endpoint appears on no Grafana HTTP API doc page, its
response is only readable from Go internals and test assertions, and the
instance-level state casing differs from the rule level with no documented
contract. Not something to build an output schema on.
* fix(grafana): declare the two block outputs the earlier fixes introduced
Renaming update_annotation's phantom `id` to `annotationId` and adding
`details` to the health check both created outputs the block never declared, so
neither was referenceable downstream. Caught by re-running the output-coverage
check over both integrations; the block now covers all 64 keys the 30 tools emit.
* fix(grafana): make Update Contact Point actually usable from the block
The new replace operation could never succeed. contactPointType and
contactPointSettings were widened to cover it, but contactPointNameNew was
left create-only — and the update maps `name` from that field, so the required
parameter was never supplied.
disableResolveMessage had the same gap, and it matters more than it looks:
the update is a full replace, so a block-driven update was silently clearing
resolve suppression on every contact point it touched. Both fields are now
shown, and required where the API requires them.
Also states a reason on each intentionally-unconstrained response field —
Zod issue objects, alert query stages, notification settings, recording-rule
config, and data-source health detail are all genuinely opaque, but that was
left implicit.
|
||
|
Waleed
|
d5701d5b2b |
fix(icons): align table block icon and 12-unit icon stroke with the emcn family (#6708)
The table/table_v2 blocks and the table trigger used a local lucide-shaped TableIcon (stroke 2.0, full-bleed 24 viewBox, 3x3 grid) while every other table surface used emcn's Table. Consolidate onto the emcn icon and drop the local copy. Nested tool-call rows in Chat applied no color class, so a non-brand block icon inherited body text instead of --text-icon. redo/undo/zoom-in/zoom-out draw 0.85 stroke on a 12-unit viewBox, rendering 0.992px at a 14px box against the family's 0.904px. 0.775 restores parity. |
||
|
mzxchandra
|
5a88ce22d1 |
feat(ashby): incremental job sync, custom field writes, and application lifecycle ops (#6703)
* feat(tools): add incremental job sync and draft postings to Ashby reads
list_jobs accepts Ashby's syncToken and returns it as nextSyncCursor, so a
scheduled sync costs O(changed reqs) instead of rescanning every req. Ashby only
returns the token once the last page is drained, which the param description
states.
The output is named as a cursor deliberately. It is an opaque resumption marker,
not a credential, so it belongs with nextCursor - and a field literally named
syncToken matches the /^.*token$/i deny-list in redaction and renders as
[REDACTED], which makes an incremental sync unusable since the operator cannot
read the value the next run needs. The wire name stays syncToken.
list_job_postings gains includeUnpublishedJobPostings, plus the posting status
field - without status a caller cannot tell a returned draft from a published
posting, which makes the flag useless.
Also widens the custom field valueLabel type, which MultiValueSelect returns as
an array, for the write operations that follow.
* fix(tools): render Ashby object-shaped API errors readably
Ashby documents two error shapes and uses both. The `errors` array form carries
`{ message, parameter }` objects, which stringified to '[object Object]' and hid
the real cause - including the 403 a key gets when it lacks a module permission.
Also adds the shared pieces the new write operations need: one definition of the
custom field value shape for the read and write paths to agree on, and a
normalizer for Ashby's case-sensitive objectType enum so a model emitting
'candidate' fails here with the allowed values rather than at the API.
* feat(tools): add Ashby custom field writes, delete, source, and anonymize
customField.setValue/setValues are the only way to annotate a job or req, since
Ashby has no job notes and no job tags. Writing null clears a value, so the
annotation is reversible.
Because null clears, every one of these operations requires explicit intent
before it can destroy data. The block's required markers do not cover the agent
path - a model calls the tool directly, so tools.config.params never runs and
validateRequiredParametersAfterMerge skips a param marked not-required:
- set_custom_field_value rejects an absent or blank fieldValue; an explicit null
still clears
- change_application_source requires unsetSource to clear, and rejects a source
id and an unset request together, since preferring either one silently
discards the other. Ashby has no 'leave unchanged' mode, so setting and
clearing are the only two intents and exactly one must be expressed
- set_custom_field_values rejects an empty array locally rather than relying on
Ashby to reject it
application.delete needs candidatesDelete, a module permission separate from
candidatesWrite. candidate.anonymize strips PII but leaves the record; Ashby
exposes no candidate deletion endpoint.
* test(tools): cover the new Ashby request and response shapes
Includes a gated live harness (ASHBY_LIVE=1) alongside the mocked tests.
vitest.setup.ts stubs global fetch for every file in the app, so the live file
restores the real implementation and asserts the restore worked - without that
guard the whole suite silently passes against a mock.
* feat(blocks): expose the new Ashby operations in the block
fieldValue is polymorphic (boolean, number, string, array, object, null), so it
decodes structured input and otherwise passes text through. The decoding is
deliberately narrow rather than a blanket JSON.parse, which corrupts real text:
1e999 becomes Infinity and serializes back out as null, which CLEARS the field;
a long numeric id loses precision past 2^53; and prose starting with { turns into
an object. Only the literal keywords, {, [ or " prefixes, and exactly
round-tripping numbers decode.
fieldValue carries no wand generationType: json-object forces braces and
json-array forces brackets, and both would wrap a value that must stay bare.
fieldValues, whose contract really is an array, uses json-array.
Setting and clearing an application source are mutually exclusive, so the Source
ID field is conditioned off while the clear switch is on and the params mapping
sends only the intent the switch selects. A value typed before the switch was
flipped cannot reach the tool and surface as an error with no visible cause.
* docs(ashby): document the new operations, permissions, and limitations
Ashby scopes permissions per module and they fail at runtime, not build time, so
the block docs now carry the permission table. Also records the hard API limits
worth designing around: no note or tag on a job, no pagination on
jobPosting.list, and no delete for jobs, candidates, or custom field definitions.
* fix(blocks): stop a stale create-path source id leaking into a source change
The executor merges { ...inputs, ...transformedParams }, so any key the params
mapping leaves unset inherits whatever inputs held. The shared create-path
sourceId subblock reaches inputs even on change_application_source: it is mode
'advanced', and the serializer includes an advanced subblock whenever its value
is non-empty without ever evaluating its condition (serializer/index.ts).
So a source id typed while on Create Application survived into a source change.
With both fields blank it silently attributed a source nobody asked for, and
with the clear switch on it collided with the unset request and failed with no
visible cause, because the field producing it is hidden in that state.
sourceId is now always assigned for this operation rather than conditionally,
so it can never inherit. The regression test asserts the merged result rather
than the mapping alone, since the gap between them is where the bug lived.
|
||
|
Waleed
|
5bb59f08ee |
feat(connectors): add 9 knowledge base connectors (#6699)
* feat(connectors): add 9 knowledge base connectors
Box, Zoho Desk, PagerDuty, Trello, Microsoft Excel, Google Slides, Google
Vault, Mintlify, and SFTP. Selected by intersecting the published connector
catalogs of Glean, Onyx, Dust, Vectara, Writer, Guru, Elastic, Microsoft 365
Copilot, Notion AI, Unstructured, and Airbyte against services that already
ship a Sim block, so OAuth providers, credentials, and icons are reused. Box
was the largest gap, appearing in 7-8 of ~10 catalogs.
Every connector was validated against live provider documentation twice, the
second pass treating the first pass's conclusions as unproven. Notable
correctness work that came out of that:
Listing truncation. The sync engine hard-deletes documents past a cap that is
not flagged with `listingCapped`, and five connectors had a path there — an
empty Mintlify discovery, Zoho Desk's exact-multiple default caps, Trello's
archived lists and 1000-card ceiling, a Google Vault cursor bailout, and a
PagerDuty stalled page. The engine also gained a backstop: an empty or
collapsed listing blocks deletion reconciliation until the same observation
repeats on a consecutive sync, reconstructed from existing sync-log counters
so no migration is needed.
API alignment. `desk.zoho.ca` does not resolve (Canada is
`desk.zohocloud.ca`, and Singapore and UAE were missing); `modifiedTime` is
absent from Zoho's ticket list projection, so every ticket re-embedded on
every sync; Trello's `dateLastActivity` is documented to miss some edits;
PagerDuty's 10,000-record ceiling bounds `offset + limit`, not offset; Excel
indexed dates as raw serial numbers while Google Sheets renders them; Google
Vault truncated at roughly 249 matters.
Security. SFTP followed symlinks in `getDocument` and composed unchecked
server-supplied filenames into paths; it now also supports optional host-key
fingerprint verification, which runs during key exchange before any password
is sent. Trello interpolated user-supplied board ids into URL paths. Google
Vault is narrowed to `ediscovery.readonly`. `getDataverseBaseUrl` accepted
any host while attaching a bearer token, and is pinned to Microsoft's
Dataverse domains — pre-existing shipped code, fixed here.
Also adds `ConnectorAuthConfig.optional` so a public source can be configured
without inventing an API key, and teaches the scope check that a granted
read-write scope satisfies a required `.readonly` sibling.
Microsoft Dataverse was built and then removed: its OAuth cannot complete
consent. Dataverse requires a per-environment resource URI, the provider
declares a static `https://dynamics.microsoft.com/user_impersonation` that is
not an Entra Application ID URI, and the environment URL is only collected
after the credential exists. That predates this change and also affects the
12 shipped Dataverse tools.
* fix(dataverse): strip the bearer token when a request redirects
The host allowlist added alongside the connector work only constrains the
initial destination. `secureFetchWithPinnedIP` follows redirects and keeps the
`Authorization` header unless a tool opts out, so a redirect away from an
allowed Dataverse origin would forward the caller's OAuth token to whatever
host answers. Dataverse redirects in normal operation — file downloads hand
back a signed storage URL, and environment hosts move between regional
origins — so this is reachable without a compromised environment URL.
Sets `stripAuthOnRedirect` on all 18 Dataverse tools, matching the existing
GitHub job-logs and Windchill precedent.
* fix(connectors): address review findings on listing and hashing
- microsoft-excel: `fetchWorksheets` read only the first Graph page and never
followed `@odata.nextLink`. A workbook with more sheets than fit in one page
dropped the remainder from the listing without setting `listingCapped`, so
the sync engine reconciled those documents away as deleted. The walk now
pages, bounded by MAX_WORKSHEETS, and only follows a nextLink that stays on
the Graph origin, since the link is server-supplied and carries the token.
- google-slides: the listing `contentHash` covered only the file id and
modified time, so toggling the speaker-notes option left every stored hash
matching and no presentation was ever re-hydrated with the new scope. The
setting is now part of the hash, in the single shared stub builder so the
list and hydrate paths stay identical.
- mintlify: `pathPrefix` filtered with a bare `startsWith`, so a prefix of
`/guides` also matched a sibling like `/guides-archive`. It now shares the
`/`-boundary rule `withinBasePath` already used, extracted as `isUnderPath`.
* fix(connectors): list newest first in zoho desk, accept a trailing slash prefix
- zoho-desk: `sortBy: 'createdTime'` is ascending — Zoho denotes descending
with a `-` prefix — so the default 500-record caps kept the oldest tickets
and articles and recent ones were never listed. Because the cap sets
listingCapped, that stale tail could not reconcile away either. Now sorts
`-createdTime`. Still ordering on createdTime rather than modifiedTime, so
rows do not reshuffle mid-walk.
- mintlify: `resolvePathPrefix` kept a trailing slash while `isUnderPath`
accepts an exact match or `prefix + '/'`, so `/guides/` matched neither
`/guides` nor `/guides/intro` and the source synced nothing. A regression
from the previous round, which replaced a bare `startsWith`. The prefix is
now normalized before comparison.
* fix(dataverse): strip the bearer token on the upload route's own redirect
`upload_file` posts to an internal route rather than calling Dataverse
directly, so the tool-level `stripAuthOnRedirect` added in
|
||
|
Waleed
|
cf78946529 |
fix(v2): close seven correctness and honesty gaps found sweeping the API (#6702)
* fix(v2): close seven correctness and honesty gaps found sweeping the API A ten-slice sweep of the live v2 surface turned up no regression from the recent cancellation work, but did surface a set of pre-existing defects where an endpoint either lost data, hid a failure, or reported something that was not true. Each is fixed at the layer that owns the behavior. Terminal execution logs. The two force-fail boundaries wrote `status: 'failed'` without `ended_at` or `total_duration_ms`, so a force-failed run dropped out of every duration-filtered log query — the same defect class already closed for cancellation, still open on its sibling. The cancellation payload factory is generalized to take the status; the cancellation call sites are untouched and still emit a byte-identical row. Custom tools. One malformed row failed the whole page, and because the list is keyset-paginated that row made every page containing it permanently unreachable. The projection now validates against the same contract schema the route builder applies, repairing only what can be repaired without inventing information — a stringified schema, and a missing `type` whose contract admits exactly one value — and omitting with a warning what cannot. Both rows observed in production are recovered rather than discarded. Table filters. `eq`/`ne`/`in`/`nin` compiled a wrongly-typed operand into a containment test that silently matched nothing, so a filter written against the value the write path had stored returned an empty page instead of its rows. The operand is now read through the same column-type registry the write used, and rejected only where that registry refuses it. Range operators already behaved this way; `null` and the cleared-cell sentinel still pass through untouched. Error messages. A custom `error` on a string schema also replaced the wrong-type wording, so supplying a number for a name reported that the name was missing. Messages now distinguish an omitted field from a mistyped one, `topK` names its own bounds, the knowledge search refine reports against a field rather than the whole body, and a workspace id is bounded before it reaches a lookup. Archived file metadata. A soft-deleted file was listed but unreadable, leaving no way to check share state before restoring it. The read takes the same `scope` selector the list already exposes; the default is unchanged, and the parameter relaxes only the `deleted_at` predicate, never the authorization. Cancellation reporting. Cancelling an already-terminal run reported a durable write that never happened. The service now distinguishes the no-op and names the state it observed, and both surfaces present one vocabulary instead of the internal route deriving its own. No claim predicate or write changed. Protocol. A 401 carries a challenge naming the header the API actually reads, and a body that failed to parse is reported as an unsupported media type only when the caller positively declared a non-JSON one — after the read has already failed, so nothing that succeeds today can begin to fail. * fix(v2): correct three regressions this branch introduced, and harden its tests Adversarial review of the previous commit found that three of its "behavior preserving" claims were wrong. Each is corrected here at the layer that owns it. Table filters no longer coerce a `date` operand, and no longer throw. `date` is the one column type whose registry `coerce` is not idempotent — it drops sub-second precision — and the leaf that compiles a filter also builds the unique-constraint and upsert-conflict probes, so re-reading an already-coerced operand could stop it matching the row it was written from and admit a duplicate inside the write transaction with no error. Throwing was the second mistake: the v2 predicate grammar type-checks structure but not operand values, so a rejected operand no longer failed at submission but inside the delete, update, dispatch and cancel runners, where a filter that cannot compile means the cells it started can no longer be cancelled. Coercion is now total — it rewrites what the registry accepts and passes everything else through unchanged, exactly as before. Reviving a force-failed run no longer inherits its terminal duration. Writing `ended_at` and `total_duration_ms` on the force-fail boundary was correct in isolation, but a partial resume flips that row back to `pending` and those columns survived. The preserved value is meant to be the pause checkpoint — the run's active time — and it had become wall clock measured at the failed resume, which the checkpoint rule then faithfully carried into the next terminal write. The revival clears them only for a row that was terminal, so an ordinary paused row keeps the checkpoint it is supposed to keep. Cancelling reports the terminal state it actually observed. Reclassification now requires that nothing else went wrong, so a genuine paused-reconciliation failure survives instead of being rewritten as an already-terminal no-op, and the claim's own row count — not a snapshot read before it — decides whether this cancel terminalized the run or lost a race to something else. The status the snapshot needed rides along on the ownership query that already reads the row, rather than the second read that query's own contract warns against. A custom tool that cannot be projected now answers the same way everywhere: the list omits it, and reading or patching it by id reports it as absent rather than as a server fault. Analytics stops reporting a cancellation for a request that cancelled nothing. The tests around all of this were audited by mutating each fix and checking the suite noticed. Where it did not, the assertion is stronger now: the absent content-type branch is genuinely exercised rather than relying on a header the client library supplies, the duration encoder is pinned to the column it must measure from, execution ownership is pinned to both ids it must match, and the archived-file concealment test proves it conceals the archived read specifically. Two tests that asserted a paused branch they could not observe are gone; the rendered-SQL test that can decide it already covers them. * fix(execution): report a workflow-group cancellation as the write it performed Cancelling a workflow-group run whose log had already been cancelled, but whose cell sidecar still needed reconciliation, durably cancelled that sidecar and then reported `already_cancelled` with `durablyRecorded: false` — because the terminal-status shortcut answered from the entry snapshot alone and never asked what this request had written. The analytics event, which now gates on that field, stopped firing for a cancellation that really happened. The outcome a cancel reports is the same question whichever path answers it, so there is now one vocabulary for it rather than one the direct claim tracked and one the group transition did not. Every group result maps to that outcome through a total map, so a new group result cannot compile without deciding what it wrote, and the reclassification leads with whether this request wrote at all. A group transition that reports itself already cancelled is deliberately mapped as unknown rather than as a no-op: it leaves the sidecar alone but still terminalizes a log that was active, and the result does not say which happened. That costs nothing today, because the only snapshot that would reclassify proves the log was already terminal. * fix(execution): have a workflow-group cancellation report the writes it made Three review findings landed on the same reporting logic, each a different face of one cause: the caller could not see what the group transaction had written, so it inferred. It inferred from an entry snapshot, then from the returned kind, and the remaining blind spot was the kind that covers two different transactions — a repair that terminalizes an active log, and a genuine no-op — which left a cancel that wrote nothing still claiming a durable write when it lost a race. The transaction now reports both writes it can make, each read from that statement's own returning row and recorded immediately before the throw that already depended on it, so the report cannot drift from the write. The caller derives its outcome from those rather than from the kind, and the kind is back to naming the situation instead of standing in for the work. The group path can now always answer whether it wrote. The only remaining unknown is the direct claim when its update throws or is never attempted, which genuinely has no row count to report. |
||
|
Vikhyath MondretiandClaude Opus 5
|
237f973a11 |
fix(condition): stop a secret value from breaking or forging a condition (#6705)
Condition expressions pasted every environment variable value into the
expression as source. Block references in the same expression go through a
proper escape and get quoted; env vars went through neither. That left three
defects:
- A bare string placeholder was a SyntaxError. `{{NAME}} === 'alice'` resolved
to `alice === 'alice'`, so the form the Function block docs recommend could
not be used here at all.
- Ordinary data broke the block. An apostrophe (`O'Brien`) or a newline in a
legitimate value produced unparseable source and failed the run.
- The quoted form was injectable. A value of `x' || true || '` turned
`'{{NAME}}' === 'bob'` into `'x' || true || '' === 'bob'`, forging a true
branch out of a comparison that should be false.
Inline only structurally inert literals — numbers, booleans, and null, with
optional space/tab padding. Every other value keeps its `{{NAME}}` placeholder
and is bound as a string by the execution-boundary compiler, the same one
Function blocks and Custom Tools already use.
Legacy outcomes are preserved. `{{COUNT}} === 3` and `{{ENABLED}} === true`
still compare as literals, and an embedded `"Bearer {{API_KEY}}"` still
compares equal — now via compiled concatenation rather than a pasted value.
Padding is admitted rather than trimmed so the inlined text stays
byte-identical to the stored value, which is what keeps a padded number
correct both bare and quoted.
A resolved secret also no longer travels to the execution boundary inside the
condition source.
The one deliberate behavior change: a value whose text is itself a quoted JS
literal (a secret stored as `'foo'`, a plausible workaround for the bare-string
SyntaxError) now compares as the 5-character string rather than as source.
That form is the injectable one, so it cannot be kept.
Docs: state the placeholder type contract, which was described mechanically but
never in terms of what a reader gets. `{{KEY}}` in Function and Custom Tool code
always evaluates to a string, so a bare `if ({{FLAG}})` is always true and a list
has to be stored as JSON. This is what a customer hit after the resolver lift in
#6247 moved Function blocks off source inlining.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Waleed
|
a7115e87ee |
feat(integrations): add Azure Data Explorer (#6701)
* feat(integrations): add Azure Data Explorer Add a 14-operation Azure Data Explorer (Kusto) integration covering KQL queries, schema and metadata discovery, table management, inline and query-sourced ingestion, ingestion-failure triage, and arbitrary management commands. Authentication uses a Microsoft Entra service principal through an internal proxy route, since the Kusto token audience is per-cluster and cannot be expressed as a static-scope OAuth provider. * fix(azure-data-explorer): only read partial-failure status from the QueryStatus table Scanning every returned table for Severity and StatusDescription columns misread an ordinary query as a failed request whenever the user's own result selected columns of those names — a common shape for a log table. Failure detection now consults only the table the response's table of contents names as QueryStatus, and primary-result selection reuses the same index instead of re-reading it. * fix(azure-data-explorer): keep the Show Operations and Show Table Details cards from painting empty check:canvas-sentences flagged the Show Operations sentence: it anchored `core` on operationId, which is an advanced-mode optional field, so an untouched card resolved to nothing and painted empty. Show Table Details had the same shape in milder form — table is optional there, since omitting it describes every table, leaving a dangling preposition. Both now lead with literal copy and treat their field as an optional refinement. Also simplifies the primary-table condition to a single `!= null` check. * fix(azure-data-explorer): authenticate sovereign clusters against their own Entra authority The cluster allowlist accepted Azure China and US Government hosts, but every token request went to login.microsoftonline.com. Those clouds are isolated instances with their own Entra endpoints, so a sovereign cluster passed URI validation and then could never obtain a token. Each Kusto service domain is now declared alongside the authority that issues tokens for it, so the two cannot drift apart, and the authority is part of the token cache key. * improvement(azure-data-explorer): warn that ingest-from-query matches columns by position Kusto aligns an ingested query result to the target table on column type and order, never on column name, so a query projecting the right columns in the wrong order lands data in the wrong columns without erroring. Surfaces that in the tool description and param the model reads, in the wand prompt that generates the query, in the rollup skill's steps, and in the docs. Also verifies the target schema first rather than after. * chore(azure-data-explorer): drop the unsourced kustomfa host from the cluster allowlist Every other entry traces to a Microsoft reference — the Kusto connection-string doc, the national-cloud endpoint tables, and the Fabric KQL-database REST reference. kustomfa.windows.net does not, and the connection-string doc states the trust boundary as hostnames ending in kusto.windows.net. An allowlist should only hold hosts we can justify, so this drops it and records the sourcing standard for anything added later. * fix(azure-data-explorer): handle commas inside quoted properties and empty extent IDs Two defects in the shared command helpers: buildWithClause split the property list on every comma before validating, so a value that legally contains one — a docstring sentence, or a tags array with more than one entry — was torn in half and rejected. Splitting is now quote-aware, and an unterminated quote is rejected outright rather than swallowing the rest of the clause. transformColumnListResponse dropped empty strings, but `.ingest inline` reports "no data shards were generated" as a single record carrying an empty extent ID. A no-op load therefore looked like a missing column instead of an empty result. Only non-strings are skipped now. |
||
|
Waleed
|
1d342722ad |
feat(rabbitmq): add RabbitMQ integration (#6700)
* feat(rabbitmq): add RabbitMQ integration * fix(rabbitmq): strip auth on redirect, require https, and bound the retrieval response * fix(rabbitmq): reserve message metadata in the retrieval response budget |
||
|
Waleed
|
7f64d5e600 | perf(tables): stop a table write refetching every loaded page in the tab that made it (#6698) | ||
|
Vikhyath Mondreti
|
fa394e5e07 |
fix(execution): resolve secrets against the acting principal, not the workflow owner (#6690)
* fix(execution): resolve secrets against the acting principal, not the workflow owner * fix(execution): resolve anonymous public-API runs as the workspace billing account * fix(execution): propagate run identity across dispatch paths and scope public runs to workspace secrets |
||
|
Waleed
|
9fc65863cd |
fix(v2): hold create to the rules update enforces, and bind the last two cursors (#6684)
Three defects, one shape: a rule applied to one path and not its sibling. Two were found by probing the live surface after the previous fixes deployed, and the third by reading for the pattern. Creating a workflow group through the public surface validated almost nothing the update path validates. An enrichment group could name an enrichment the registry does not define, or an output the enrichment does not have, or carry no output id at all — each a 201 storing a column no run can ever write, discovered only when the caller later tried to edit the group and got the 400 create should have given. The workflow half was the same: a fabricated block-and-path coordinate was stored on create and refused on update. Create now runs the same two registry helpers and the same workflow-output check the update path uses. The discriminator there is the backing workflow id, not the declared type. The workflow sidebar creates enrichment-template groups labelled `enrichment` while backed by a real workflow and carrying no enrichment id, so keying on the label would have refused the first-party create path outright. A group's producer type could also be relabelled after the fact into a state creation refuses. Nothing rejected it and nothing could repair it, since the update body carries no enrichment id to supply. Relabelling an enrichment group as workflow-backed is the harmful direction: it keeps the enrichment id while moving the group onto the workflow branch with an empty workflow id, so every cell run fails. An update may now only restate the type the group already has. The workflow-version and workspace-member lists were the last two paged reads minting cursors with no route identity, so a token from one parent resumed another at a position that silently skips rows — the defect the previous change closed everywhere else. Both now wrap their domain token with the same scope binding, and the pagination guardrail gained a declaration of every nested list's parent path param, because the old one recorded only query filters and so could not tell an unfiltered list from a forgotten parent. |
||
|
Waleed
|
ab8a64fcae |
fix(v2): close the defects live probing found (#6681)
* fix(v2): close the defects live probing found Staging finally deployed the merged release, so the surface could be exercised for real. Every fix already shipped held up. These are the defects only live traffic surfaced, plus the ones a static sweep had found and left. A cursor named a position in a sequence without naming the sequence. `cursorScopeKey` hashed only the caller's filters, so any two lists filtering on nothing but `workspaceId` produced one fingerprint and accepted each other's tokens: a tables cursor replayed against the knowledge list answered 200 and silently skipped a row. Table rows never reached that check at all, so a cursor from one table paged another. Identity now comes from the route's own contract — method plus resolved path — because a hand-written name is the step an author forgets, and forgetting it is invisible. An unresolved path placeholder throws rather than fingerprinting the template, so a misconfigured route fails on every request instead of an unlucky one. Every token minted before this is refused with an accurate message; they are single-walk and unpersisted. Knowledge search and the document list answered different questions. Search grouped same-tag filters by slot and joined them with OR while the list conjoined every filter, so `gte 9` and `lte 2` on one tag returned nothing from the list and a full billed page from search. Search now conjoins. The OR grouping replaced an explicit `|OR|` mechanism that was deleted outright, was never documented in any contract, and cost the ability to express a range on a single tag; the union it gave is still reachable as separate searches. The search body also accepted an unbounded query that was billed and then silently truncated to the embedding model's window, and ignored the tag-filter cap the list enforces. A body over ten mebibytes was reported as malformed JSON. Next's proxy truncates there, well under this app's fifty-megabyte ceiling, so the parse failed on a body the caller sent whole and the size branch was unreachable. The ceiling is now clamped to what the proxy will pass. Also: a group's output columns accepted a `workflowGroupId` and discarded it; an enrichment group could never gain an output, because a new output coordinate demanded workflow metadata a group with no workflow cannot have; `newOutputColumns` alone reported success and created nothing; a saved view stored layout references to columns that do not exist while refusing the same name in a filter; an MCP server stored `retries: 0` as three and overrode an explicit auth type; a disabled server answered tool discovery with an unclassified fault; rotating a header server's headers left it reading connected; and a run whose workflow was deleted reported the root folder path while also reporting the workflow deleted. Where the honest fix was out of reach, the contract was corrected instead of half-fixing the code: the polled run resource rebuilds `error.code` by matching the persisted message, so it can never report the two codes that need block attribution, and now says so. `OUTPUT_TOO_LARGE` is removed — no path ever emitted it. `triggers` was left alone deliberately. It reads as a closed enum but production holds 43 distinct values, because a webhook run stores its provider id; pinning the enum would refuse legitimate history a log search exists to find. The description now says the vocabulary is open. * fix(v2): clamp explicit body caps to the proxy ceiling too The previous commit clamped the default JSON body cap but left explicit per-route overrides alone, so a route declaring a larger `maxBodyBytes` still fell into the truncation it was meant to report: the four inline workspace-file routes at 70 MB and the deployed-chat route at 220 MB. Next attaches `proxyClientMaxBodySize` to every request and clones the body unconditionally for any non-GET method on a matched path, pushing EOF at ten mebibytes with only a warning, so the handler reads a truncated prefix. Those routes therefore already fail above that size — as a malformed-JSON 400. Clamping the effective limit inside the two body readers makes the same request fail as payload-too-large, quoting the limit actually in force. One existing test asserted the unreachable case, allowing a sixty-mebibyte base64 body; it now asserts what the proxy will forward intact. The inline-file path still advertises fifty mebibytes and cannot exceed the proxy ceiling until that ceiling is raised, which changes buffering for every route and belongs in its own change. * fix(v2): close the two holes the first review round found Both are places where a fix in this branch shut one door and left a smaller one open in the same wall. Letting an enrichment group gain an output meant skipping workflow resolution — but that resolution was the only thing validating a new output, so a PATCH began storing coordinates the runner can never fill. It fills a cell from `result[out.outputId]` and skips an output with no `outputId` at all, while the writer diffs on that same id and the sidebar reads and writes by it; the contract leaves it optional. The regression test added with that fix was itself asserting such a dead coordinate. Create's registry checks are now two shared helpers both paths call, and on update an output is exempt only when an identical binding already existed, so renaming a group whose enrichment has since changed still works while anything added or repointed must name a real output. `mappingUpdates` on an enrichment group now says it is inexpressible rather than resolving an empty workflow id into a missing workflow. The layout-reference check was handed the tolerant column set, so a placeholder minted to keep a dangling filter writable also whitelisted a brand-new layout reference — storing an entry the next read discards, which is the inconsistency the check was added to remove. Layout now resolves against the live columns, which is exactly what pruning keeps, while filters and sorts keep the exemption they need. |
||
|
|
1424809614 |
feat(netsuite): add Oracle NetSuite integration (#6476)
* revise netsuite integration * fix(netsuite): align selector route with snowflake * test(netsuite): remove selector route coverage * test(netsuite): align coverage with snowflake * fix(netsuite): complete integration validation * refactor(netsuite): align integration with codebase patterns * test(netsuite): correct async job citation * fix(netsuite): address final audit findings * fix(netsuite): surface upsert/transform Location, relax task link check Oracle documents the Location response header for create and update, and both tools already require it. Upsert and transform also produce a record but Oracle documents no response headers for either, so they dropped the header entirely and the new record's ID was unreachable. Add a `resource-optional` location mode that captures Location when NetSuite sends it and never fails when it is absent, and wire it to upsert and transform along with their tool and block outputs. Async task discovery rejected the whole response if any task link carried a rel other than `self`, collapsing the picker into a 502. Oracle documents a `self` link per task but never guarantees it is the only one, so skip other relationships and fail only when no self link exists. Also use the shared `truncate` helper in the error sanitizer per the repo convention instead of an inline slice. * fix(netsuite): validate SuiteQL pages against their documented shape The shared collection-page validator required links, items, count, hasMore, offset, and totalResults on every 200, and a missing field turns a successful call into a reported failure. Oracle documents all six for record collections and SuiteAnalytics dataset pages, but its SuiteQL reference lists only links, count, offset, totalResults, and items. A documented SuiteQL response that omits hasMore would therefore have been rejected. Split out a suiteql-page validator that requires the five documented SuiteQL fields and type-checks hasMore only when the account returns it. Record collections and dataset pages keep requiring all six. * chore(netsuite): regenerate tool metadata after rebase on staging The rebase conflicted only in the generated tool-id, tool-metadata, and tool-output artifacts, which NetSuite and the newly landed LogRocket integration both extend. Regenerated from the merged registries: the result is staging's catalog plus the 27 NetSuite tools, with LogRocket's entries intact and no other tool changed. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Bill Leoutsakos <billleoutsakos@Mac.localdomain> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Waleed
|
58b5ee9337 |
feat(logrocket): add LogRocket integration (#6678)
* feat(logrocket): add LogRocket integration * improvement(logrocket): fail loudly on a non-numeric identify timestamp and cover releases in the catalog copy * fix(logrocket): require a highlights identity and treat pagination cursors as opaque strings * fix(logrocket): trim request fields so whitespace-only input fails validation * fix(logrocket): declare the release version in the block inputs map |
||
|
Waleed
|
e2b7335644 |
fix(v2): stop telling callers something the server did not do (#6676)
A works-as-advertised sweep of the v2 surface found one defect class in five
places: input is validated for shape, then its meaning is re-derived
independently by each consumer — so a filter compiles differently than it
validated, or a write commits and the response then reports failure.
Knowledge tag filters were validated once and re-parsed three times. The
document list read `Number()`, search read `parseFloat()`; the list matched
booleans case-insensitively, search compared against the literal `'true'`; the
list escaped LIKE metacharacters, search did not; and the date pattern was
tested against the untrimmed string the validator had already trimmed. Two of
those paths dropped the predicate entirely and answered 200 with the whole
knowledge base — on a billed endpoint. Values are now coerced once, where the
resolved field type is known, and both builders consume the result. A builder
that cannot compile an already-validated filter now raises instead of silently
widening the result set.
`PUT /api/v2/secrets/{name}` with `scope: personal` committed the secret and
then answered 500, because a user-global write was reported through a
workspace-scoped mirror lookup, and an org admin's inherited access has no
`permissions` row for the fan-out to find. The personal path no longer decides
success from a per-workspace mirror. The `workspaceId` descriptions said a
personal secret lives in one workspace; it does not, and they now say so.
A custom tool could be stored with a schema the read path cannot serialize —
`POST /workflows/import` and Copilot both wrote through name-only checks — so
one row made the whole workspace list 500, and a title-only PATCH committed,
audited, then reported failure. Every write now passes the same guard the
response schema is derived from.
`POST /api/v2/tables` accepted `workflowGroupId` on an initial column. Nothing
can populate it legitimately, and it made every later column-add and group-add
fail with no way to clear it. The key is refused at the boundary, and
`createTable` now runs the invariant every later mutation already runs, closing
the internal and v1 ingresses too. Those invariants moved to a leaf module:
reaching them through `workflow-columns` pulled the executable tool registry
into the tables page graph, taking it from 1,767 modules to 6,999.
An out-of-range upload part number answered 500 rather than the 400 its
published contract promises, because the throw happened above the route's
try/catch and was not an `HttpError`.
|
||
|
Waleed
|
8d319a430a |
fix(v2): close the correctness gaps the release audit found (#6671)
An end-to-end audit of the v0.8.1 release surfaced one regression the release itself introduced and a set of filter/cursor gaps that let a caller's spelling change what a query answered. An `enrichment` workflow group stored no `workflowId`. The public contract invites a caller to omit it, the write persisted caller input through an `as WorkflowGroup` cast that hid the omission from the type checker, and the response schema still required it — so the write committed and then the outbound parse threw. Because the list presenter maps every group through that schema, one such row made GET, PATCH and DELETE on that table's groups fail from then on, with no public way to remove it. The cast is gone rather than papered over, so the same class of omission cannot recur silently. Knowledge tag filters accepted any operator string and then ignored an unrecognized one in opposite directions: the document list dropped the predicate and answered with the whole knowledge base, while search fell through to equality and answered a different question. Both now reject at the boundary. `.strict()` is applied on the v2 chain only — v1 has always stripped unrecognized keys, and the `between`-requires-`valueTo` rule already closes the mis-cased `valueTo` trap on both versions. Cursor scopes bound set-valued filters to the caller's ordering: `all`/`any` clause order and `in`/`nin` operand order in the table predicate, and the raw `resourceType` text on audit logs, whose query splits it into an `inArray`. Audit logs is canonicalized on both sides, because canonicalizing the scope alone would have given two genuinely different result sets one fingerprint. Also: the exposed-header list reached only the fallback CORS policy, so all five matched rules — including the wildcard-origin execute route, the only one that emits `X-Run-Id` — could not hand a browser the run id or a 429's `Retry-After`; a bulk row update reported an uncoercible value only when its filter happened to match; the cost and duration windows accepted an inverted pair and answered it with an empty page; and the sortless runs list advised callers to fix a `sortBy` it rejects. |
||
|
Waleed
|
6de8ba2504 |
fix(v2): close the correctness gaps an end-to-end audit found (#6655)
* fix(v2): stop a third-party tool description from 500ing MCP discovery
`v2McpToolInputSchema` declared `description: z.string().optional()` inside a
`.catchall(z.unknown())` object, and a declared key beats the catchall. The MCP
SDK's own `ToolSchema.inputSchema` does not declare `description` at all, so any
value — including the JSON `null` a Python server emits for an absent one —
passes its validation and reaches Sim unchecked. The builder's outbound `.parse()`
then threw, and the discovery error policy correctly declines to classify a
Sim-side schema defect, so the endpoint that completes MCP onboarding answered a
bare 500. The key is dropped and left to the catchall; `type`, `properties`, and
`required` stay pinned because the SDK enforces those at least as tightly.
Also in the v2 resources family:
- The single-resource query schemas for MCP servers, skills, custom tools, and
secrets are now `.strict()`, matching every list in the same family. A mistyped
flag was silently ignored behind a 200.
- `openapi/resources.ts` re-derived `RESOURCE_ERRORS` and
`RESOURCE_CONFLICT_ERRORS` inline in 21 of 22 operations. They now import the
shared constants; the generated spec is unchanged, which is the point.
- The internal MCP refresh route stamped `updatedAt` alongside `lastToolsRefresh`.
`updatedAt` means "configuration last changed" and is a public keyset sort, so
a refresh moved rows out from under an in-flight page. `updateServerStatus`
already held that invariant; the route now matches it.
- The discovery cooldown is a typed `McpServerCooldownError` rather than a
substring search for `cooldown`. `McpConnectionError` interpolates the server's
display name into its message, so a server named after the word was reported as
a transient cooldown when its connection had genuinely failed.
* fix(v2): close correctness gaps in the workflows deployment surface
Deploy and rollback bodies were plain objects, so a misspelled key was
stripped rather than rejected. On rollback that is silent misbehavior:
an omitted `version` legitimately means "reactivate the preceding
version", so `{"versoin": 5}` rolled back somewhere else and answered
200. Both v2 bodies, the run-read query, and the versions cursor are now
strict.
Deployment versions are an `integer` column, but the path param, the
versions cursor, and the v1 body each bounded it differently or not at
all — an out-of-range value overflowed the comparison into an
unclassifiable 500. One exported bound now covers all three.
Resume admission raised bare `Error`s for a stale contextId or an
already-resumed run, which the resume surfaces could not classify and
reported as 500. They now use the sibling `ResumeAdmissionError` already
in that file, carrying 404/409/400 and whether an automatic retry can
clear the refusal.
Docs corrections: rollback publishes the 409 its webhook-path conflict
already produces; deploy/undeploy/rollback reject a workspace key with
403, not the concealed 404 they documented; the workflows OpenAPI module
imports the shared error sets instead of re-deriving them; import and
the folder ops explain their folder-tree 413. The export route is marked
`headSafe: false` so a HEAD probe stops filing a WORKFLOW_EXPORTED audit
event for an export that never happened. `runId` is one bounded schema
across the run and log resources.
* fix(v2): conceal knowledge upload existence, tighten knowledge/files bounds
Security: the four knowledge document-upload routes rendered a bare upload
error policy with no resource concealment, while every sibling knowledge route
uses one. Because the use case resolves the knowledge-base context before
workspace authorization, the unconcealed 403 told any valid API-key holder that
a knowledge base exists in a workspace it cannot reach — the exact signal
GET /api/v2/knowledge/{id} withholds by answering 404 either way. All four now
use the composed concealing policy, which also renders the 415/402/413 the
route-local renderer already handled; that duplicate renderer is deleted.
Contracts:
- POST /knowledge/search is strict. It was the only non-strict v2 request body,
so a mis-cased rerankerEnabled or topK returned 200 with the key stripped,
changing what the caller was billed and silently disabling reranking.
- The document list takes limit, cursor, and search from the shared v2 schemas.
search was an unbounded, empty-accepting v1 string, so ?search= answered 200
with a full page here and 400 on GET /knowledge, and the term reached an
unindexed filename LIKE scan with no ceiling.
- The 16 non-strict single-field workspace query slices across both families are
strict, matching GET /knowledge/{id}/tags.
- GET /audit-logs takes workspaceIdSchema instead of a bare string (?workspaceId=
was forwarded as a filter and returned zero rows) and the shared run-window
bounds for startDate/endDate.
Documentation:
- listAuditLogs drops the 404 it has no code path to emit.
- upsertFileShare describes its workspace-key refusal as the 403 it renders;
the operation denies the key by principal kind, which the concealment policy
does not rewrite.
- The 12 body-reading knowledge and files operations publish the 413 their
pre-validation body read raises, and the file list publishes the folder-tree
413 its now-capped path index raises.
Correctness: queryWorkspaceFilePage loads its folder path index under
MAX_FOLDERS_PER_WORKSPACE like the workflow, table, and knowledge lists. An
uncapped index does not fail on truncation, so a real folder outside the read
rows resolved to undefined and answered "Folder not found".
* fix(v2): publish the reachable 413 on body-carrying resources ops
`parseRequest` buffers a JSON body through `parseJsonBody` under
`DEFAULT_MAX_JSON_BODY_BYTES` before any schema runs, and the v2 builders supply
`V2_PARSE_DEFAULTS.payloadTooLargeResponse`, so every operation whose contract
declares a body already answers 413 above the cap. The resources family
published it on none of them. A status a caller cannot see in the spec is a
status they will not handle.
Adds `RESOURCE_BODY_ERRORS` and `RESOURCE_CONFLICT_BODY_ERRORS` to the shared
sets and applies them to the seven affected operations: createMcpServer,
updateMcpServer, createSkill, updateSkill, createCustomTool, updateCustomTool,
and setSecret. All seven are `defineV2JsonRoute` handlers on non-GET methods
with no `parseOptions` override, so the 413 is genuinely reachable on each. The
new sets are opt-in rather than folded into the base sets precisely because
reachability is not automatic — an operation with no body, or one whose payload
reaches it through an uncapped path, would be publishing a response that can
never arrive.
A sweep test pins the invariant across the resources, billing, and logs
documents. It is one-directional by construction: several bodyless operations
publish 413 for their own folder-tree and render ceilings, so the converse would
flag correct documentation.
Also completes the shared-constant consolidation started in
|
||
|
Waleed
|
29853fbbcf |
improvement(docs): make the API reference read as code and unify its type token (#6653)
* improvement(docs): make the API reference read as code, and unify the type token The API-page font override matched every span/div/p inside the page, which outranks the .font-mono class on specificity, so every parameter name, type, and identifier silently rendered in the body sans face. Exclude .font-mono so code tokens stay monospace. Consolidate the three divergent type-slot treatments — plain scalar, union, and schema reference each carried their own chip definition, differing in size, weight, face, and box height — onto one code token that reuses the docs inline-code recipe and the platform's 20px chip height. Demote the row metadata: 'required' and 'header' were filled pills, 'required' on the error token, making a constraint the loudest element on the page and a page of required parameters read as a page of alarms. Both are now uncontained text, leaving the type token as the only box on the row. Pin the two 'application/json' labels to one treatment; the Request Body and Response headers rendered the same string at different weights and faces. * improvement(docs): mono status-code tabs, and match fumadocs' lucide icons to emcn Status codes in the example panel are numeric literals and render as code everywhere else on the page, including the Response header's own trigger, but fumadocs rendered the strip in the body sans face. Language tabs sit in a separate container and stay sans — those are product names, not code. fumadocs draws a few lucide glyphs on API pages that its client-component overrides do not expose (the heading anchor and the code-block copy button). emcn strokes at 1.55 and lucide at 2, so those icons read heavier than every icon around them; match the weight. * fix(docs): align the auth type chip with every other property row, and wrap example code The auth row collapses its real `<token>` type and renders the chip through ::after, so the span is only a wrapper — but it still matched the type-token rule and kept that rule's border, height, and gap. The border drew a second empty box around the real chip and the gap opened in front of it, because the collapsed text remains an anonymous flex item; together they pushed the chip right by roughly 8px that no other row had. Example-panel code overflowed sideways instead of wrapping: fumadocs sizes the block with `w-max`, so it grew to its longest line inside a 400px scroller and the existing pre-wrap never applied. Cap the width, switch break-all to overflow-wrap anywhere so only unfittable tokens split, and reserve room for the copy button fumadocs floats over the first line. * revert(docs): let example code overflow instead of wrapping Wrapping restarts every continuation line at column zero, and in a JSON body indentation is what carries nesting depth — so a wrapped response misreports its own structure. A hanging indent keeps the depth but needs the shiki lines forced from flex rows to blocks, which breaks the line rhythm. Removes the pre-wrap rules rather than repointing them: fumadocs sizes the block with w-max, so the previous rule never took effect and overflow was already the behaviour on the page. * fix(docs): tighten array type tokens and keep the union separator legible An `array<T>` slot holds its angle brackets as bare text nodes, which become anonymous flex items, so the slot's gap prised `array<` and `>` away from the type they wrap. Drop the gap and let the union separator carry its own margin; this also makes the auth row's gap override redundant. The separator was dimmed twice, by a muted token and again by opacity, which on the dark chip fill left `string | null` reading as `string null`. * fix(docs): restore the hidden API key description, and drop dead API-reference CSS The rule hiding the trailing `In: header` line matched `p:has(> code)`, which is a shape, not a target — every scheme description in our specs cites a status code, so the whole explanation of personal vs workspace-scoped keys was display:none on every API reference page. Match the last child instead, and shorten the description to one line now that it renders. The dropdown trigger's hover rule had been left below a new id-qualified base rule that outranked it, so the trigger could no longer change colour on hover. Removes what does not run: the four `::-webkit-scrollbar` rules (specifying a non-auto scrollbar-width makes Chromium ignore them, and Firefox never had them) and an `order: 2` block whose selectors and declaration the type-token rule above it already carried. Names the two values the API reference repeats — the monospace stack, written out eleven times, and the 12.5px code size, written nine — as --font-mono-stack and --text-code. Also drops four !important declarations that already won on specificity, a --text-muted fallback that can never fire, and a lucide selector subsumed by the one beside it. * refactor(docs): define the API-reference chrome once, and cut the commentary The metadata face — size, leading, weight, mono stack — was written out in seven rules that a comment asked future readers to keep in sync by hand; it is now one rule those seven consume, each adding only its own colour, content, and order. The auth row's chip likewise re-derived all eleven declarations of the type token and now joins that rule, keeping only its label. Comments were running longer than the rules they documented — 88 added comment lines against 73 declarations. Trimmed to the load-bearing facts: cascade traps, browser behaviour, and the bugs a rule prevents. Dropped the block narrating why the wrap rules were reverted, which duplicated its own commit message. Also retires a scrollbar token left unreferenced by the webkit removal, moves the last two fumadocs colours in our own components onto platform tokens, and brings the callout icon to 1.55 so the docs really do have one icon weight. * fix(docs): keep the union separator in the type token's own face The `|` between union members is a classless span, so the page-wide `span:not(.font-mono)` rule assigned it the body sans face while the members beside it stayed mono — one chip rendering in two faces. Applies the inherit reset to every descendant of a type token rather than just its links, so anything fumadocs nests there later is covered too. |
||
|
Waleed
|
1fa40b8118 |
feat(v2): complete and align the v2 API surface (#6643)
* fix(v2): close four validation holes in the logs and billing surfaces
Each of these answered a caller-supplied value with a 500 or a silently
wrong result instead of a 400.
- `GET /api/v2/logs` accepted any string as `startDate`/`endDate`. The
route constructs a `Date` from it, so `?startDate=abc` reached the
driver's timestamp mapper as an `Invalid Date` and 500'd. Both bounds
now carry `.datetime()`, matching the sibling run list so one timestamp
works on both collections. This narrows the accepted set: a date without
a time and an offset-bearing timestamp are now rejected, and the field
descriptions say "UTC ISO 8601" rather than overpromising "ISO 8601".
- `v2BillingStatusQuerySchema` was the only non-strict query schema in its
family, so a mis-cased `workspaceID` was stripped and the caller got
account-scope billing in place of the workspace scope it asked for — a
wrong answer about money, served as a 200.
- An unresolvable `cursor` on `/api/v2/billing/logs` applied no cursor
condition and restarted the sequence at page 1 while still reporting
`hasMore`, so a pager holding a cursor across a deploy loops over the
first page and counts the same credits on every lap. It is now a 400.
The message does not reuse `INVALID_CURSOR_MESSAGE`, which names
`sortBy`/`sortOrder` params this collection does not accept.
- The logs `status` field disagrees with the run resources for the same
run: the run projection overlays `paused` from `paused_executions`,
so an ordinary human-in-the-loop pause reads `paused` there and
`pending` here. Reconciling would mean joining `paused_executions` in
this read and silently moving live runs between two buckets of a
shipped field, so the divergence is documented on the contract instead.
* feat(v2): expose the MCP tool plane and page the MCP server list
Registering an MCP server through v2 dead-ended: nothing on the public
surface ever ran tool discovery, so connectionStatus, toolCount, lastError,
and lastToolsRefresh stayed at their registration defaults and there was no
way to read a server's tools without opening the UI.
Adds GET /api/v2/mcp-servers/{id}/tools over a thin use case composed from
the existing mcp_servers.tools.discover operation, resolveServerContext, and
mcpService.discoverServerTools. It is personal-API-key-only — discovery
resolves the acting user's own OAuth credentials, which a workspace key
cannot supply — and the contract says so rather than letting callers meet an
unexplained 403. Discovery failures are classified instead of collapsing
into a 500: an unreachable or cooling-down server is a retryable 503, a
stale OAuth grant is a 401.
Also pages GET /api/v2/mcp-servers. It was the one unbounded list on the v2
surface, classified full-set on a bounded-by-construction rationale that
only holds for folder lists; nothing caps how many servers a workspace
registers.
* feat(v2/tables): strict row bodies, a filtered row count, and round-trippable required columns
Three tables gaps from the v2 capability evaluation.
Strictness. Every v2 tables request body is now `.strict()`. The row family
was the whole hole: `POST /query` sent v1's `filter` key answered 200 with a
fully unfiltered page, because Zod strips unknown keys unless told not to. The
same laxity covered the row create/update/delete/upsert/find bodies, the
run and cancel-runs bodies, the enrichment body, and — outside the row family
but the same class — the column delete, view create/update, and export bodies.
A contract sweep now walks every body-bearing tables contract and fails if one
of them stops rejecting an unrecognized key.
Filtered row count. `POST /api/v2/tables/{tableId}/query/count` answers the
question v1's `includeTotal`/`totalCount` answered and the `{data, nextCursor}`
envelope has nowhere to put: how many rows a predicate matches. It binds the
existing `queryTableRows` use case with `includeTotal: true, limit: 1` — no new
domain logic and the same `tables.rows.query` read policy. The use case types
`totalCount` as nullable because paged callers can decline it; this route always
asks for it, so a null is treated as a broken invariant rather than presented as
a fabricated zero.
Required columns. `required` is accepted on create-table, add-column, and
update-column, matching v1. v2 emitted the flag on every read while stripping it
from every write, so a column could not round-trip. Enforcement was already
complete: turning it on over rows with null, missing, or empty cells is rejected
by the domain.
* test(skills): pin the workspace-API-key split as structural, not accidental
A workspace API key can create a skill it can then never update or delete,
which no sibling resource does — so the asymmetry reads like an oversight
worth widening. It is not. Skill edits are authorized by the per-skill
editor row belonging to the acting user, which is why update/upsert/delete
declare a 'read' floor rather than 'write': workspace role is not the
authority. A workspace key carries no user subject, so allowing one replaces
a 403 with an unclassified PrincipalSubjectUserRequiredError that the v2
surface renders as a caller-reachable 500.
Records the reason on the registry and pins it, so the next reader finds the
argument instead of flipping the flag.
* feat(v2): read deployment state, and undo a file delete
Two v2 reads that existed only as a side effect of a mutation.
`GET /api/v2/workflows/{id}/deployment` publishes the state the deploy,
undeploy, and rollback responses carry, plus `needsRedeployment` — which
those responses structurally cannot carry, because they answer at the
moment the draft and the live version are equal. A caller that lost the
mutation response, or that polls from another process, had no way to ask.
Reuses `readWorkflowDeploymentStatus` behind `workflows.read`, the same
use case the internal status and deploy GETs already adapt.
`DELETE /api/v2/files/{fileId}` was a soft delete with no way to see what
it archived and no way to reverse it. `GET /api/v2/files?scope=archived`
pages the archived set and `deletedAt` on the file resource dates each
one; `POST /api/v2/files/{fileId}/restore` reverses the delete through
the existing `files.restore` operation. Restore is not a pure undo — it
returns the file to the root and renames it on a collision — so the use
case now reads the file back and both the response and the OpenAPI
description say what actually came back rather than what was deleted.
`scope=all` is rejected on the list for the reason the internal contract
already gives: it drops the `deleted_at` predicate and cannot use the
partial index. `scope=archived` combined with `folderPath` 404s when the
containing folder was archived too, which the contract documents.
* fix(v2): keep the unresolvable-cursor rejection a 400 on every surface
The cursor rejection lived in shared billing core but was an OrchestrationError
only, which the session-only GET /api/users/me/usage-logs cannot project: that
route is raw withRouteHandler and readTypedError matches instanceof HttpError,
so any signed-in caller typing ?cursor=x got a 500. UnknownUsageCursorError is
an HttpError carrying the OrchestrationError as its cause, so the v2 route still
renders BAD_REQUEST off the cause chain and the internal route answers 400.
Also closes the other half of the run-list parity: an inverted window on
GET /api/v2/logs is now a 400 instead of a silently empty page.
* fix(v2/tables): sweep union bodies per member and name the shapes on a rows 400
Review follow-ups on the strictness work.
The sweep was vacuous on the one union body it covers. Parsing
`{ notAContractField: true }` against `v2CreateTableRowsBodySchema` and looking
for `unrecognized_keys` anywhere in the issue tree is satisfied by either member
alone, so dropping `.strict()` from the single-row branch shipped green —
reproduced, 36/36 passing with the regression in place. The sweep now flattens a
union body into its members and asserts each one separately; removing `.strict()`
from either branch now fails a case that names it.
`POST /rows` answered an unknown key with `Invalid input`, the exact message the
v2 conventions name as failing the actionable-error rule, because a union
surfaces `invalid_union` first. The union now carries a message naming both
accepted shapes; the per-member failures still ride along in `details`.
Two TSDoc corrections. The `required` docstring claimed the domain rejects
turning the flag on over rows with empty cells — true of the update path, false
of add-column, which applies the flag as given (the same shape `unique` already
had here). And `.strict()` binds the top level only, so the view `config` object
and the shared sort-spec elements still strip unknown keys; both docstrings now
say so instead of implying full coverage.
* fix(v2): classify MCP discovery failures by type, not by substring
The tool-discovery error policy consumed categorizeError's status, whose
fallback is a substring match on the upstream message. Three consequences,
all caller-visible:
- A ZodError from the builder's own response `.parse` contains `invalid_type`,
so a Sim-side response-schema defect answered 400 "Invalid request
parameters" and suppressed the builder's 500 and its unhandled-error log.
- An upstream `Invalid params` or `not found` became the caller's 400/404 on a
request the contract had already validated.
- A stale OAuth grant to the third-party server answered 401, the status this
surface reserves for a missing or invalid Sim API key, so a client would
rotate a credential that was never the problem.
The policy now dispatches on the MCP error families and returns null for
anything else. Reauthorization is a 409 carrying
`details.code: MCP_SERVER_REAUTHORIZATION_REQUIRED`; an unreachable, slow, or
cooling-down server is a 503 with a constant message.
Also: widen the shared server path-param description now that it covers tool
listing, map the list query explicitly so no undeclared `cursor` reaches the
use-case input, and document the endpoint's write side effects.
* merge: bring in the MCP tool plane workstream
* feat(v2): make knowledge tags usable and let documents be updated
v2 accepted tag slots on upload and filtered search by tag display name,
but no response ever returned a tag value and nothing listed the
vocabulary, so a shipped feature dead-ended in the public API. A document
that failed processing could only be deleted and re-uploaded, and
retiring 500 documents cost 500 requests.
- GET /api/v2/knowledge/{id}/tags returns the vocabulary (display name,
slot, field type) as a full-set list.
- Document list and detail responses carry `tags`, keyed by display name
exactly as search keys its result metadata. Writes stay slot-keyed; the
tags endpoint is the mapping and the contract documents the split.
- PATCH /api/v2/knowledge/{id}/documents/{documentId} renames, enables,
disables, retags, or requeues processing. Derived indexing state is not
writable: asserting `processingStatus` on an unindexed document would
corrupt search. A retry may not ride along with field updates.
- PATCH /api/v2/knowledge/{id}/documents bulk-enables or bulk-disables.
Bulk delete is deliberately absent — that operation records no semantic
audit, and a public bulk delete would empty a knowledge base leaving no
DOCUMENT_DELETED entries.
- The document list accepts the same name-based `tagFilters` as search;
the name-to-slot resolver moves out of search into a shared helper, and
the filters are stamped into the offset cursor scope so a replayed
cursor cannot cross a filter change.
- Search accepts `rerankerEnabled`, `rerankerModel`, `rerankerInputCount`
and returns `rerankerScore`; `rerankerApiKey` and `skipUsageBilling`
stay unexposed. Every result now names its `knowledgeBaseId`.
knowledge.tags.list flips from workspaceApiKey 'deny' to 'allow' (and
gains the workspace_api_key principal kind) so it matches the sibling
reads knowledge.documents.list / read / search. The vocabulary is
required input for two operations a workspace key can already perform.
Every tag write stays human-delegated.
* fix(v2): name every 403 cause, unfork boolean params, close nested strictness holes
Four cross-cutting consistency gaps on the v2 public surface.
**403s now carry a machine-readable cause.** The conventions skill mandated
`error.details.code` on 403 and nothing emitted one, so a client had to
string-match prose to tell "raise this member's role" from "this workspace
refuses personal keys" from "buy an enterprise plan" — four different
remedies behind one status, and every message reword a silent break. The
vocabulary is a closed set, `FORBIDDEN_DETAIL_CODES`, with a `Record` of
descriptions beside it that the generated OpenAPI 403 description is built
from, so a code cannot reach the wire unpublished. Refusals throw
`ForbiddenOperationError` in the domain and `v2CaughtOrchestrationError` —
the function every v2 error policy falls through to — attaches the code, so a
route cannot forget it. The audit-log resolver distinguished four causes and
collapsed them into one; it now names each.
Cross-tenant refusals deliberately get no code: they are concealed as 404 and
naming their cause would hand back the existence signal the concealment
withholds.
**Two boolean query params rejoin the majority.** `?includeDeparted` and
`?includeOutput` were `'true'`/`'false'` string enums inherited from the
internal shapes they reused, while four sibling params were real booleans.
Both move to `booleanQueryFlagSchema`, which still coerces both strings — a
strict widening, so an existing caller is unaffected, and the spec stops
telling callers to send a string.
**Two nested strictness holes close.** `.strict()` binds the top level only,
so `sort: [{ field, direction, nulls: 'last' }]` was answered 200 with the
null-ordering request dropped, and an unknown key inside a saved view's
`config` was accepted and discarded — the headline `filter` bug one level
down. `sortSpecSchema`'s element and both view-config schemas are now strict.
Safe on the read side because `normalizeStoredViewConfig` projects the
schemaless stored blob onto the declared keys first, so a legacy row cannot
turn into a 500.
The two sort dialects stay as they are. `/logs` and `/workflows/{id}/runs`
have one sortable column, so there is no `sortBy` to pair with; renaming
`order` breaks every caller and an alias is a second spelling of one thing
with undefined precedence. Both contracts and the skill now state the rule.
* style: format the files the workspace-scoped lint gate does not reach
`turbo run lint:check` runs `biome check .` per workspace, so `scripts/` at the
repo root is outside the graph and four changed files were unformatted — one of
them a merge artifact from reconciling the route baseline across branches.
* fix(v2): collapse the four knowledge document projections onto one null-tolerant summary
Extracts toV2DocumentSummary in app/api/v2/knowledge/utils.ts and composes the
list, upload-acknowledgement and detail presenters from it. toV2TaggedDocument
serialized uploadedAt with a bare .toISOString(), so a document with no upload
timestamp threw where every sibling returned null and the contract declares the
field nullable.
Also consolidates the two Zod strictness walkers onto one shared introspection
helper that unwraps wrappers and expands unions, closing the hole where a
union-shaped schema answered null and was skipped by the pagination sweep.
* fix(v2): stop HEAD driving MCP discovery, and unbreak the updatedAt keyset page
B1: Next aliases HEAD onto GET, which RFC 9110 permits only because GET is safe.
The MCP tool-discovery GET is not: it opens a live connection to the registered
endpoint and writes the outcome onto the server row. The v2 JSON builder gains a
headSafe option, default true, and the discovery route declares itself unsafe —
a HEAD is authenticated and rate-limited, then answered bodiless.
B2: a discovery status write stamped updatedAt, which this branch added as a
keyset sort, so any concurrent discovery duplicated and skipped servers across a
caller's pages. Discovery liveness already has lastConnected, lastToolsRefresh,
lastError and statusConfig.
B4: a public refresh now skips the positive cache but keeps the failure cooldown,
so it cannot be used to drive a connection attempt per request at a failing
endpoint. An explicit user action on their own server keeps the full bypass.
B6: the consecutive-failure counter is incremented SQL-side rather than read,
incremented and written back, and the success branch carries the same workspace,
liveness and staleness guard the failure branch already had.
* fix(v2): bound the bulk update echo, close the search leak, and make the docs true
B3: a selectAll bulk document update echoed every changed identifier, which the
request does not bound — a 100k-document knowledge base produced a multi-megabyte
array, materialized and then element-wise validated. The use case now reports
whether the selection was unbounded and the presenter omits the echo.
A1: the knowledge search presenter spread the whole use-case result, which also
carries userId, workspaceId, a cost breakdown and a live secret-trace registry.
Only Zod's default key-stripping kept them off the wire. Projected explicitly.
P1-a: GET /knowledge/{id}/tags advertised all 17 slots while the document PATCH
accepted only the seven text ones. The writer already coerces every slot type,
so the PATCH now takes all 17 in their declared types, with a 400 where a
malformed value used to silently clear the tag.
P1-b: both new PATCHes deny workspace API keys and now say so.
P1-c: the two table query reads declare maxBodyBytes and now document the 413.
P1-d: getWorkflowDeploymentV2 loses its legacy suffix.
C3: deletes two orchestration error mappers with no callers that mapped
'forbidden' with no details.
D2: a stored null in table_views.config survived the pick and failed the
response schema.
Also folds the six 'bounded set' paraphrases onto one FULL_SET_LIST constant,
shares the run-window date bound between the logs and runs lists so their
documented parity is enforced rather than asserted, adds the missing barrel
export for FORBIDDEN_DETAIL_CODE_DESCRIPTIONS, and strictens two response
schemas whose peers were already strict.
Migrates 40 v2 route tests onto the shared @sim/testing harness: 26 asserted a
rateLimitSubjectIds shape v2 auth never returns, 26 asserted the wrong
refillRate, 33 could not exercise their 401 path at all, and 6 hard-wired the
rollout gate to null.
* fix(mcp): bound the connect handshake, and stop the 403 description over-claiming
B5: the connect clamp was getMaxExecutionTimeout(), the workflow ceiling of
seven days, so the real bound became the server row's own timeout — which the
registration contract permits up to 300s — times the connect retries. A slow
server could hold a Node request for roughly twenty minutes. Connecting is not a
workflow run, so the handshake now shares the one-minute ceiling tools/list
already applies to itself.
C2: the generated 403 description asserted that error.details.code names the
cause on every 403. Nine domain refusals still throw a bare forbidden
OrchestrationError and reach the wire codeless, so the wording now says 'where
the cause is one a caller can act on'. Reparenting those throws is left as a
deliberate change: one of them is a cross-tenant refusal that belongs in the
codeless class and would change its status.
* chore: reconcile the route ratchet with staging
* style: sort imports and format the three files biome flagged
* fix(openapi): import the forbidden-code constants from their module, not the application barrel
The barrel also re-exports the authorized use-case layer, which loads
@sim/db at import time. That pulled a database connection into the
OpenAPI spec check, so check:audits failed wherever DATABASE_URL is
absent, including CI.
|
||
|
|
2805a8def9 |
feat(windchill): add document integration (#6577)
* feat(windchill): add document integration * fix(windchill): align tool contracts and docs * fix(windchill): use official integration icon * fix(windchill): correct response and paging semantics * fix(windchill): align execution and API contracts * refactor(windchill): inline route authentication * fix(windchill): correct OData query encoding, content download, and cleared-field handling Validated the integration end to end against PTC Windchill REST Services 2.7 documentation and fixed every divergence found. Protocol correctness: - Encode OData query spaces as %20 rather than the form-encoded `+` that URLSearchParams emits. Every multi-token $filter and $orderby reached Windchill as a literal `+` and could not match. - Download content through the documented typed navigation `<content>/PTC.ApplicationData/Content/URL`, which returns a signed vault URL, instead of a `$value` segment that WRS does not implement. The resolved URL is pinned to the configured HTTPS origin. - Terminate every Stage 2 CacheDescriptor_array entry with `;` to match the documented grammar. - Raise the $top bound to Windchill's documented 2000 maximum, keeping 200 as the default page size. Cleared-field handling: - The executor merges raw block inputs before the block's param transform, so omitting a key could not clear it. A cleared numeric or boolean field reached the URL builder as '' and threw, and cleared optional strings failed contract validation. Coercions now emit an explicit undefined, and the internal-route body strips blanks centrally. Robustness and contracts: - Bound the document-structure walk to the depth actually requested. - Loosen response schemas that re-applied request-side bounds to provider-returned values, which turned committed mutations into opaque parse failures. - Return contract-shaped bodies for oversized, malformed, and unhandled request failures. - Normalize downloaded content types and drop charset parameters. Presentation and docs: - Square the icon to a centred tile on white. - Replace WT.Document and PATCH-compatible jargon with plain language. - Fix canvas sentence noun stutters on the bulk operations. - Correct the revision skill's unverified working-copy claim to read the OID back rather than assume it, and add retirement and stale-checkout skills. - Add a manual intro section to the integration docs page. * fix(windchill): align tool copy with the docs page and rebase the route baseline Tool descriptions feed both the integration catalog and the generated docs page, so the plain-language pass had to reach them too: drop WT.Document and PATCH-compatible from the operation copy, and correct the $top bound the descriptions still advertised as 200. Correct the docs intro's attachment wording, gloss OData on first use, and attribute the bulk-atomicity claim to PTC's documented behavior. Raise the API route-count baseline, which staging advanced while this branch was behind. * feat(windchill): add update common properties Name, Number, and Organization are rejected by the PATCH-based update operation, and the rejection message told users to reach for Windchill's UpdateCommonProperties action that the integration did not expose. Add it. PTC documents UpdateCommonProperties as a bound DocMgmt action taking an Updates wrapper, available when hasCommonProperties is set on the Documents entity, and refused while the document is checked out. The subblock and param descriptions carry that constraint, and the rejection message now names the operation that does the job. * test(windchill): assert block and tool params stay aligned for every operation Validating the new operation surfaced that nothing enforced the block-to-tool alignment the review process had been checking by hand. Assert it for all 27 operations instead: every required tool param has a required, non-advanced input under that operation's condition, and no operation shows an input its tool cannot accept. Both fail on a deliberately broken condition or a dropped required flag. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Waleed
|
81108d5a4e |
fix(v2): serve HEAD, advertise PATCH, and document the reachable 403 (#6623)
* fix(v2): serve HEAD, advertise PATCH, and document the reachable 403 Three HTTP-semantics defects on the v2 surface, all found by probing the published contract rather than the happy path. **HEAD answered 500 on every v2 endpoint.** Next implements a missing `HEAD` export by aliasing it onto `GET` and dropping the body when it sends, so a route's `GET` legitimately runs with `request.method === 'HEAD'`. The builders' method guard compared that against the contract's declared method and threw, so `HEAD /api/v2/workflows` and every sibling replied 500 — which is what health checkers, uptime monitors, link checkers, and some CDNs send, all of them reading the API as hard-down. RFC 9110 §9.3.2 makes HEAD identical to GET but for the body, which is exactly what running the GET path produces. Fixed once in `methodMatchesContract`, shared by all five route builders; every other mismatch stays a hard error so a handler exported under the wrong verb still fails loudly. **CORS advertised `GET,POST,OPTIONS,PUT,DELETE`** while the v2 spec has 17 `PATCH` operations, so a browser preflight for any of them was rejected. It also advertised `PUT`, which two operations use — the shape of a hand-maintained list outgrown by its surface. The list stays hand-written because middleware cannot import the contract tree without pulling Zod into the edge bundle, but it is now pinned by a test that sweeps the real contracts and fails on any method it omits. **Six operations omitted a 403 their siblings documented** — three knowledge reads and three file-upload operations. Traced from the code rather than the spec: `requirePermission` throws `NoWorkspaceAccessError` for no access at all (concealed as 404) but `InsufficientWorkspacePermissionsError` for access below `minimumRole` (a real 403), and `PersonalApiKeysDisabledError` reaches every operation a personal API key can call. So 403 was reachable on all six and the omission was an accident of hand-assembled error lists, not a policy. They now use the shared `RESOURCE_ERRORS` / `RESOURCE_CONFLICT_ERRORS` sets, and two operations spelling those same sets by hand were normalized onto them. All 128 documented operations now declare 403. The rules for HEAD, for the 403/404 split, and for using the shared error sets are recorded in `.agents/skills/v2-api-conventions/SKILL.md`. * test(proxy): update the CORS policy assertion to the served method list `proxy.test.ts` pinned the previous hand-written method string, so widening `resolveApiCorsPolicy` to advertise PATCH and HEAD left it asserting a list the middleware no longer returns. The literal is kept rather than imported from `proxy.ts` so the test still pins the exact wire value independently of the implementation. * refactor(v2): retire the error sets that could omit Forbidden The three knowledge reads and three upload operations lost their `403` by assembling `[...VALIDATED_ERRORS, ...]` by hand, and `VALIDATED_ERRORS` / `STANDARD_ERRORS` were the only exported sets that omit `Forbidden`. Migrating the last consumers to the shared `RESOURCE_*` sets left both unreferenced, so deleting them turns the fix from a one-time cleanup into an invariant: there is no longer a building block from which a workspace-scoped operation can assemble an error list without `Forbidden`. Regenerating the specs produces no diff, so the migration is output-neutral. Also folds `method-match.test.ts` into `definition.test.ts` to match the repo's `feature.ts` -> `feature.test.ts` convention, types `contractMethod` as `HttpMethod` so a contract declaring `HEAD` is unrepresentable, and drops the duplicated Next-aliasing rationale so `methodMatchesContract`'s TSDoc is its single home. * fix(cors): expose the API response headers a browser client needs Without `Access-Control-Expose-Headers` a browser can read only the six CORS-safelisted response headers, so the rate-limit budget, the `Retry-After` a 429 or 503 asks the caller to observe, and the request/run correlation ids were all on the wire but invisible to `fetch()`. Server-to-server callers were unaffected, which is why it went unnoticed. Exposed on the default `/api` policy only. The per-route `CORS_RULES` entries are wildcard-origin public endpoints and opt in individually if they ever need it, so this does not widen what an anonymous cross-origin caller can read from them. |
||
|
Waleed
|
6541a22aa6 |
fix(v2): tell a caller when to come back on every failure meant to be retried (#6625)
* fix(v2): tell a caller when to come back on every failure meant to be retried Three related gaps in retry signalling, found auditing the v2 surface against RFC 9110/6585 and against how Stripe, GitHub and Google's AIPs handle the same problems. **No 503 carried `Retry-After`.** Every one of them — the three route builders' `unhandledErrorResponse`, the execute and resume routes, and `serviceFailureResponse` — funnels through `v2Error`, so the default lands there, keyed on the response *status*: `Retry-After` is defined against the status, and the status is the only half of the code/status pair a client sees. A caller that supplies its own value still wins. RFC 9110 §15.6.4 makes this a `MAY` rather than a `SHOULD`, so it is a deliberate improvement, not a conformance fix: without it a client's only defensible policy on a 503 is an immediate retry, and Sim raises 503 exactly when a dependency is too degraded to absorb one. **A 429 that already knew its wait threw it away.** The admission descriptors declare `retryAfterSeconds` per denial, but mapping a descriptor onto a preprocess error copied only `statusCode`, `code` and `retryable`. A concurrency denial therefore reached the client as a bare 429 with no `Retry-After` despite the policy layer having named the wait five seconds earlier. The value now travels `descriptor.retryAfterSeconds` → `PreprocessExecutionError.retryAfterMs` → `ExecuteWorkflowServiceFailure.retryAfterMs` → `serviceFailureResponse`, so the transport reads a number the policy owns instead of re-guessing one. The 503 default is now only the floor for paths with no policy signal. **One failure must not advise a retry at all.** `ASYNC_ENQUEUE_AMBIGUOUS` is a 503 whose enqueue may have succeeded — it deliberately retains its execution-ID claim because a job may already exist. Telling that caller to come back in five seconds invites a client with no `X-Run-Id` to start, and bill, a second run of the same workflow. It opts out via `omitRetryAfter` and returns the run id so the caller reconciles instead. `ADMISSION_RETRY_AFTER_SECONDS` is reused rather than restated, so the execute route's capacity 429 and every other surface's 503 cannot drift apart. Also records the audit in `.agents/skills/v2-api-conventions/SKILL.md`: the retry rule, the cursor-tampering invariants, and reasoned rejections of RFC 9457 problem+json, the `RateLimit-*` draft fields, renaming `X-RateLimit-*` under RFC 6648, 422-for-semantic-validation, `Location` on 201, ETag/`If-Match`, and `merge-patch+json` — each with the spec text and the industry evidence, so they are not re-litigated. `Deprecation`/`Sunset` on v1 is left open pending a retirement date, which is a product decision. * docs(v2): name the one 503 that omits Retry-After in the shared contract The shared ServiceUnavailable description claimed every 503 carries the header, which the ASYNC_ENQUEUE_AMBIGUOUS response deliberately does not. It now says the header is normally present and names that exception, so the published contract matches the runtime behaviour for all 128 operations. |
||
|
Waleed
|
9dfd9db3b0 |
feat(xai): wire reasoning effort through the Grok adapter (#6627)
* feat(xai): wire reasoning effort through the Grok adapter The catalog never declared reasoningEffort for xAI and the adapter never sent reasoning_effort, so the flag was dead for every Grok model. Values are per-model and verified against the live API rather than the docs, which are wrong in three places: grok-4.5 does accept xhigh, grok-4.3 supports the parameter at all (undocumented) including none, and grok-4.20-0309-reasoning rejects it outright despite being a reasoning model. Also corrects grok-4.5's missing cachedInput and drops an inline comment the new provider TSDoc now covers. * test(xai): type the provider test helper instead of casting to any * fix(agent): correct reasoning-effort copy that still claimed GPT-5 only |
||
|
Theodore Li
|
afa02939bb | fix(docs): include API key header in generated code samples (#6630) | ||
|
Waleed
|
5cf60f6a41 |
fix(v2): give every collection one pagination contract and close four envelope holes (#6620)
* fix(v2): give every collection one pagination contract and close four envelope holes
A fractional `limit` reached Postgres as `LIMIT 2.5` and answered 500 on both
`GET /workflows` and `GET /audit-logs`: each list re-declared the param inline,
and these two copies lost their `.int()`. The same divergence left `limit`
validated five different ways and five collections emitting `nextCursor` while
accepting no `limit` at all, or accepting one and silently discarding it.
Adds `v2PaginationFields()` in `contracts/v2/shared.ts` — a bounded integer
`limit` and an opaque `cursor` — and adopts it across all 17 paged lists, so the
family cannot drift again. `/files`, `/logs` and `/tables` keep the truncate-and-
clamp leniency they published, now as an explicit named mode rather than three
hand-rolled copies.
Gives `/skills`, `/custom-tools`, `/secrets`, `/credentials` and `/knowledge`
real pagination using the existing cursor codecs: a keyset for the four whose
page comes from one ordered SQL read, and the offset cursor for `/skills`, whose
merge of the static builtin registry into DB rows cannot be expressed as a SQL
keyset. Each keyset sort now ends in a unique `id`; knowledge tie-broke on
`createdAt`, which cannot separate rows sharing a millisecond.
Two correctness fixes pagination forced: the secrets visibility filter moved from
a post-query JS pass into SQL, because trimming rows after the page is cut
returns fewer than `limit` while `nextCursor` claims more; and the skills list
stopped selecting the 50k-char `content` column only to discard it.
Also restores the canonical error envelope where it had holes: a malformed JSON
body returned a bare `{"error":string}` because the envelope was a per-route
opt-in only 8 of 77 routes remembered, and an unknown `/api/v2` path returned an
HTML 404. Both are now defaults — `V2_PARSE_DEFAULTS` on the builders and the two
raw routes, and a catch-all whose body is byte-identical to the rollout gate's so
an unknown path stays indistinguishable from an ungated one.
Consolidates the keyset paging block (`resumeKeyset`/`keysetPage` in
`list-query.ts`) that had been open-coded in six modules, and folds the bespoke
`InvalidWorkflowListCursorError` into the `OrchestrationError` every other list
already used.
Prevention: the contract sweep in `list-pagination.test.ts` now also asserts that
every paged list rejects a fractional `limit`, that every list query is
`.strict()`, and that the three clamping lists still truncate. The fractional-
limit assertion is what caught `/audit-logs`. Documented in
`.agents/skills/v2-api-conventions/SKILL.md`.
405 responses still carry no `Allow` header — Next.js generates those before any
handler runs. Recorded as a known gap.
* fix(v2): bind the offset cursor to the query state it counts positions in
An offset names a position in one exact sequence. `GET /skills` accepted a bare
`{offset}` cursor and applied it to whatever sequence the next request asked
for, so following `nextCursor` with a different `search`, `sortBy` or
`sortOrder` silently skipped rows, repeated them, or landed past the end and
returned an empty page while the cursor implied more.
Fixed in the codec rather than the route so the sibling could not keep the gap:
`decodeOffsetCursor` now takes a scope stamp and rejects a cursor minted under a
different one, which is what `decodeSortedCursor` has always done for keysets.
`offsetCursorScope()` builds the stamp from every param that filters or orders
the sequence; `limit` is excluded because it selects how much of the sequence to
return, not what the sequence is, so paging with a different page size still
works.
`GET /knowledge/{id}/documents` had the identical latent gap and gets the same
treatment — the compiler surfaced it as soon as the signature changed.
|
||
|
Waleed
|
366829b6b0 |
fix(v2): derive the log and run status enums from the persisted status list (#6612)
* fix(v2): derive the log and run status enums from the persisted status list
`GET /api/v2/logs` and `GET /api/v2/logs/{runId}` parse the raw
`workflow_execution_logs.status` column against a six-value enum that omits
`paused`, so a run holding that value returns 500. The list response is
validated whole-page, so one such row 500s every page it lands on, and the
row is durable until the run is resumed, cancelled, or failed.
`paused` is not written by an ordinary human-in-the-loop pause — that path
persists `pending` (logging-session.ts:1180). It is written by
`PauseResumeManager.markResumeAttemptFailed`, which fires on any
`ResumeAdmissionError`: a workspace over its usage limit, an archived or
undeployed workflow, or a concurrent resume losing the claim race. That is a
routine business path.
The enum was supposed to be protected by an `AssertNever` exhaustiveness gate,
but the gate was vacuous: it compared against `PersistedWorkflowExecutionStatus`,
a hand-written union that was itself missing `paused`, because the write goes
through a raw `sql` CASE fragment Drizzle cannot type-check. Adding `paused` to
both lists would leave the same vacuous gate in place for the next status.
Instead, `PERSISTED_WORKFLOW_EXECUTION_STATUSES` becomes the single runtime
source of truth, `PersistedWorkflowExecutionStatus` is derived from it, and both
v2 contracts derive their enums from the const rather than re-declaring them.
Both surfaces pass the column through verbatim, so their reported set is the
persisted set by definition — there is no editorial choice for a gate to force,
only the question of whether a newly persisted status should be public, which
the option-list tests now pin. The `[...V2_PERSISTED_RUN_STATUSES, 'paused']`
append on the runs contract is deleted rather than adjusted; it would otherwise
be a duplicate.
Alternatives rejected:
- A `.catch()` or `safeParse` in the presenters is dead code:
`v2-json-route.ts:271` re-parses the whole body with the same schema.
- Normalizing `markResumeAttemptFailed` to write `pending` would remove the
distinction the resume claim query at human-in-the-loop-manager.ts:973 relies
on, and leaves the contract wrong for any other future status.
- Typing the Drizzle column does not help: the offending write is a raw `sql`
fragment, and `packages/db` cannot import the app's status list.
The v2 workflows spec changes are reordering and description only — the value
set there already contained `paused`. The v2 logs spec gains `paused`, which is
additive and safe while the whole `/api/v2` surface is behind the off-by-default
`v2-api` flag; it must land before v2 GA, after which it would be breaking.
* fix(v2): document both provenances of a reported paused run status
* fix(v2): stop promising a paused discriminator the response cannot always provide
* fix(v2): describe the paused discriminator as the code actually records it
|
||
|
Waleed
|
47f143016e |
fix(docs): restore api-reference URL continuity and fix translated SDK bodies (#6617)
Two docs-only defects introduced by #5273 (`263e3ca67e`), which re-founded the public API reference on the v2 surface. 1. Ten translated SDK snippets produce a deterministic 400. The streaming example in the five translated `api-reference/typescript.mdx` and `python.mdx` pages was repointed from `/api/workflows/{id}/execute` to `/api/v2/workflows/{id}/execute` and nothing else was changed — fr/ja/zh typescript.mdx are literally one-line diffs. `message` stayed at the body root. That was correct against v1, whose route treats the whole non-control body as workflow input, but `v2ExecuteWorkflowBodySchema` ends in `.strict()` and the route parses before executing, so every copied snippet returns `400 Unrecognized key: "message"`. The same commit fixed the English bodies to `input: { … }`, so this is an oversight, not a decision. The ten fences now match `en/api-reference/typescript.mdx:959` and `python.mdx:681`. Not relaxing `.strict()`: it is deliberate house style across the v2 contract and is what makes a typo'd option fail loudly instead of silently. 2. Thirty-two published operation pages 404 with no redirect. Replacing the single v1 `openapi.json` with seven v2-only specs changes page identity, because fumadocs derives every generated page as `slugify(tag)/operationId` from the specs at build time. Re-deriving both sets gives 52 old slugs and 128 new ones: 32 disappear and 20 keep their URL while silently retargeting v1 -> v2 (`knowledge-bases/updateKnowledgeBase` also flips PUT -> PATCH). All 52 are in the live sitemap — parsing `<loc>` from docs.sim.ai/sitemap.xml gives 458 URLs of which 56 are `/api-reference/`: the four static pages plus all 52 generated ones by name, including every one of the 32 that die. They are 200 today under an allow-all robots.txt. The spec swap itself is deliberate and CI-enforced (`check-openapi-specs.ts` requires every published operation under `/api/v2/`), so restoring the v1 operations is not an option. The missing piece is the redirect map, in a file that already carried 56 such rules from earlier doc moves. `permanent: true` (308) is used only for a true 1:1 successor — same operation, renamed. A 308 is cached indefinitely and effectively unrecallable, so anything that collapses two pages onto one, changes the identifier model, or lands on a merely adjacent operation is `permanent: false` (307). That splits 21/11. Four destinations differ from the mapping proposed in review, each on evidence from the specs rather than from the operation names: - `workflows/getJobStatus` is not destination-less. The v2 queued-execution receipt (`QueuedWorkflowRun`) returns `statusUrl` `/api/v2/workflows/{id}/runs/{runId}`, so `workflow-runs/getWorkflowRunV2` is the successor poll target — far better than a generic landing page. - The three HITL read operations go to `getWorkflowRunV2`, not to the resume page: `WorkflowRunStatus` carries a `paused` object with `contextId`, `pausedAt`, and `pauseKind`. Pointing a GET doc at a POST doc would be wrong. - `human-in-the-loop/listPausedExecutions` goes to `listWorkflowRunsV2`, whose `status` filter includes `paused`. - `tables/batchUpdateRows` is 307, not 308. v2 `updateTableRows` is "Update Rows by Filter" — the successor of v1 `updateRows` (PUT, predicate-based), which keeps its 308. v2 has no by-id batch update at all, so batchUpdateRows lands on a genuinely different operation. 3. A guard, so the map cannot rot silently. `scripts/openapi/docs-redirects.test.ts` recomputes the generated slug set the way fumadocs does and asserts no `/api-reference/` source shadows a live page and every destination resolves. Nothing else in the repo reads docs URLs, so a future spec regeneration would otherwise break the map with no signal. It needs no wiring: `check-openapi.ts` already runs this vitest config. The redirect array moves to `apps/docs/lib/redirects.ts` because the guard cannot import `next.config.ts` — `createMDX()` runs the fumadocs-mdx generator at import time, which made vitest emit an unhandled build error and warn about false positives. The 56 pre-existing rules are byte-identical to before, verified programmatically; `next.config.ts` keeps the same public shape and Next's own `checkCustomRoutes` accepts all 88 rules. Open question for the owner, larger than the redirects: all 82 `/api/v1` route files survive on staging, so a live public API now ships with zero reference docs, while the documented `/api/v2` surface returns 404 for any caller outside the off-by-default `v2-api` flag cohort. Is that the intended end state or transitional? |
||
|
Waleed
|
34d65df7d6 |
fix(sdk): make the 0.2.0 SDK release safe to publish (#6616)
The v2 SDK migration (#5273, #6564) shipped five breaking changes in both
SDKs but got the release mechanics wrong in three separate ways, and left
one of the two rewrites unable to complete a single successful call.
Versions. packages/ts-sdk/package.json read 0.1.3 -- a patch digit added
inside an unrelated compatibility commit, never deliberated. npm expands
^0.1.2 to >=0.1.2 <0.2.0, so every existing consumer would have picked the
break up on a lockfile refresh: AsyncExecutionResult.jobId renamed to
runId, executionId dropped from that interface, a failed sync run now
throwing instead of resolving {success:false}, the request body reshaped,
and the endpoint moved to /api/v2 with no fallback. 0.2.0 excludes every
existing range, so the upgrade becomes opt-in. packages/python-sdk carries
the identical break and was never bumped at all, so its publish job would
have skipped green at the "version already exists" gate and left the repo
and PyPI silently divergent; it moves 0.1.2 -> 0.2.0 in lockstep, along
with the __version__ string in simstudio/__init__.py, which tracks
pyproject and would otherwise have started lying. setup.py is left at
0.1.1: it is unchanged from main and demonstrably unread (0.1.2 published
from pyproject while setup.py already said 0.1.1). It wants deleting, in
its own commit.
A 404 fallback was considered and rejected. The legacy 202 body's statusUrl
points at /api/jobs/{jobId}, so mapping jobId onto runId would hand the
caller an id that getWorkflowRun cannot resolve against that same old
server -- a successful execute followed by an inexplicable failure on the
next call is a worse contract than a clean 404. Both READMEs instead state
the minimum server version and name the endpoint to check for.
Cancelled runs. packages/python-sdk computed success as status != 'failed',
so a run cancelled out of band reported success=True. The TypeScript SDK
uses a closed whitelist and reports False, and before the migration both
SDKs read the server's own value, which was False -- so this was a Python
regression, not merely an inconsistency. Fixed by mirroring the whitelist.
The v2 contract enumerates exactly completed|failed|paused|cancelled, so
narrowing the blacklist to a whitelist cannot drop a live value, and a
status added later now defaults to "not successful" rather than silently
reporting True. WorkflowExecutionResult gains a status field because
Python, unlike TypeScript, does not throw on 'failed' -- so success=False
alone is ambiguous there in a way it is not in the TypeScript SDK, which
is why status is not added to both.
Rate-limit header. Found while auditing the two SDKs for further
divergence, and the reason the Python bump could not have shipped as it
stood: every authenticated v2 response now carries X-RateLimit-Reset as an
ISO 8601 timestamp (recorded by v2RateLimits.publicApi, stamped by
withRouteHandler). The Python SDK parsed it with int(), raising a bare
ValueError that no handler in execute_workflow catches -- so every
successful v2 execution raised instead of returning. None of the legacy
endpoints the SDK previously called record a rate-limit snapshot, which is
why the latent int() survived until the v2 move. The TypeScript SDK
already branches on the format; _parse_reset_header mirrors it, including
degrading an unrecognised value to 0, because a quota hint must not take
down the call it rode in on.
Timing metadata. The v2 rewrite stopped forwarding startedAt/endedAt, which
main passed through and the TypeScript SDK still reports; restored under
the same startTime/endTime keys the TypeScript SDK uses.
Tests: cancelled/failed/paused status coverage, the ISO reset header, and
the restored metadata keys, each verified red against the unfixed line
first. The TypeScript suite gains matching cancelled/paused and ISO-reset
pins -- they pass against today's source by design, and were confirmed to
fail against a deliberately degraded copy so they are not toothless.
Deliberately not included: a CI guard failing a PR that changes SDK source
without a version bump. It would have caught this twice over, but it is a
new script and workflow rather than a fix to the defect at hand.
Review revision. bun.lock recorded packages/ts-sdk at 0.1.3 and was left
stale by the first pass, so the repo asserted two versions for the same
workspace package -- in a change whose whole thesis is that the version
strings had diverged. It does not break CI (bun 1.3.14 accepts the
mismatch under --frozen-lockfile, confirmed here), but
|
||
|
Waleed
|
892401a8d0 |
fix(uploads): sign Azure upload URLs create-only (#6607)
getBlobPresignedUploadUrl signed its SAS with BlobSASPermissions.parse('w').
Per Azure's service-SAS reference, `w` is "create or write content" and permits
overwriting an existing blob; `c` is "write a new blob" and does not. main
signed `c` before this signer moved out of core/storage-service.ts, so Azure
deployments lost create-only enforcement in the move.
The `If-None-Match: '*'` the signer returns cannot carry the guarantee on its
own: an Azure service-SAS string-to-sign covers the resource, times,
permissions and the five rsc* response-header overrides, never request headers,
so a client is free to drop it. The header is signed on the other two providers
-- inside the PutObjectCommand on S3, and as x-goog-if-generation-match in
signed extensionHeaders on GCS -- which is why only Azure regressed.
Without this, a signed upload URL stayed a plain overwrite grant on the final
key for its full hour. A caller could replace the object after complete had
already verified size and content type, written the workspace file row and
metered storage from that verified HEAD, leaving durable metadata and billing
describing content that no longer exists.
The multipart block-staging signer keeps `w`: block staging is overwrite-shaped
and matches main.
The existing test asserted parse('w'), so it locked the defect in; it now
asserts create-only, and its name states the guarantee so a future flip reads
as deleting a security property rather than adjusting a value.
|
||
|
Theodore Li
|
23318a13a0 |
fix(security): redact workflow snapshot secrets (#6581)
* fix(security): redact opaque workflow snapshot inputs * fix(security): document fail-closed tool redaction * fix(security): redact malformed tool params * fix(security): redact nested credential references * fix(security): isolate opaque tool schemas |
||
|
Justin Blumencranz
|
a72457b9f5 | fix(docs): preserve items response fields (#6587) | ||
|
Waleed
|
7c05e36049 |
fix(api): close five defects found auditing the v2 migration against main (#6575)
* fix(tables): stop a column retype from nulling empty-string cells A type conversion rewrote every cell holding '' to null. Main only nulled a blank the target type could not read; '' is a real stored value that both string and json columns accept, so string->json and json->string silently destroyed those cells. Worse on a required target: countEmptyCells matches only a missing key, SQL NULL, or '[]', so '' passes the required guard and the rewrite then wrote null behind a constraint that had just succeeded. The per-cell decision is now the pure retypeCellRewrite, restoring main's rule: null a blank only when the target cannot read it, otherwise coerce. * fix(execution): release the concurrency slot when a group cancel is refused The stop-the-work effects (durable Redis abort record, queue-job cancel, in-process abort) all fire before the workflow-group sidecar is consulted, and none can be undone. When the sidecar refuses the claim we throw a conflict, which skipped releaseExecutionSlot and stranded the plan concurrency reservation until it expired. Every conflict return is a terminal-or-absent state - a missing log row, a log already completed or errored, or a terminal cell - so a refusal never means the run is still executing. The slot is released before the throw, keeping the exact success && !isPausedCancellationPath predicate rather than a blanket finally that would free reservations for live runs. * fix(uploads): recover an ambiguous PUT instead of discarding the object Main recovered an upload whose bytes committed but whose response was lost, via a verify endpoint. The session client retries the PUT instead, but every provider now signs a create-only precondition, so the retry returns 409/412, is classified non-retryable, and the session aborts - deleting the object that had already landed. A transient blip on the final ack cost the whole upload. A conflict on a retry attempt is now treated as our own earlier PUT having committed, and completion proceeds. That is safe because completeUploadSession independently verifies the object through assertObjectIdentity, which rejects on uploadId mismatch before anything durable is registered. A first-attempt conflict still fails loudly. * fix(folders): enforce the workspace folder ceiling on the create path Readers bound the active path index at MAX_FOLDERS_PER_WORKSPACE and throw once a workspace exceeds it, but POST /api/folders reached createFolder, which has no maxFolderRows field and never counts. A workspace could therefore be driven past the ceiling, after which the 27 capped read sites failed on a state the product had allowed. createFolder now asserts room inside its transaction, right after the mutation lock, so the count cannot be raced. The refusal is a typed conflict rendering 409 with an actionable message rather than a 500. The check counts rows directly instead of loading the path index, so an already-over-cap workspace gets a clean refusal rather than a read error, and no reader gained a cap. folderMutationStatus also gained the payload_too_large mapping it was missing, which had been rendering a delete-cascade cap breach as an unexplained 500. * fix(skills): route internal skill writes through the shared use cases The internal route made the workspace authorization decision itself, never consulting the skills operation policy, never loading canonical workspace context, and recording an audit entry with no operation id or actor projection. v2 and Copilot already went through the use cases; only this surface did not. GET/POST/DELETE now authenticate, parse, call the shared use case, and present. Request and response shapes are unchanged. Two behavior changes fall out: a write against a deleted workspace is now refused with 404 rather than accepted, and permission-denial text matches the rest of the platform. Legacy internal-JWT auth is dropped because no principal kind expresses that caller and nothing calls it: the whole repo references /api/skills only in two comments, no tool declares an internalRoute to it, and the executor reads skills through a direct listSkills call rather than over HTTP. * fix(folders): enforce the workspace ceiling on the remaining create paths Folder duplication, admin workspace import, and workspace forking all inserted folders without consulting the ceiling that 27 read sites enforce, so any of them could leave a workspace whose reads then fail. Each now asserts room for the rows it is about to add rather than one at a time: duplication measures the whole subtree up front, forking counts its bulk insert, and import counts per segment because that is genuinely one row. assertFolderCollectionHasRoom gained an additionalRows notion for the bulk case, and short-circuits when nothing is being added so an over-cap workspace still reads and still syncs. Duplication deliberately does not take the folder mutation lock. Holding it across the copy would block folder creation workspace-wide for an unbounded time - there is no cap on workflows per subtree and duplicateWorkflow runs sequentially - and narrowing it is impossible because an advisory transaction lock cannot be released early; splitting the transaction would leave a half-copied tree on failure. A rare few-row overshoot near the ceiling is the better trade, and it matches what forking already does. A test asserts the lock is absent so re-adding it is a visible decision. Admin import gained the transaction and lock it never had. Its folder-full refusal escapes the per-workflow result list, because a full tree is a property of the workspace and would otherwise be buried as N failures behind a 200. The fork and promote routes had no catch at all, and withRouteHandler only classifies HttpError, so a refusal rendered as an opaque 500 - twice over, since drizzle wraps the throw. Both now project a classified conflict as 409 and rethrow anything unclassified. * fix(uploads): bound the signed PUT lifetime and advertise its real expiry A single-PUT transfer was signed for the whole 24h upload-session TTL, because expiresAt was reused as both the session lifetime and the signing lifetime. Multipart part URLs in the same file kept 1h, and the pre-migration presigned route signed every PUT for 1h, so the widening was unintended rather than a policy change. No provider clamps below 24h. The PUT presign is now clamped at the provider boundary by a shared UPLOAD_URL_TTL_MS, which the part-URL path also uses so the two cannot drift. An expired PUT URL is deliberately not recoverable: unlike multipart, which re-signs per part call because its progress is durable, a PUT is not resumable, so an expired URL and an interrupted PUT have identical recovery. Nothing leaks, since every provider signs a create-only precondition. Clamping alone would have made the contract lie: the URL would die an hour before the session's advertised expiresAt, with nothing telling an integrator why the 403 happened. The PUT transfer now carries its own expiresAt, mirroring the multipart part-URL field. It is provider-dependent on purpose - cloud transfers report the clamped signature expiry, while the local data plane has no signature and admits against the session, so reporting an hour there would have been a new inaccuracy in the other direction. * chore: delete the dead presigned-upload and skill-adapter paths The presigned upload routes and the internal skills adapters were both replaced during the v2 migration, leaving their implementations behind with no callers. Removed generatePresignedUploadUrl and verifyPresignedUploadReceipt with their three provider helpers, QUOTA_EXEMPT_STORAGE_CONTEXTS and the types it orphaned, and the performCreateSkill/performUpdateSkill/performDeleteSkill adapters with recordSkillEvent and statusForSkillOrchestrationError. Each was verified unreachable across apps, packages, scripts and ee, including barrel re-exports and string access, not just direct imports. recordSkillEvent needed the closest look, since deleting an audit writer can silently drop coverage. The use cases declare the same action, resource, and description, and the framework adds the operation and actor the old helper lacked; recordAudit back-fills actorName and actorEmail from the user table when both are omitted, so the one field the helper passed is not lost. The self-hosting architecture doc described a directUploadSupported flag on an endpoint that no longer exists, and now describes the upload-session flow that replaced it. * refactor(folders): keep the cheap resource facts out of the schema graph Reading a folder resource type's label or its lock support meant importing folderResourceConfig, which imports the db schema for every table it serves and from there reaches lib/table/service, the executor, and the tool registry. That mattered as soon as lib/folders/queries needed a label: queries is reached from workspace-file-manager, which is reached from the files and chat pages, so one import edge put roughly 4,700 modules into those page graphs and broke the tool-registry boundary audit. Labels and lock support now live in a leaf module that imports only a type, and config composes them so there is still one source of truth. The three folder routes that pulled the whole config in for a single boolean read the leaf instead. * fix(skills): apply an upsert batch in one transaction The internal skills route looped the batch, calling an independently committing use case per item. A rejection on a later item left the earlier ones written and audited while the request reported failure - the compound-mutation rule in CLAUDE.md exists for exactly this. No new transaction plumbing was needed: upsertSkills already wraps its whole item loop in one db.transaction, so the partial commit came from calling it N times rather than once. upsertSkillBatch now validates and per-skill authorizes every item before issuing a single write, and createSkill and updateSkill became thin wrappers over it so v2 and Copilot keep one authority for the rules. The compound operation declares the read floor that skills.update already used, and the use case additionally authorizes skills.create when any item lacks an id, still ahead of every write. A read-only member who is a skill editor keeps their edit, and creates are not authorized more loosely than before. Audit projects one entry per committed skill, and analytics moved after the commit so nothing is reported for a rolled-back item. Note metadata.operation for these writes is now skills.upsert rather than skills.create/update; the action field still carries the distinction. * fix(security): close two disclosure gaps and finish the slot-leak fix The payer-pool gate only covered the workspace branch. A personal API key that omits workspaceId takes the account branch, where getHighestPrioritySubscription resolves an organization subscription from any member row regardless of role - so a plain member read the organization-wide credit and storage pool by dropping one query parameter. The account branch is now gated by the same authority, and the storage pool is not queried when it may not be disclosed. Forcing that branch self-scoped instead would have downgraded plan, period, and status, which is what a member needs to see whether the org is blocked. GET /api/v1/logs/executions/[executionId] emitted the workflow snapshot raw, carrying password sub-block values and oauth-input credential ids. It now shares the sanitizer the v2 read already used, extracted so there is one implementation rather than two. Env-var references are still preserved. cancelWorkflowGroupExecution itself was unguarded, so an unexpected throw from its transaction escaped ahead of every release site - the same reservation leak this branch set out to close, still open on the adjacent path. It now releases through the shared predicate and rethrows, because a failed transition means the cell state is unknown and a success-shaped answer would be a lie. The comment claiming the abort record cannot be taken back was false and now states the real reason: a refusal is always a terminal-or-absent state. * fix(v2-api): cover every persisted run status and every capped body The workflow-runs endpoints carried the same omission the logs contract had: the execution logger persists redacting, the run schemas did not list it, and because validation is whole-response one such row returned 500 for an entire page. Both schemas now derive from PersistedWorkflowExecutionStatus behind the same AssertNever gate, so a future status is a type error rather than a production 500. The single-run read keeps its extra queued value, which only it can observe. That schema was also serving as the run-list status filter. Widening it would have accepted a filter value the application input cannot express, so the reported set and the accepted filter are now separate schemas. Routes declaring maxBodyBytes without payloadTooLargeResponse fell back to a bare string with no error code and no private cache header. Rather than patch the four, the default moved into the builders - all three had the hole - which covers 58 body-bearing handlers, and a route override still wins. The five per-route overrides that merely restated the default are gone. Also documents the 413 on the one knowledge route that has a real body cap, adds the rollout gate's 404 to the last v2 operation missing it, and rewords the nextCursor description, which read as though every list were a full-set list. * fix(api): make the shared traits and lifetimes single-sourced The folder resource-traits leaf composed labels into config but restated lock support independently, so the routes reading the trait and the orchestration reading the config could disagree about which resources lock. Config now composes both, and supportsLocking is required rather than optional so a new resource type cannot silently omit it. The upload commit claimed the PUT clamp and the multipart part URLs shared a constant and could not drift. That was true only of the advertised expiry - the three provider signers each hardcoded an hour, so changing the constant would have moved what we advertise while leaving what we sign, recreating exactly the mismatch the clamp removed. Each provider now receives the lifetime in its own unit from the one constant. Table restore hand-rolled its status map and returned the driver message verbatim at 500, leaking the failed statement and its bound parameters - the same defect this branch closed at nine other sites. It and import-csv now use the shared projection; import-csv's result type also had to carry the lock the classifier already set, so a 423 can name it. Deletes v2RowWriteError, which had no callers and would have rendered a locked table as 400 by discarding the 423 it was handed. |
||
|
Theodore Li
|
9f3c20290b | fix(tables): cap row pages and use native cursors (#6582) | ||
|
Siddharth Ganesan
|
7f39678264 |
improvement(desktop, executions): fix desktop update script and throw error when thinking enabled on stream false (#6578)
* Fix update check * fix include tool calls error * fix comments |
||
|
Theodore Li
|
092311ea68 |
fix(api): restore migrated endpoint and SDK compatibility (#6564)
* fix(api): restore migrated endpoint compatibility * fix(api): close remaining migration regressions * fix(files): validate ensured folder paths * fix(files): project folder path validation * fix(files): align archive regression fixture * fix(tables): avoid partial bulk update failures * fix(auth): project legacy knowledge audits |