mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-21 13:00:04 +08:00
81ff24a2fc0f5fadb6b8361cb54cd2a6c260287a
940
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vikhyath MondretiandClaude Opus 5
|
81ff24a2fc |
improvement(secrets): gate Copilot code mounting at use level (#7004)
* improvement(secrets): gate Copilot code mounting at use level Mounting a saved secret into Copilot code required credential-admin on that key, while a workflow Function block resolves the same secret for the same person at use level through getPersonalAndWorkspaceEnv. Copilot reaches that path itself — edit_workflow plus run_workflow — so the admin bar contained nothing. It redirected a Credential Member through a detour that mutates a persisted workflow, while the direct path is ephemeral and files a usage row. The inconsistency was also internal to Copilot: the secret names advertised to the model come from getAccessibleEnvCredentials and getPersonalAndWorkspaceEnv, both role-agnostic, so Copilot listed every secret the caller could use and then refused to mount all but the admin ones. Widen the workspace and shared-personal predicates to any active grant, and drop the matching role filter from the query. Workspace write is still required, revoked and pending grants are still refused, and a caller with no grant still gets nothing. The view gate stays where Copilot cannot route around it: values remain masked under Settings, and See usage remains admin-only, so a member's use is recorded for whoever can rotate the key. Model-egress projection is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(secrets): stop implying Personal secrets are shareable The Copilot code-execution paragraph listed "any secret shared with you as a Credential Member or Credential Admin" among what mounts, which reads as though a Personal secret can be shared. It cannot through any product surface: CredentialMembersSection renders only for workspace secrets and OAuth credentials, and the personal-credential sync only ever grants the owner. Narrow the sentence to Workspace grants. The comparison table's "Only you can use" row for Personal was correct and is left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Theodore Li
|
8104c33bba |
Fix knowledge connector sync follow-up (#6927)
* Fix knowledge connector sync follow-up * Fix connector sync pause race * fix(knowledge): surface connector sync dispatch failures * fix(knowledge): make connector sync recovery durable * fix(knowledge): deduplicate connector sync dispatches * fix(knowledge): preserve pending connector syncs * fix(knowledge): lock connector sync snapshot |
||
|
|
c26529a82e |
feat(bitbucket): add repository webhook triggers (#6934)
* feat(bitbucket): add repository webhook triggers * fix(bitbucket): harden webhook trigger delivery * fix(bitbucket): address final trigger review * chore(bitbucket): address review conventions * fix(bitbucket): harden triggers and connector sync --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Waleed
|
8937bb3550 |
fix(kb): let the server say a connector sync is queued (#6968)
* fix(kb): let the server say a connector sync is queued The connector chip inferred "a sync is coming" from `createdAt` inside a 2-minute window, because nothing on the row distinguished a queued sync from an idle connector until a worker took the lock. The guess was wrong under queue backlog and under client clock skew, and it forced a pile of client state to stand in for it. Adds `pending`, written as the sync is handed to the queue and cleared when a worker takes the lock or the hand-off is found to have been lost. It is a phase of the same lock `syncing` holds, so it opens the lease and takes an ownership token the same way — the lease is what the scheduler ages a stranded queue entry against (`updatedAt` cannot serve: a pending connector is still editable, so any unrelated write would renew the recovery it should trigger), and the token is what proves a late release belongs to this dispatch. Deletes the 2-minute window, the in-flight id sets, the 5-minute cooldown timers and the forced re-render they needed. The cooldown lived in a ref inside a modal, so it evaporated whenever the modal closed; the disable now comes from durable server state and is shared across tabs. Also fixes, all found while tracing the lifecycle: - An on-demand sync on a paused or disabled connector silently resumed it for good. Nothing could put the pause back: success writes `active`, a lost queue entry writes `error`, and the due-sweep keeps syncing that. Refused. - A failed hand-off no longer advances the connector's auto-disable breaker. A queue outage would otherwise increment every connector in the fleet until they all disabled themselves for a fault that was never theirs. - Manual sync on an established connector gave no feedback at all: the poll only ran while the predicate matched, which it never did. - Four over-broad invalidations that refetched every cached chunk page and chunk search in a base when one connector document was excluded. - The dead-process reporter re-sent a PATCH per stale document on every poll. * fix(kb): refuse to start a queued run on a paused connector The queue outlives the decision to sync. Pausing a connector after its run was queued cleared the queue entry's token but left the task itself alive, and the lock CAS accepted any row that was not already `syncing` — so the worker took the paused row and wrote its own terminal `active` over the pause. Moves the rule to the two points that can enforce it: an explicit `LOCKABLE_CONNECTOR_STATUSES` allowlist on the lock acquisition, and the same allowlist on `markSyncPending`, which closes the mirror race where a dispatch already in flight rewrites a just-paused row back to `pending`. Queueing and starting now agree on one rule, and a skipped hand-off is reported as its own outcome rather than a concurrency conflict. Also patches the connector detail cache alongside the list on an optimistic status write, so an already-expanded card starts its own sync poll instead of showing stale history behind the list's spinner. * fix(kb): make a queued sync prove it is the run that was queued `markSyncPending` minted an ownership token but only `releaseFailedDispatch` checked it, so the worker could consume a queue entry that was not its own. A task delayed past its lease is reclaimed and replaced; the status check alone let that stale task take the replacement's entry and run superseded options — a plain sync where the user had just asked for a full resync — while the replacement was turned away as `sync_in_progress`. Carries the token in the task payload and matches it at lock acquisition, the same discipline `holdsSyncLockToken` already applies to the `syncing` phase, extended to the phase before it. A superseded run is now reported as such rather than as a concurrency conflict. The payload field is optional for the rollout window only: tasks already in the queue carry no token, and stranding them would be worse than letting them fall back to the status check for one deploy. * fix(kb): report a paused connector as paused, not superseded Pausing a queued connector releases its token, so testing ownership before status reported every pause-while-queued — the common case — as a superseded dispatch. The mismatch is the symptom there; the status is the reason. * fix(kb): stop a status update landing on a run that already started The update's guards ran against a row read moments earlier and the write carried no compare-and-set, so a worker taking the lock in between meant the write landed on a `syncing` row — overwriting the run's status and, because leaving `pending` also clears the lock columns, wiping the token its heartbeat and terminal write match on. That stranded a sync that had already begun. The write is now conditional on the status the request was authorized against, and a lost race is reported as a conflict rather than "not found". Also restores the in-flight guard on the pause control. The optimistic status flip relabels it Pause -> Resume immediately, so a second click could send `active` before the first pause settled and resume a connector the user meant to pause. Read from the mutation's own pending state rather than the local id set this PR removed — React Query already knows which row is in flight. |
||
|
Vikhyath MondretiandClaude Opus 5
|
71129cd112 |
feat(custom-blocks): let a publisher decide whether their block's runs reach consumer traces (#6950)
* feat(custom-blocks): let a publisher decide whether their block's runs reach consumer traces Joining a custom block's child run into its caller's trace shipped on by default, gated at read time by whether the person reading could already open the source workspace. That gate is doing the wrong job: a custom block's whole point is that consumers need no access to the source, so the check refuses exactly the readers the feature exists for, and it makes the answer depend on who is looking rather than on what the block's owner agreed to publish. The decision moves to the party whose data it is. `custom_block.trace_child_runs` is set by the publisher in Settings, applies org-wide, and is the entire policy — nothing downstream re-checks a caller. `getCustomBlockAuthority` already resolves per invocation and is the one lookup both the canvas handler and the Agent-tool runner pass through, so one column covers both surfaces and no consumer input can assert it. It defaults to FALSE. With the viewer check gone, an opted-in block publishes the source workflow's block names, inputs, outputs, and prompts to anyone who can read a consuming workflow's log. That is the same boundary curated outputs and redacted errors hold, so it opens by an affirmative act of the publisher or not at all — never as the residue of a column default on rows nobody revisited. Closed means the handle is withheld outright rather than persisted behind a flag: with no `childExecutionId` there is nothing for a reader, a migration, or a later refactor to join. What replaces it is a `_childTraceDisabled` marker, because a boundary span with no children renders exactly like a leaf block and an untraced run would otherwise read as one that did nothing. The consumer-facing failure `ref` is untouched either way — it is the only thing that makes an untraced failure reportable. Custom blocks invoked as Agent tools now join too. The child's handle already reached the agent's persisted `toolCalls[].result` (`postProcessToolOutput` strips only `__`-prefixed keys); nothing lifted it onto the tool span. Both span builders lift and strip it, and `hydrateChildTraces` needs no change — its boundary walk already recurses. The same handle is stripped from the model-facing copy of the tool result in `executeProviderTool`, the single point where the raw and model copies diverge: an opaque execution id in a tool result reads to a model like data the tool returned. The live SSE stream keeps one condition beyond the policy: an identified consumer. Not an authorization check — no workspace query — but chat deployments and the public API leave `liveTraceViewerUserId` unset because their consumer may be anonymous, and opting into org-wide tracing is not consent to stream a publisher's raw agent tokens to the internet. Copilot deliberately cannot set the field; exposing a team's internals org-wide is a human decision, not one an agent makes while publishing on their behalf. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(custom-blocks): read the publisher's trace policy at read time, not from the handle's presence Treating a persisted `childExecutionId` as proof of publisher consent is only true for handles this PR's writer produced. Every handle written before it meant something else — "a child ran; authorize the reader" — and the rows carrying them outlive the migration, so removing the reader check turned them into an open door: a consumer could open an old parent log and receive the source workflow's block names, inputs, outputs, and prompts from a block whose publisher never opted in. `hydrateChildTraces` now resolves the policy live, per boundary, from `custom_block.trace_child_runs`. The child log row's `workflowId` is the key — publish enforces one block per workflow — which also covers an Agent-tool boundary, whose span carries no block type to look up. A workflow with no block row (never published, or since deleted) has no publisher left to consent and stays shut, as does a failed policy read. This is not redundant with the write-time withholding. The handler still emits no handle for a block that was closed when the run executed, so such a run stays closed forever even if the block is opened later; this check decides whether the runs that DO carry a handle may still be shown. Turning the policy off therefore also closes what is already recorded, which is what a governance switch has to do to mean anything. Reported by Greptile on #6950. Also drops `any` from the trace-policy tests: outputs read through `Record<string, unknown>` (the handler's declared return does not name these internal keys) and failures narrow through `ChildWorkflowError.isChildWorkflowError`, which pins the failure type as well as its fields. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(logs): sum the child-trace drop counters from the struct, not a hand-listed set `totalDropped` re-listed four of the five counters, so a read whose only drops were policy refusals computed zero and skipped the log entirely. That is the commonest drop there is now — every handle written before the publisher policy existed refuses at that gate — so the one signal telling an operator the live check is closing joins went silent exactly when it started mattering. Summed from the struct instead. A hand-maintained list beside a struct is stale the moment a field is added, which is precisely how `policyClosed` was left out. Reported by Cursor Bugbot on #6950. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(db): renumber the custom-block trace migration around a 0299 collision Staging landed its own 0299 (`table_run_dispatches.heartbeat_at`) while this branch was open. The two migrations are independent — different tables, no shared statement — so only the number and drizzle's snapshot chain collided. Regenerated rather than hand-merged: a drizzle snapshot is a full-schema dump whose `prevId` links it to its parent, so editing one by hand to sit after a migration it was not generated against is how the chain silently stops matching the database. Staging's 0299 and its snapshot are taken verbatim; this is 0300, generated against them, and its SQL is byte-identical to what it replaced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Siddharth Ganesan
|
58aa6379e0 |
feat(cli): add chat command (#6937)
* feat(cli): add chat command * fix(cli): harden chat command execution |
||
|
Waleed
|
dbbe99e473 |
fix(integrations): validation pass over Crunchbase, PitchBook, and CB Insights (#6925)
* fix(crunchbase): widen tier-gated collection allowlists and cap the deleted feed The deleted-entity, autocomplete, and fields-metadata allowlists each held only the collections the narrowest package tier publishes, so requests valid on a richer package were rejected locally before any request went out. An Advanced Financials key could not read the funding-round deletion feed at all. - deleted-entity collections: 9 -> the 14-collection union across all tiers - autocomplete and fields-metadata: 14 -> all 43 collections - clamp the deleted feed to its documented max of 25, not Search's 1000 - offer "All collections" so the cross-collection feed stays reachable - name the richer-tier card additions instead of presenting the base set as exhaustive Also rewrites a test that asserted the broken behavior and tightens a substring URL assertion that passed on the value it was meant to reject. * fix(pitchbook): stop a rejected API key reaching block output and logs PitchBook's 401 body echoes the submitted key back inside `message`. No PitchBook tool declared an `errorExtractor`, so the failure fell through to the generic chain, whose first entry returns `data.message` verbatim — putting the credential in the block error, the run log, and any agent context reading the failure. The existing scrubber sat in `transformResponse`, which never runs on a non-ok response. - add a `pitchbook-errors` extractor that replaces the unauthorized message with a fixed string, and wire it through all 91 tools - the extractor returns undefined unless the body carries a `message`, so a foreign 401 on the shared fallback chain is never labelled a PitchBook failure - correct `investor_preferences.preferredIndustry` to the shape the API returns - make `company_industries.emergingSpaces` opaque; its item shape is undocumented - reject a non-list of article ids instead of throwing a bare TypeError * fix(cbinsights): reject malformed input instead of silently rescoping a billed query CB Insights is metered, so a filter that fails to parse must fail the request — dropping it does not narrow the result, it charges for a query the caller never asked for. - reject an unrecognized boolean rather than dropping it, which had been widening a VC-backed firmographics search - reject a non-numeric limit instead of falling back to the endpoint default - reject non-text filter entries instead of stringifying them to "[object Object]" - accept only asc/desc for sort direction; a typo had returned the bottom of a metered result set as though it were the top - treat a whitespace-only numeric bound as unset, not as zero - drop `totalHits`/`totalHitsRelation` from list business relationships; that endpoint reports no total, so both were permanently null - trim `nextPageToken`, matching the id fields Also moves the token cache onto `lru-cache` per the in-process caching rule, replacing hand-rolled TTL arithmetic and a manual prune. * chore(harmonic): drop the team-key help text from the credential descriptor * chore(tools): regenerate tool metadata for the validation fixes * fix(tools): redact the retained error body, not just the message Scrubbing the extracted message left the raw provider body reachable: `createTransformedErrorFromErrorInfo` attaches `errorInfo.data` to the thrown error and the executor surfaces it on the failed tool's `output.data`, so a PitchBook key rejected with an echoing 401 still reached block output and agent tool results via `output.data.message`. - add an optional `redactData` to the error-extractor contract, so an extractor that exists because a provider echoes a credential can replace the body too - retain `redactErrorData(errorInfo, extractorId)` in place of the raw body - PitchBook replaces only the unauthorized body; every other failure is untouched - cover the executor path itself, since asserting on the redactor directly still passes when nothing is wired to it |
||
|
Theodore Li
|
5b28da1989 |
fix(tables): accept plain row query predicates (#6916)
* fix(tables): accept plain row query predicates * fix(cli): show table predicate group syntax |
||
|
|
42f6287911 |
feat(byok): add organization-wide key inheritance (#6834)
* feat(byok): add organization key management * feat(byok): inherit organization keys at runtime * feat(byok): add organization scope to BYOK settings * fix(byok): refresh org key state after mutations * fix(byok): hide stale inherited status badges * chore(db): drop colliding byok migration ahead of staging merge Staging independently claimed 0293. Remove ours so the merge is clean; it is regenerated at the next free index right after. * chore(db): regenerate byok migration at 0296 Staging claimed 0293-0295 during the merge; the regenerated SQL is byte-identical to the dropped 0293. * docs(byok): document organization scope, precedence, and the full provider list The BYOK section described workspace-scoped keys only. Add the organization scope, its Enterprise requirement, the per-provider precedence rule, what an entitlement lapse does, and the Pi sandbox exposure. Refresh the provider table from the settings page, which had drifted from 14 to 34 entries. * feat(byok): open organization keys to every organization plan Organization BYOK was gated on Enterprise, but an organization is the only thing that can hold the keys, so every plan that can own an organization should qualify — Pro for Teams, Max for Teams, and Enterprise. Add checkOrgPlan/resolveOrganizationPlan beside the Enterprise pair rather than widening checkEnterprisePlan, so the Enterprise-only gates (Access Control, whitelabeling) are untouched, and restore resolveOrganizationEnterprisePlan to module-private now that BYOK no longer needs it. * perf(byok): cache the organization entitlement, not the key material getBYOKKey runs once per agent block and once per hosted-capable tool call, so a loop over N items resolved N times — and each organization-inheriting resolution paid three sequential billing queries on top of the two key reads. Split the two reads by staleness tolerance. Key rows stay fresh, because revocation must be immediate. The entitlement is a billing gate that tolerates bounded staleness in the harmless direction (a lapsed organization keeps using its own key for <=60s), so cache it per organization with an in-flight share so concurrent blocks issue one query set. The management surfaces keep reading it fresh, so an organization that just upgraded is never told otherwise. Also run the block check and subscription read in parallel inside resolveOrganizationPlan, and carry the resolved scope on BYOKKeyResult so a log line can say whether a run used the workspace's key or an inherited one. * feat(byok): let workspaces store the Z.ai and Cohere keys the runtime reads Both ids were already in the BYOK contract enum and both are resolved at execution time — getApiKeyWithBYOK reaches 'zai' (GLM models are in the hosted catalog, so the BYOK branch runs), and 'cohere' backs both the Embeddings block and Knowledge Base reranking — but neither appeared in the settings list, so there was no way to store the key either path looks for. Cohere had no icon; add one from the official multi-color mark so it stays legible on a light and a dark page. Cohere's embed-v4.0 is kbEligible:false, so the description says 'Embeddings and Knowledge Base reranking' rather than claiming KB embeddings. * improvement(byok): shorten the workspace scope chip to 'Workspace' It sits beside 'Organization', so the scope reads from the pair; 'This' only added width. * fix(byok): do not cache a billing outage as an unentitled organization resolveOrganizationPlan maps a failed billing read to false, which is indistinguishable from a real plan lapse. The entitlement cache stored that, so one transient outage held the gate shut for the full TTL and every inheriting run silently fell back to a metered hosted key — and the cache's rejection path, which exists to prevent exactly this, was unreachable. Give the resolver the onError option its neighbours already have and let the cached read ask for 'throw', so a failure stays out of the cache and the next resolution retries. Behavior for the call that saw the error is unchanged: getBYOKKey still fails closed. Reported by Cursor Bugbot. * fix(byok): propagate the subscription read's failure too The previous commit threaded onError through resolveOrganizationPlan's own catch, but getOrganizationSubscriptionUsable soft-fails to null on its own, so a failed subscription read still arrived as an ordinary 'no usable subscription' and returned a successful false — which the entitlement cache then stored for the full TTL. Thread the option into that call as well. Test it at the billing layer rather than the cache layer: the entitlement test mocks resolveOrganizationPlan wholesale, so it could never have caught this. Verified the new test fails against the previous commit. Reported by Cursor Bugbot. * refactor(byok): cache the entitlement with LRUCache, like copilot entitlements The hand-rolled version reinvented three things the codebase already has a canonical answer for. lru-cache is a declared dependency of apps/sim and lib/copilot/entitlements.ts already caches an entitlement with it — by storing the in-flight Promise, which is what makes concurrent callers collapse onto one resolution with no in-flight bookkeeping at all. TTL and the size bound come from the library. That removes the second Map, the manual eviction (and its interaction with an in-flight entry), and the dead value-while-refreshing state: 23 executable lines. The one thing the library does not cover is dropping a rejected promise so a billing outage is not cached for the TTL, which is kept and pinned by a test that fails without it. TTL expiry is no longer re-tested — that is the library's behavior, not ours, and lru-cache reads its clock at module load so faking timers never moved it. * refactor(byok): coalesce the entitlement read with the shared singleflight lib/concurrency/singleflight.ts is the codebase's coalescing primitive and oauth/credential-service.ts already pairs it with a read-through cache. Adopting that shape fixes a case caching the promise directly did not: a *hung* billing read wedged every caller for the full 60s TTL, where coalesceLocally evicts and rejects at its settle deadline. It also removes the hand-rolled rejection eviction — the cache is written only on the success path, so an outage leaves no entry by construction. The cache now holds booleans, which introduces the one trap worth a test: a truthiness check would read a cached false as a miss and re-query billing on every resolution for lapsed organizations. Pinned. * fix(byok): keep an abandoned entitlement producer from writing the cache coalesceLocally does not cancel a producer it timed out — its docstring says so explicitly — so writing the cache from inside the producer let a late billing result overwrite a fresher answer a retry had already cached, and hold it for a full TTL. Move the write onto the value the caller actually received. A caller that timed out throws before reaching it, so an abandoned producer now resolves into nothing. The test reproduces the overwrite and fails against the previous shape. Reported by Cursor Bugbot. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> |
||
|
|
ea70f8dcf1 |
feat(harmonic): add contact workflow integration (#6902)
* feat(harmonic): add contact workflow integration * fix(harmonic): sync docs manifest * fix(harmonic): address integration review findings * feat(harmonic): add the missing people endpoints and fix two error paths Extends the integration from 4 to 13 tools, covering every non-deprecated people-scoped Harmonic endpoint, and repairs two defects found by validating the existing tools against Harmonic's OpenAPI and API reference. New tools: - Enrich Person (POST /persons) — the only path from a LinkedIn URL or email a workflow already holds to a Harmonic contact. - Get Person, Get Company Employees — account-based sourcing; employees returns URNs that chain into Batch Get People. - Saved-search net-new results and their acknowledgement, so a monitor stops reprocessing the entire result set on every poll. - Bulk email enrichment: submit, poll, and quota, plus Get Enrichment Status. Fixes: - The error extractor dropped Harmonic's string and object `detail` envelopes. A tool that names an extractor gets no fallback chain, so every FastAPI abort surfaced as "Request failed with status 403". The enrichment 404 also carries the scheduled `enrichment_urn`, which was being discarded — that URN is the only handle on the job, so it is now kept in the message. - The saved-search selector failed the whole dropdown instead of degrading: the response cap was half the sibling value on an endpoint that is unpaginated and returns every saved search with its full query object, and the option ceiling threw rather than truncating. Raised to 1MB and switched to truncate-and-warn, matching the other data-driven selectors. Clearing net-new results now requires an explicit scope. Harmonic treats an absent `entity_urns` as "clear everything", so an empty field would have silently discarded the backlog. Scope deliberately excludes company-side, deal, typeahead, network, and Scout streaming endpoints, and every endpoint retiring on 2026-11-05. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Waleed
|
865f8173ab |
feat(affinity): add Affinity CRM integration (#6908)
* feat(affinity): add Affinity CRM integration Adds the Affinity v2 API as a block with 70 tools, covering 86 of the 87 documented endpoints. Only Send Feedback is omitted — it reports product feedback to Affinity rather than doing workflow work. Endpoint families that differ only by an entity segment are one tool with an entityType param, so companies/persons field, list, row, and relationship reads, the company/person merge endpoints, and entity notes each collapse into a single operation. * chore(affinity): regenerate the docs manifest for the new integration page |
||
|
Theodore Li
|
a99f61bee9 |
feat(api): add v2 resource management endpoints (#6900)
* feat(api): add v2 resource management endpoints * fix(cli): gate destructive v2 commands * fix(test): update v2 request-slice count * feat(cli): add shared workspace profiles |
||
|
Waleed
|
a27f376164 |
feat(connectors): add Bitbucket, Databricks, Google Chat, and Workday Help KB connectors (#6895)
* feat(connectors): add Bitbucket, Databricks, Google Chat, and Workday Help KB connectors
Adds four knowledge base connectors, closing the gap where Sim shipped tool
blocks for these services but could not index their content.
- Bitbucket: repository source files and pull request descriptions over the
existing Bitbucket OAuth credential
- Databricks: notebooks (Workspace API) and saved SQL queries, PAT auth
- Google Chat: spaces indexed as message transcripts, new google-chat OAuth
service under the shared Google client
- Workday Help: knowledge article versions via the public helpArticle/v1 API
* fix(connectors): second-pass validation fixes and test coverage
Adversarial re-validation of all four connectors plus a combined-change
regression audit.
- bitbucket: stop declaring incremental sync (deletion reconciliation is
disabled for incremental runs, so deleted files were never removed on the
default code configuration); drop a wasted listing round-trip after the
frontier drains; add 33 tests
- databricks: reject an explicit maxDocuments of 0, which meant unlimited;
add 34 tests
- google-chat: correct the sender displayName documentation (user auth
populates only name and type) and emit second-precision RFC-3339 in the
message filter; 15 -> 24 tests
- workday: fix a crash when maxVersions is persisted as a number; refuse a
configuration whose status filter Workday did not honor; cap the
unresolved-name error; 18 -> 27 tests
- document the Google Chat service-account omission
- docs: list all four connectors and correct the connector count
* fix(connectors): index Google Chat spaces with no messages in the window
Review round 1.
- orderBy takes a full ordering expression, not a bare direction. The reference
documents the default as `createTime ASC`, so send `createTime DESC`; a bare
`DESC` either 400s every hydration or is ignored, which would make the cap keep
the oldest traffic and the later reverse render the transcript backwards.
- getDocument no longer returns null when the message window is empty. A space
with no messages is still a live space, and null is the "document is gone"
signal the engine treats as last-known-good: returning it dropped spaces whose
only prose is their description or guidelines, and left a stale transcript
indexed after a space was cleared or lookbackDays was tightened past every
message.
- The transcript header is omitted when no message contributed text.
* fix(connectors): only flag a Bitbucket listing capped when the cap withheld something
Review round 2.
takeIndexableWithinCap reports capReached as soon as the running total equals
maxItems, which is also true of a listing that ended at exactly that count.
Setting listingCapped there suppressed deletion reconciliation for a complete
listing, so upstream-deleted files and pull requests could stay in the knowledge
base indefinitely. applyMaxItemsCap now takes whether Bitbucket had more content
beyond the page -- a next link, or directories still queued on the frontier --
and flags the listing only when the cap actually withheld something, matching the
Databricks, Google Chat, and Workday connectors.
* fix(connectors): keep the Bitbucket cap flag set when it skips the pull request phase
Review round 3. Fixes a regression from
|
||
|
|
d9cfd7c68e |
improvement(mothership): v0.9 (#6815)
* checkpoint
* Checkpoint
* dot fixes
* Make async tool resume delivery recoverable
* Support split table tools and option recovery
* Harden VFS mutation handling
* feat(platform): platform subagent support — docs corpus VFS, search_docs, account context
Squash of the feat/platform-agent branch (sim side): mounts the Sim docs
corpus in the copilot VFS, wires search_docs and retires the legacy docs
search tools, and syncs the generated tool catalog and trace contracts for
the platform subagent.
* Align Copilot tools and resource handling
* Expand workflow log query support
* checkpoint
* Port desktop-improvements-0 desktop and browser-agent work
* fix(desktop): keep browser-agent input alive through live-SPA re-renders
* Harden workflow sanitization and Slack setup
* feat(desktop): coordinate clicks, caret insertion, and drag for the browser agent
* Revert subagent group eager auto-collapse
* fix(chat): keep sends FIFO across the streaming-to-idle drain gap
* Add the steering backend surface for mid-turn sends
* Sync generated contracts for async subagent orchestration
Pulls the mothership tool catalog (wait_agents / tail_agent / steer_agent /
interrupt_agent), trace spans (chat.async_subagent.*, chat.orchestrate.*), and
trace attributes (copilot.async_subagent.*) into the generated TS contracts.
* Add display titles for the async subagent orchestration tools
wait_agents / tail_agent / steer_agent / interrupt_agent get natural-language
running titles (naming the agent id being waited on, tailed, steered, or
stopped) and a Steering→Steered completed-verb rewrite.
* Show orchestrator-chosen subagent names on agent groups
A subagent_start whose payload data carries a name (the orchestrator's new
name trigger parameter) now labels the agent group with that mission name —
the agent-type icon stays. The name flows through the live stream path, the
turn model (AgentNode.displayName) and its serialize/rebuild round-trip, and
persisted transcripts (PersistedContentBlock.name), so reloads keep the label.
* Improve Copilot error handling and logging
* Backfill the subagent display name from the second start event
The dispatch-time subagent_start fires before the trigger args (and therefore
the name parameter) have streamed; the phase-3 start re-announces the lane with
the name. The block builder was dropping that duplicate wholesale, losing the
name on streaming providers — now it backfills subagentName onto the existing
block instead. (The home turn-model path already reconciled this case.)
* Support Slack bot connection flow
* Harden Copilot error and VFS handling
* Harden VFS resource operations
* Show 'Waiting for the first of N agents' for mode-any waits
The wait_agents title ignored the mode argument, so an any-mode wait over
three agents read 'Waiting for 3 agents' while the model narrated waiting for
the first — contradicting the transcript.
* Collapsed-by-default agent cards with live intent status lines
Subagents now narrate their work through <intent>3-5 words</intent> tags (a
fleet-wide prompt protocol on the mothership side). The turn model streams
each subagent's text through a split-safe tag parser: complete tags update the
agent's currentIntent and disappear from the prose, tags split across deltas
are carried until their close arrives, and a tag that never closes flushes
back as plain text.
The agent card renders as one line — display name (or agent label) plus the
latest intent, replaced inline as the agent shifts gears — and never
auto-expands; expanding to the full tool log is a deliberate click. Only an
outstanding permission prompt or a browser hand-back forces a group open.
Intents persist on the subagent block (and through the legacy persisted-
message paths) so reloads keep the last status, and a renamed reinvocation
now takes the latest name instead of pinning the first.
* Add the internal in-band tool execution route for live mothership turns
POST /api/copilot/tools/execute (INTERNAL_API_SECRET, Go→Sim) runs one
sim-server tool through the same server tool router the resume driver uses
and returns the result synchronously — no checkpoint. This is what lets
background (async) subagents write files/tables/knowledge, and lets the main
lane keep streaming (instead of checkpoint-pausing and killing every
background run) while async agents are live.
* Persist resource side effects for in-band tool execution
Files/tables created through the internal execute route now register on the
chat's resources exactly like the resume driver's executions — the route runs
the same handleResourceSideEffects pass (persistence only; an out-of-band
route has no live event sink, so mid-turn chip pushes are a follow-up).
* Extract intents from group text on every path, sync and async
The turn-model intent filter only fires for span-scoped subagent lanes, but
this surface also delivers subagent text through the legacy block path — so
<intent> tags flowed through unparsed and rendered as prose rows. Groups now
extract intents from their accumulated text at append time: the last complete
tag becomes the card's status line and every complete tag is stripped from
the rendered prose. Covers span-scoped, legacy, and persisted-reload paths
for both synchronous and background delegations.
* Fall back to the live tool title for the agent card status line
Persisted data proved tool-first subagents (grok search agents) emit zero
prose, so intent tags never stream no matter what the prompt says. The
collapsed card now always narrates: the agent's own <intent> tag when present,
else the latest tool's display title while the lane is live.
* Catch subagent <intent> tags in the server relay
The relay's subagent text handler now runs the split-safe intent extraction
as chunks stream: the latest complete tag is stamped onto the lane's persisted
subagent block (subagentIntent) and stripped from the stored prose, so live,
persisted, and replayed views all agree. Per-lane carry handles tags split
across chunks; a never-closing tag flushes back as plain text.
* Drop the tool-title fallback: the status line is the agent's intent
With the intent protocol now injected into every spawn's task message, agents
open with an <intent> tag; the card shows that narration or nothing.
* Replace intents with live tool-title status lines on agent cards
Intent parsing is fully removed (turn model, relay handler, persistence
fields, group extraction). The collapsed card's status is the latest tool
call in its RUNNING phrasing — never the completed rewrite, which stays in
the expanded log. Parallel tools show the most recently started still-running
title with a +N for concurrent siblings; between rounds the last title stays
frozen; a closed lane shows the bare name. Nested agent cards compute their
own status recursively from their own items.
* Keep the main Sim lane live-expanded; collapse only real subagent cards
The mothership group is the turn's own narration, not a delegation card —
collapsing it hid main-lane text and tools until manual expand, which read as
mis-ordered streaming while async subagents interleaved. It keeps the
original live-expand behavior and no status suffix.
* Persist subagent lane lifecycle blocks from the span handler
Lane-scoped span events route to the span handler, which only recorded trace
side effects — no subagent start block was ever persisted (verified: a
seven-agent run stored 104 blocks with zero starts). Grouping then fell back
to keying lane content by agent NAME, so a respawned agent of the same type
merged invisibly into the first one's card until it resolved. The handler now
persists the start block (spanId-keyed and deduped, carrying the display
name) and stamps endedAt on close, giving every invocation its own card.
* Name agents in orchestration titles; '+ n more' overflow format
wait/tail/steer/interrupt titles humanize the slugified agent ids back to
their display names ('Waiting for the first of Digest Workflow Build + 4
more'), and the agent card's parallel-tool suffix uses the same '+ n more'
format.
* Harden in-band tool execution and resources
* Route in-band execution through the comprehensive tool dispatcher
The internal execute route used the bare server-tool router, which rejects
VFS tools with 'Unknown server tool: read/glob/grep' — so nearly every
background agent's first discovery call failed (102 in-band calls in one run,
dozens rejected). It now uses the relay's executeTool dispatcher: registered
handlers (VFS, function execute) with permission checks and param
normalization, falling back to the app tool router — the same surface
foreground execution gets.
* Harden chat stream transition handling
* Harden VFS provenance and resource writes
* Standardize tool environment references
* Harden browser panel and chat cleanup
* Descriptive, user-language tool titles across the board
House rules applied everywhere: use every argument the call carries, never
name internal machinery, and never lead with Getting (the Got rewrite is
deleted so it cannot return).
- Deployments name the workflow: Deploying {workflow} as API/chat app/MCP tool
- Workflow reads name the part: Reading {workflow} meta/state/deployment/notes;
generic reads always name the file (Reading {leaf}), never bare Reading file
- Block runs name block and workflow: Running {block} in {workflow}, Running
from {block} in {workflow}, Running {workflow} until {block}, and
Enabling/Disabling {block} in {workflow}
- The six split-table tools get per-operation verbs (Adding column {name},
Updating rows, Wiring automation, Creating view {name}) instead of a wall
of Querying table
- The manage quartet drops X-action system-speak for gerunds
- get_* internal names become user language (Checking run settings, Tracing
block inputs, Reading the deployed version); web_fetch says Fetching
- Scheduled-task titles removed entirely (feature deleted from the Go catalog)
- New verb rewrites: Fetched, Traced, Wired, Configured, Looked, Rotated
* Deploying {workflow} as chat, not as chat app
* Loader gerunds; mv names both ends; mkdir names the folder
search_integration_tools -> Finding the right integration;
load_integration_tool -> Loading {integration} tools; load_skill ->
Loading skill {name}; run_enrichment -> Looking up {subject}. mv prefers the
model's phrasing, else reads 'Moving {files} to {destination}'; mkdir reads
'Creating folder {name}' from the path.
* Overflow counts read '+ n', dropping 'more'
* Unify workspace find and search
* Scale desktop title bar with page zoom
* Serialize account and organization truth into the copilot VFS
Workspace standing, membership, billing, org role, access-control
restrictions, published-block provenance, and fork topology were reachable
only through three parameterless tools (or not at all). They are ambient
read-only facts, so they belong in the VFS where they are greppable, cost no
tool round-trip, and every agent that can read gets them — the same move that
retired get_blocks_and_tools and list_user_workflows.
Adds account/{workspace,workspaces,members,billing}.json (always mounted) and
organization/{organization,access-control,custom-blocks,forks}.json (only when
the workspace is org-hosted). Every file projects an existing use case or util
after getOrMaterializeVFS's access assert — no new queries, no new
authorization. One relation per file, cross-referenced by id-and-name stub, so
overlapping facts cannot disagree. Volatile content (billing, access control,
forks) is lazy, so numbers are read-time fresh and unasked-for reads cost
nothing.
Projection follows the viewer: member emails are admin-only, fork detail
requires workspace admin on a forking-enabled org, and the whole organization/
namespace is absent for a personal workspace — which is itself the answer.
Retires get_account_billing, get_enterprise_context, and list_user_workspaces
along with their handlers; display titles stay for transcript replay.
* Fix insert_text refusing an editable field focused inside a frame
describeFocusedEditable descended shadow roots but not frames, while
activeElementReadback descends both. Focus inside a same-origin frame therefore
surfaced to the first as the FRAME element — not an input, not contentEditable,
not a canvas, no textbox role — so it fell through to 'not-editable' and
insert_text refused a field that press_key had just typed a character into.
Two functions answering 'what is focused' with different answers is the bug;
the descent loops now match exactly.
The refusal also names what actually held focus (tag, role, contenteditable).
A bare 'not-editable' gave the agent nothing to act on, so it guessed at the
cause — a real run spent twenty rounds on the wrong theory and had to be
stopped by the user.
* Keep retired browser takeover renderable in history
The tool is gone from the catalog, so its generated constant went with it and
every path that referenced it stopped compiling. Deleting those paths instead
would have silently downgraded every past transcript containing a takeover card
to a generic tool row, and dropped the no-timeout budget that an in-flight
takeover still needs while a rolling deploy finishes.
retired-tools.ts gives the literal a documented home that says what it is and
why it survives its tool.
* Follow the agent into a tab it opened to work in
browser_open_tab created the page with activate: false, so the agent worked in
a tab the user could not see while the panel sat on a page where nothing was
happening. The panel now follows a tab the agent deliberately opened.
Scoped to that tool only. A page spawning its own tab (popup, target=_blank) is
the site grabbing the view rather than the agent choosing a workspace, and
stays in the background as before — two existing tests pin that and caught the
first version of this change, which moved both.
A tab the user claimed still wins over both: the work starts in the background
instead of pulling the page out from under them mid-read.
* Make the browser tools agree with each other
An audit of the module found the frame-descent bug was one instance of a
pattern: six independent definitions of 'is this editable' and seven of 'what
is focused', disagreeing with each other. A tool refusing what its sibling
accepts on identical page state is invisible at runtime — the agent follows a
snapshot that says one thing into a tool that says another.
- browser_type now accepts role="textbox" like browser_insert_text does. The
snapshot advertises those elements as [textbox] with a ref, so refusing them
meant rejecting exactly what the outline told the model to type into. Both
the native and synthetic paths, and their descendant scans.
- pressKeyOnPage descends shadow roots and frames like every other focus
reader. It was dispatching synthetic keys at the shadow host or <iframe>
element, where they bubble but never reach the editor, while reporting
success — and contradicting the activeElement reported beside it.
- not-editable and ambiguous-editable name what was found: the element's tag
and role, and the candidate fields. Both had the data and discarded it, which
is what turns one blocked step into twenty rounds of guessing.
- obstructedAfterNavigation requires a dialog that ARRIVED with the
navigation. It compared against nothing, so every SPA route change under a
persistent role=dialog reported a successful click as obstructed. The test
that covered this asserted the false positive; it now pins both directions.
- browser_insert_text observes the top document when typing inside a frame,
like every other input tool. A submit that navigates the top page was
invisible to its frame-scoped observation.
* Let hover actually see what it mounted
Four independent defects made browser_hover blind to the most common thing a
hover produces — a row's action bar — so it reported no effect on a hover that
worked, and the agent fell back to clicking pixels off screenshots.
- The popup scan matched only role=tooltip/menu/listbox. Slack's message
shortcuts bar is a labelled toolbar/group, so it registered as nothing at
all. Added toolbar, menubar, labelled group, and [popover].
- The baseline was captured BEFORE prepareElementSurface scrolled the target
into view, so scrollChanged was always set by the tool's own probe. That
pinned every unproductive hover to 'background DOM churn' instead of the
honest 'nothing happened', and hid scrolling the hover really caused.
Re-baselined once the scroll settles and before the pointer moves.
- The MutationObserver attached only on the first observation, while the roots
list is rebuilt every call and grows as shadow roots mount. Components that
appeared later were never observed, so their DOM changes raised no revision.
Roots are now observed as they show up.
- observationTruncated was computed and never read, so a scan capped at 12k
nodes reported 'nothing appeared' with the same confidence as a complete
one — and portalled overlays live at the end of <body>, exactly what the cap
drops. Hover now says the page was too large to scan and to confirm visually.
* Stop the browser agent acting on the wrong element, and say why it refused
Four findings from the module audit, the first of which could silently do the
wrong thing rather than merely fail.
- A ref whose node is gone is re-adopted by structural resemblance, matching on
ORIGIN only so a pushState between snapshot and act does not kill every ref.
That leniency also let a ref to a row control in one view rebind to the
identical control in a view the app had since navigated to — acting on the
wrong message, signalled by nothing louder than recovered: true. Adoption now
requires the same path; a view swap reports the ref stale, and the caller
re-snapshots. Revalidating a still-connected node stays lenient, because that
is literally the node the model chose.
- A hit INSIDE the requested element is its own nested control, not an overlay.
Both produced 'covered by X — close or move the overlay', advice that cannot
be followed because there is nothing to close. Nested hits now say so and
point at retargeting.
- browser_click_at, browser_insert_text, and browser_drag listed targetChanged
in their effect formulas, but none passes an elementId, so no targetState is
ever captured and the term was always false — coverage that read as real.
Removed, with a test pinning the dependency.
- The seven effect formulas are deliberately NOT collapsed into one predicate:
drag must trust domChanged where others must not, hover must ignore field and
focus changes, click counts focus only for editables. Forcing one would make
each tool wrong differently. The differences are now documented in one place
next to the shared computation, so divergence is a declared policy rather
than an accident.
* Let edit_workflow configure block retries
* Updates
* Always focus the resource the agent is working on, and its browser tab
The resource panel had a carve-out: an already-open browser session declined
to replace another selection and only got an attention marker, so agent
browser work happened off-screen. The panel now follows the agent to whatever
it touches — browser included — and an event can still opt out explicitly.
The browser panel also follows the agent BETWEEN tabs: the store already
tracked automationTabId (and the strip marked it), but the visible tab never
changed. It now switches when the agent's target tab changes, so watching the
agent never means hunting for the tab it moved to. Keyed on the target
changing rather than on it being set, so a user who browses elsewhere
mid-run is only pulled along when the agent itself moves.
* Never paint a browser snapshot at stale geometry (the modal-open flash)
Opening a modal locks scroll, which removes the window scrollbar and reflows
the panel — so a capture taken before the lock describes a rect the panel no
longer occupies. The handshake painted that frame anyway and only then
retried, so the replacement landed visibly offset from the page it stands in
for: the flash. A capture is now checked against the host's live rect before
it is painted; a mismatched frame is skipped and re-captured at the settled
layout instead (modal retries go 2 -> 3 to absorb the extra settle).
* Name the workflow in deployment and workflow-scoped tool titles
'Checked deployment status' never said which workflow — nor did the deployed-
state read, run settings, block outputs/inputs, redeploy, promote, or the
global-variable write. These tools carry a workflowId (often defaulting to the
current workflow), so only the client can resolve a name: the enrichment layer
now resolves it for the whole workflow-scoped family and passes it as
workflowName, which every workflow title already reads.
Titles: Checking {workflow} deployment status, Reading deployed {workflow},
Checking {workflow} run settings, Reading {workflow} block outputs, Tracing
{workflow} block inputs, Redeploying {workflow}, Promoting {workflow} version
{n} to live, and 'Adding workflow variable {name} in {workflow}' — each
falling back to its unnamed form when no workflow resolves.
* Name the block that ran; never fall back to a raw block id
run_block and set_block_enabled carry only a blockId, so their titles would
have printed an opaque UUID ('Running 7f3a2b91-… in Invoice Sync'). The
enrichment layer now resolves blockId against the workflow store the same way
it already did for run_from_block's startBlockId, and the base titles no
longer accept an id as a name — an unresolved block reads 'Running block'
rather than a UUID.
* Add the missing Removing -> Removed rewrite
The table work introduced 'Removing automation'/'Removing enrichment' with no
past form, so those rows kept their present tense after completing.
* Name the target resource in the remaining tool titles
Table tools keep their operands nested under args and identify the table by
id, so their rows said 'Adding rows' with no hint where: enrichment now lifts
the nested args and resolves tableId against the cached workspace table list,
giving 'Adding rows to Runtimes', 'Adding column status in Runtimes',
'Reading views of Runtimes'.
Also: a block-schema read names the block instead of the file ('Loading
Slack', 'Loading Google Sheets tips'); browser type/insert show the text they
send, middle-ellipsized; downloads name the file; library-docs searches name
the library and query; knowledge-base searches include the query; generated
media names its output file; and diff_workflows, list_deployment_versions,
and publish_custom_block joined the workflow-name enrichment set.
* Bubble nested agents' tool calls into the parent's status line
The collapsed status only scanned a group's OWN tool items and skipped nested
agent groups, so a parent that had delegated froze on its last own tool while
its child did the actual work — the line described nothing that was running.
Status now walks the whole subtree: any tool at any depth counts, the most
recently started running one is shown, and the rest become the same '+ n'
overflow. With nothing running it falls back to the last tool at any depth,
so an idle parent still reflects where its subtree got to.
* Align nested tool call status rows
* Revert branch-local KB connector error-message edits
Restores apps/sim/connectors/ to staging state. Two copilot-focused commits
on this branch (
|
||
|
Vikhyath MondretiandClaude Opus 5
|
97c1688c49 |
feat(modal): add Modal Labs integration (#6896)
* feat(modal): add Modal Labs integration Modal has no public REST control plane — the Python/JS/Go SDKs all speak gRPC — so this covers the two surfaces that are reachable over HTTP: deployed Web Functions/Servers, and the OpenAI-compatible Endpoints API. Three operations: call a deployed function with proxy-token auth, generate a chat completion on an Endpoint, and list the models a token can reach. Auth sends the token pair as Modal-Key/Modal-Secret rather than the combined bearer form, so a Web Function that validates its own bearer token keeps the Authorization header free. Both URL fields require https since Modal terminates TLS everywhere, and a cleartext URL would leak the token. Chat completion declares request.modelInput so the system prompt and user message project to canonical placeholders before egress. Call Function deliberately does not — a Web Function runs arbitrary user code, and nothing proves its body reaches a model. /v1/models fields beyond `id` are inferred from OpenAI compatibility rather than printed in Modal's docs, so they are marked optional. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(modal): type the wire payloads and default chat to the shared endpoint Chat Completion required an endpoint URL and passed a blank one straight into modalOpenAiUrl, which throws — while List Models already fell back to the shared inference host and the generate-on-modal-endpoint skill tells agents to leave the field empty for Shared Endpoints. Skill-driven chat calls against the shared host failed instead of using that default. Chat now falls back the same way and the block field is no longer required. Replaces every `any` in the Modal tools with declared wire types for the OpenAI-compatible /v1 payloads. Fields stay optional because the shape comes from whichever inference engine backs the endpoint, so the readers keep their defensive `??` guards — the types exist so a future change to that mapping fails the compiler instead of shipping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Waleed
|
f5728fa887 | fix(icons): restore the Crunchbase mark's counter and framing (#6887) | ||
|
Theodore Li
|
4214a891f4 |
fix(setup): publish unscoped setup package (#6886)
* fix(setup): publish unscoped setup package * fix(setup): strip renamed status command |
||
|
Waleed
|
f6a9f0dd87 |
fix(integrations): white CB Insights tile and a borderless Crunchbase mark (#6884)
CB Insights moves from a dark navy tile to white, matching Jira, Confluence, and Bitbucket. Its icon carries its own fills, so it stays legible on the lighter tile. The Crunchbase icon drops the white rounded-square plate and its border, leaving just the `cb` mark on `currentColor` so the block's bgColor supplies the tile. The viewBox is retargeted to the glyph's true curve extrema with padding that keeps it at the same optical weight as the surrounding brand marks. |
||
|
Justin Blumencranz
|
0b71717241 |
perf(icons): reduce and guard SVG path precision (#6839)
* perf(react): reduce SVG path precision * fix(react): preserve Sim wordmark precision * fix(icons): preserve Quartr scale * test(icons): ratchet SVG path precision * fix(icons): make precision exceptions local * perf(icons): enforce three-decimal paths |
||
|
|
e3a4874ece |
feat(integrations): add Bitbucket Cloud (#6860)
* feat(integrations): add Bitbucket Cloud * fix(bitbucket): enforce selector workspace slugs * fix(bitbucket): overfetch small pipeline log tails * fix(bitbucket): harden provider edge cases * fix(bitbucket): accept provider diff redirect specs * fix(bitbucket): stop advanced-field leakage and harden log, status, and selector paths Splits the `closeSourceBranch` advanced subBlock into per-operation ids. Advanced fields serialize without evaluating their condition, so a value set on Create Pull Request reached Merge Pull Request and closed the source branch unprompted. Also: - read step logs through the byte-capped server transport and map an empty-log 416 to an empty result, keeping a genuine 416 an error - trim a step log's partial leading line after the character cap rather than before, and never return an empty log when the retained window held content - surface Bitbucket's `error.detail` alongside `error.message` - treat commit-status `key`/`state` as nullable so one malformed row cannot drop a page - match repository `full_name` case-insensitively and reject dot segments in a workspace slug before the outbound request - type `reviewerAccountIds` as the comma-separated string it is - trim optional Bitbucket query strings; correct the token lifetime to two hours --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Waleed
|
9f346765fe |
feat(granola): complete API coverage, note triggers, and connector validation (#6880)
* feat(granola): complete API coverage, note triggers, and validation fixes
Granola's public API exposes nine endpoints; Sim implemented three. Adds the
remaining six and wires the new programmatic webhook-endpoint lifecycle into a
managed trigger.
Tools (6 new, 9 total):
- get_transcript, list_audit_events
- create/list/update/delete_webhook_endpoint
Triggers: note.generated, note.edited, note.access_granted, plus an all-events
trigger. The provider handler registers the Granola endpoint on deploy and
deletes it on undeploy, scoped to the trigger's own event names, and verifies
every delivery with the Standard Webhooks HMAC-SHA256 signature Granola returns
on creation. event_id is the idempotency key, which Granola reuses across
retries.
Validation fixes to the shipped tools:
- get_note dropped speaker.attribution ("me"/"them"); now surfaced
- a 413 on get_note now explains that the transcript is too large inline and
points at get_transcript, instead of surfacing a bare status code
- note IDs are URL-encoded rather than interpolated raw
- base URL, auth headers, and status-aware error handling are shared runtime
helpers; params/outputs stay literal per file so the docs generator still
reads them
Tests cover signature verification (including replay and body-tamper
rejection), event matching, subscription create/delete, and the block/tool
contract — plus a guard that ids shared between the tool and trigger surfaces
seed the same default, since block state is keyed by id and last-wins.
The knowledge-base connector was validated against the spec and needed no
changes.
* fix(granola): correct array output schemas, listing-truncation signal, and docs
Findings from validation passes over the tools, trigger, and connector.
Tools — array outputs were declared as `type: 'json'` with `properties`, which
describes an object, not an array. Agents and the output picker therefore saw
`notes.title` instead of `notes[i].title`. All 15 array outputs (including the
pre-existing three tools) now use `type: 'array'` with `items`, matching the
2000+ other tool files. The audit event `data` field stays `json`; it is
genuinely free-form per the spec.
Connector — `hasMore` was ANDed with the cursor, so a `hasMore: true` response
with no cursor was reported as a complete listing. The sync engine treats
exactly that shape as truncated and sets `listingTruncated` to block deletion
reconciliation; masking it meant a partial first page could be taken for the
whole corpus and reconciliation would hard-delete every note past it. Granola
would have to violate its own contract to emit that shape, but the engine
already handles it and the connector was hiding the signal. Also aligns
mimeType with the `.txt`/text-plain bytes the engine actually writes (it was
the only connector of 101 claiming text/markdown).
Trigger — the setup instructions named a Granola settings path that does not
exist; the help center says Settings > Connectors > API keys in the desktop app.
Both list parsers now split commas inside array entries, so an array-wrapped
free-text value cannot be sent as one malformed identifier.
Block — `id`, `events`, and `hasMore` are produced by several operations but
their descriptions named only one, unlike `folders` which already documented
both meanings.
Adds connector tests pinning all four listingCapped quadrants and the
truncation signal, and tool tests for the list parser and the PATCH body's
per-field "omit means unchanged" semantics.
* fix(granola): clean up webhook endpoints created by a failed registration
Raised independently by both reviewers. The registration service only rolls
external state back when createSubscription *returns* — its rollback is guarded
on `preparedProviderConfig`, so a handler that throws is assumed to have left
nothing behind. Granola's handler broke that contract: when Granola accepted the
POST but the success body was missing `id` or `signing_secret` (including a body
that failed to parse and became `{}`), it threw with the endpoint already live.
Nothing then recorded an external id, so undeploy could not remove it, and
Granola kept delivering to a callback whose signature could never be verified —
duplicating on every deploy retry.
The handler now removes what it created before rethrowing, matching the pattern
grain's multi-hook create already uses. It deletes by id when Granola returned
one, and otherwise recovers the endpoint by matching the callback URL, which
also covers a connection that fails after the request reached Granola.
Endpoints whose URL was redacted to its origin are never matched — that
comparison could delete another workflow's endpoint on the same host. Cleanup is
best effort and never masks the original failure. A non-2xx is left alone, since
no endpoint was created.
Also folds the delete call shared with deleteSubscription into one helper.
* fix(granola): never recover an orphaned endpoint by callback URL
The previous commit's URL-based recovery was unsafe. A redeploy reuses the live
registration's `path`, so the candidate and the currently serving endpoint share
a callback URL — listing by that URL and deleting every match would remove the
live deployment's endpoint and silently stop a working trigger, which is worse
than the leak it was trying to prevent.
Cleanup is now keyed solely on the id Granola returned. When the success body
carries no id there is no way to tell the candidate's endpoint from the live
one, so it is left in place: a leaked endpoint produces unverifiable deliveries
that Granola disables on its own, whereas deleting the wrong one takes down live
traffic with no signal.
The 2xx-missing-signing-secret case this originally fixed still cleans up, since
that response does carry an id.
Adds a test asserting no lookup or delete is attempted when the response has no
id, so URL matching cannot be reintroduced unnoticed.
|
||
|
Waleed
|
f17938c09e |
feat(cbinsights): add CB Insights API v2 integration (#6879)
* feat(cbinsights): add CB Insights API v2 integration Covers every non-streaming v2 endpoint across 25 tools: free organization lookup, firmographics search, funding rounds and cap tables, investments, portfolio exits, business relationships, management and board, the Mosaic / Commercial Maturity / Exit Probability outlooks and their histories, funding windows, revenue, strategy maps, Scouting Reports, ChatCBI, and RAG context. CB Insights authorizes by client-credential exchange rather than a static key, so the tools run through directExecution: the shared executor trades the credentials for a bearer token, caches it briefly, and re-authorizes once on a 401 — the token lifetime is undocumented, so expiry is discovered rather than predicted. ChatCBI and RAG declare request.modelInput so an activated Sim secret in the message is projected to its canonical label before reaching a third party's model. directExecution still runs projectToolModelInputParams, so the two are compatible. The two streaming endpoints are deliberately excluded; they deliver incremental JSON chunks and their non-streaming counterparts return the same content in one piece. * fix(cbinsights): reject malformed ID lists and bound the token cache - Reject an organization ID list containing an invalid entry instead of dropping it. Silently filtering meant a typo ran the request against a narrower set — spending credits on the wrong organizations, or quietly widening a filtered search — and still reported success. - Apply the same rule to the optional firmographics ID filters, where a dropped filter broadens the search rather than narrowing it. - Bound the process-wide token cache so a long-lived worker serving many CB Insights accounts does not grow with the cumulative number of accounts seen. Expired entries are swept on write, then the oldest evicted. * fix(cbinsights): stop paging and blank input bypassing the search guards - Measure the firmographics empty-search guard against the filters alone. limit, nextPageToken, and sort were in the same object, so a request carrying only paging slipped past it and issued an unfiltered search over the whole database — which still spends credits. - Reject a mistyped numeric bound instead of dropping it. A bad headcount, funding, or valuation filter silently widened the search, the same failure mode already fixed for ID lists. - Treat an empty comma segment identically on the required and optional paths. A trailing or doubled comma is a separator artifact that cannot change which records are requested, so both paths now discard it; every other malformed entry is still rejected. * fix(cbinsights): accept only plain decimal organization IDs Number reads "0x10" as 16 and "1e2" as 100, so either notation resolved to a real but unintended organization and the request spent credits on it. Both the path-scoped and the bulk validators now require a plain run of digits, and use Number.isSafeInteger so an ID past the precision limit cannot round to a neighbouring one. * fix(cbinsights): bound a numeric organization ID to the safe-integer range The string path already required a safe integer; the numeric path still used Number.isInteger, which accepts a value past the precision limit. JSON parsing has already rounded such a value, so the request would target a different organization than the caller supplied. |
||
|
Waleed
|
a9cf760c0f | feat(pitchbook): add PitchBook integration (#6876) | ||
|
Waleed
|
40aa8ad5eb |
feat(crunchbase): add Crunchbase Data API integration (#6875)
* feat(crunchbase): add Crunchbase Data API integration Covers the v4 Data API end to end: dedicated search and lookup operations for organizations, people, funding rounds, and acquisitions, plus generic collection-parameterized search and lookup reaching the remaining 39 collections, single-card paging, autocomplete, the deleted-entity feed, and fields metadata. Adds a crunchbase-errors extractor: the API answers failures with a bare JSON array, which no existing extractor reads, so an auth or predicate failure would have reported only its HTTP status. * fix(crunchbase): honor card paging limits and cursor exclusivity - Cap a card page at the documented 100-item maximum instead of Search's 1000, which the shared Limit field made easy to carry over - Always request the card's identifier so a narrowed cardFieldIds cannot return a full page with a null cursor and stall a paging loop - Reject the mutually-exclusive afterId/beforeId pair on the card and deleted-entity endpoints, not just on search - Report an unexpected card shape as empty rather than wrapping the envelope as a one-row page |
||
|
Theodore Li
|
1372977d07 |
feat(setup): publish standalone self-hosting package (#6849)
* feat(setup): publish standalone self-hosting package * fix(setup): refresh discovered compose installs * improvement(setup): unify repository command * fix(setup): harden standalone package launch * Update README.md * fix(setup): isolate standalone compose installs * fix(setup): restore default stopped installs |
||
|
|
5d6268db91 |
fix(branding): refresh Google branding (#6786)
* fix(branding): refresh Google logo * refactor(branding): trim Google icon tests and correct the SVG wrapper Drop the GoogleIcon and SocialLoginButtons snapshot tests: they pinned exact attribute strings, the asset byte length, and the absence of markup the component never contained, so they broke on any legitimate tweak without catching real regressions. Correct the wrapper's viewBox to 0 0 200 204 so it matches the artwork, which bleeds to all four edges. The previous 204-wide box pinned four units of dead space to the right via xMinYMin, offsetting the mark within its box. Rewrite the TSDoc: it described avoiding a WebKit foreignObject gradient bug, but this file never used foreignObject and already ships 106 linearGradient definitions. Document the real reason instead - Google publishes the current G only as a raster. Align the auth button icon on shrink-0 with its sibling callsite. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Waleed
|
4f9d5f33b0 |
improvement(search): search every folder, and document real API error bodies (#6861)
* improvement(search): search every folder, and document real API error bodies Search on Files, Tables, and Knowledge was ANDed with the open folder, so a query only ever matched that folder's direct children — and the query was not cleared when you entered a folder, filtering the folder you just opened down to the same matches. A non-empty query now searches the whole workspace, a Location column names each result's folder, and opening a folder ends the search. Also gives GET /api/v2/files a `recursive` flag, and replaces the single shared OpenAPI error example — which showed `BAD_REQUEST` under every status tab — with one real body per status. * fix(search): discard the search term on clear instead of masking it `useSearchFilterValue` returned the debounced term whenever the input was non-empty, so clearing only hid the settled needle. The mask lifted on the next keystroke while the debounce still held the pre-clear term — opening a folder and typing within the window searched the whole workspace for the query the user had just abandoned. A clear now resets the settled term rather than hiding it, adjusted during render so the reset is visible to the render that follows the clear. The initial state is seeded from the first value so a deep-linked `?search=` still filters on the first render. |
||
|
Vikhyath MondretiandClaude Opus 5
|
521348b529 |
feat(secrets): record which secrets each run resolves, and surface it per secret (#6823)
* feat(secrets): record which secrets each run resolves and surface it per secret
Redaction stops a value at a boundary but cannot stop code that never emits it —
a Function block can print a key one character at a time and nothing ever matches
the secret. That is undecidable in general, so this adds the other half of the
posture: attribution.
Every run now records which configured secrets it actually resolved, under whose
identity, through which surface (workflow, Sim agent, MCP). The data already
existed in ResolvedSecretTraceRegistry.addActiveEntry and was persisted only for
paused runs; this persists it for every terminal path.
Execution logs cannot answer this. They store the whole available encrypted
environment rather than what a run referenced, they evidence a secret only where
value-matching redaction happened to fire, and they expire under
logRetentionHours — while "who has touched this key" outlives any single run.
- secret_usage: per-UTC-day rollup keyed by workspace, secret, scope, owner,
source, workflow, actor. A one-minute schedule touching three secrets would
otherwise write thousands of rows a day, which is also why this is not
audit_log. workflow_id/actor_user_id use '' sentinels rather than null so the
unique key works on Postgres 14 without NULLS NOT DISTINCT, and are not FKs:
they are historical facts, and an onDelete would rewrite a key column.
- secret_owner_user_id is part of the key. Two people can hold a personal secret
under one name and a shared personal secret resolves for a caller who does not
own it, so name and scope alone do not identify a secret. It is NOT the actor:
a scheduled run resolves the workflow owner's personal slice under the
workspace's execution actor.
- Direct environment reads are now detected in JS (TypeScript AST), Python
(tokenizer-checked) and shell (quote/heredoc-scanned), so a secret read as
environmentVariables['K'] or $K enters the run's provenance instead of going
unredacted. Each detector prescans for names that are actually configured
secrets before paying for a lex or quote-frame pass.
- Copilot integration tool calls are covered: resolveCopilotEnvReferences
substitutes {{SECRET}} into user-only params, which is a real use.
- See usage lives behind a credential-admin gate, using the same predicate that
reveals the value; members get a disabled chip explaining why.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(audit): register the secret-usage route in the validation baseline
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): keep rollup metadata with its run, and stop shadowed bindings faking usage
Review round 1.
- record.ts: last_execution_id/last_trigger were assigned unconditionally while
last_used_at was chosen by greatest(), so two runs completing out of order split
one row between them — the newer run's timestamp beside the older run's execution
id, making "View log" open a run the row does not describe. Both are now guarded
on the timestamp actually advancing, so the row's metadata always belongs to the
run that owns its timestamp.
- javascript.ts: a local binding named environmentVariables (declaration, parameter,
destructured binding, or bare reassignment) made reads off the user's own object
look like mounted-secret reads. Any such binding now disables detection for the
file; the AST already had parent pointers, so this is a kind check during the
existing walk.
- python.ts: same class of bug with no parser available, so the rule is an allowlist
— every mention of the binding must be a literal subscript or .get(), otherwise
detection is off for the file. This also subsumes the cross-line attribute case
(other.\n environmentVariables['K']), which the previous space-and-tab look-behind
missed.
Under-reporting is the safe direction here: a trail that claims a use that never
happened is worse than one that misses a use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(db): format the generated migration snapshot
CI runs lint:check across every workspace; the drizzle-kit output in packages/db
had never been through biome, so the branch was green locally (where lint had
only been run inside apps/sim) and red on CI. Whitespace only — both files are
byte-for-byte identical once parsed, and drizzle-kit still reports no pending
schema diff against the reformatted snapshot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): detect every rebinding of the environment identifier, not just declarations
Review round 2. A bare `for (environmentVariables of rows)` has no declaration to
key off, so the previous check missed it and reads of the loop value were still
recorded as secret usage.
Rather than extend the hand-rolled node-kind list, this reuses the pair the same
file already applies to reject a placeholder in a write position:
isDeclarationIdentifier covers declarations, parameters, destructured bindings and
imports, and isWriteIdentifier covers every assignment operator, ++/--,
destructuring targets, and for-in / for-of initializers.
That also closes four forms neither the review nor the original check named:
logical (||=) and nullish (??=) assignment, and object and array destructuring
assignment. Six of the eight added cases fail against the previous check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): apply the rebinding rule to shell, and say when a run's log is gone
Review round 3, plus the docs that were left claiming the old behavior.
- shell.ts: a script that writes a configured name (API_KEY=local, export/local/
readonly, read, for, unset) expands its own value from that point on, not the
mounted secret, so recording it claimed a use that never happened. Every mention
of the name must now be a `$NAME` / `${NAME}` expansion, matching the allowlist
shape the Python detector already uses. Applied per name rather than per file:
JavaScript and Python shadow one object holding every secret, whereas rebinding
one shell variable says nothing about the rest.
- The usage trail deliberately outlives execution logs, so a row routinely names a
run whose log has been pruned. The read now left-joins workflow_execution_logs on
its unique execution_id and reports availability, and the panel renders the chip
disabled with the platform tooltip instead of linking into an empty Logs view.
Three states: no run to link, a run whose log is gone, and a live link.
- Docs said a direct environmentVariables/$KEY read does not activate masking,
which this branch changes. Corrected in credentials.mdx, function.mdx and the
logging FAQ, and the recognition limits are now written down: runtime-built
names, reassigned bindings, and reads that cannot be told apart from text.
Added a "See usage" section covering who can see it and why an empty trail
means "nothing recognized" rather than "never used".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): writing a name is not reading it, and a bare mention is not a rebinding
Review round 4.
- javascript.ts / python.ts: `environmentVariables.API_KEY = 'x'` and
`delete environmentVariables.API_KEY` touch the name without ever reading the
mounted value, but the detectors matched the member access and recorded a use
that never happened. JavaScript now asks the same isWriteIdentifier the
placeholder rewriter uses (its parameter is widened to ts.Node — the body
already walked generic nodes, so this is a type change, not a behaviour one)
plus a delete check; Python excludes a subscript followed by `=` and a `del`
target.
- shell.ts: requiring every mention of a name to be an expansion also fired on
text that binds nothing — a comment naming the key, or `echo "API_KEY=$API_KEY"`
where the literal is an argument rather than an assignment — and dropping those
cost masking on a genuine read. It now looks for actual writes: an assignment at
command-word position, a binding builtin, `printf -v`, or a `for` target.
The two directions are not symmetric, which is why this errs toward detecting
the read: missing a write records a use of a secret the script only had in its
environment, a misleading audit row and nothing more, since masking still
searches for the real value and will not find it. Over-detecting a write
suppresses masking on a value that does reach the log.
This also makes the code match what the docs already described — skipping after
a rebinding, not after any mention.
13 tests added; 11 fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): an update reads before it stores, and a del target may be parenthesized
Review round 5. The first of these is a regression from round 4.
- javascript.ts: reusing isWriteIdentifier to answer "is this a read" was wrong.
That predicate answers the rewriter's question — is this a target the
substitution must refuse — so it treats every assignment operator alike, which
is correct there and wrong here: `+=`, `||=`, `??=`, `++` and `--` all load the
current value before storing, so they are genuine reads and were silently
losing their masking. Only a plain `=` stores without reading. Replaced with a
purpose-named predicate, and isWriteIdentifier's parameter is narrowed back to
ts.Identifier now that nothing else needs it widened.
A test committed last round asserted the wrong behaviour for `+=`; it has been
corrected rather than left to pin the bug.
- python.ts: `del (environmentVariables['K'])` slipped past a check that looked
only at the characters immediately before the match. It now isolates the
enclosing logical line and tests whether that is a del statement, which also
covers `del((x))`, `del(x)`, `del a, x`, and a del after a semicolon.
12 tests added or corrected; 10 fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): stop excluding Python writes, which kept leaking in the unsafe direction
Review round 6. Greptile found that `del environmentVariables[environmentVariables['K']]`
had its inner access — which computes a key, so it is a genuine read — skipped
along with the delete, leaving that value unmasked.
The narrow fix was another textual rule. Instead this removes the write and delete
exclusions from the Python detector entirely, because they were optimizing the
wrong direction.
`resolvedSecretNames` feeds `outputSecretMatcher`, an exact-value matcher over the
output. Naming a secret the code never read costs nothing there: the matcher scans
for a value that does not appear. Failing to name one that was read leaves it
unmasked. The two error directions are therefore not comparable, and the
exclusions bought only audit-trail tidiness while every heuristic they needed has
so far leaked into the dangerous side — first a parenthesized target, now a nested
read. A `del` or an assignment is reported like any other access.
JavaScript keeps its exclusion: a real AST answers the question per node, with no
text to misread, and it has produced no such hole.
Net 30 lines removed from python.ts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): report recognized reads instead of proving they are not reads
Review round 7. Greptile flagged both directions at once — false usage from
reporting a write target, and unmasked secrets from the file-wide shadow flag —
so I traced what the signal actually drives before choosing.
The chain: the compiler's names feed outputSecretPlaintextsByName and the
exact-value matcher, NOT context.resolvedSecretNames, which starts empty. After
execution activateOutputSecretProvenance scans the output and adds only names
whose plaintext actually appeared; those become __resolvedSecretNames, which
tools/index.ts turns into recordResolved calls, which is what the usage trail
reads.
So a compile-time false positive produces no usage row on the ordinary path — it
only hands the matcher a value the code never emits. It does produce one on the
!projection.safe fallback, where the system already over-approximates by design.
A false negative, by contrast, keeps the value out of the matcher entirely, so a
genuinely read secret is never masked on any path.
That asymmetry decides it, so every "prove this is not a read" mechanism is gone:
- javascript.ts: the file-wide shadow flag. A helper declaring its own
environmentVariables discarded genuine reads of the mounted binding everywhere
else in the file — Greptile's security finding, and real.
- python.ts: the allowlist requiring every mention to be a subscript or .get().
Same hole: passing the dict to a function suppressed unrelated reads.
- shell.ts: the rebinding check. It had the same hole in a form nobody flagged —
`echo "$API_KEY"; API_KEY=local` dropped the first read, which is of the real
secret.
What stays is the question of whether the text is code at all — strings, comments,
single quotes, quoted heredocs — plus the receiver check that `other.environment
Variables['K']` is a different object, and JavaScript's node-precise write/delete
exclusion, which cannot suppress a read elsewhere.
Net 215 lines removed across the three detectors and their tests. Docs updated:
the rule is now stated as reporting rather than proving, and that See usage may
occasionally list a secret the code had available but did not read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(secrets): drop the last write-vs-read special case
`environmentVariables` is a plain object deserialized from the run payload
(route.ts:206), not a handle on the stored secret. Assigning to it changes
nothing outside the sandbox and is discarded when the run ends, so separating a
write from a read bought almost nothing while leaving JavaScript as the one
language still trying to prove a read is not a read.
Every language now follows the same rule: report a recognized read of a
configured secret name. The only exclusions left are facts rather than
inferences — the text is not executable (string, comment, single quote, quoted
heredoc), the receiver is a different object, or the name is not statically
knowable.
Docs note that assigning to the binding does not edit the secret.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(secrets): ship only the fields the trail actually shows
Five fields crossed the API and reached no reader: usageDate, firstUsedAt,
actorEmail, workflowId and actorUserId. The panel renders the timestamp, the
trigger, what used the secret, the actor's name, the run count and the run link;
everything else was projected, serialized and discarded.
first_used_at is dropped from the table as well. Nothing read it, and inside a
per-day bucket "first used that day" says nothing next to "last used that day" —
so it was a column written on every run for no question anyone asks. The upsert
loses its least() with it. Migration regenerated; the identifier columns behind
the joins stay, they simply are not returned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): report referenced code secrets, not only ones that surface in output
The Function route activated a secret's provenance — and therefore its usage row
and downstream masking — only when the exact value appeared in the result,
stdout, or error. That gate made the trail miss silent use entirely: a key that
authenticates an API call and is never echoed reported nothing, and so did the
founding scenario of this feature, a key exfiltrated character by character. The
innocent run that echoed a key got a row; the run worth catching did not.
Activation now follows the referenced set the compiler already computes: resolved
{{KEY}} bindings plus recognized direct reads, filtered to configured values —
the same set the unsafe-projection fallback already activated. An extra name only
hands the output matcher a value that never appears; configured-but-unreferenced
values are still never included. The output-scan activation path and its surface
helper are deleted rather than kept alongside.
One old test pinned the gate ("does not activate a referenced secret that does
not cross the Function result"); it now asserts the reverse, with the reasoning
attached. Two new tests pin the char-split exfiltration and the silent API-call
case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): shell escaping is backslash parity, not adjacency
Review round 8. `\\$API_KEY` is an escaped backslash followed by a LIVE expansion
— bash prints `\` plus the value — while `\$API_KEY` is an escaped dollar and
stays literal. Checking only the character adjacent to `$` read every even run as
escaped, dropping a real read from usage and masking alike; verified against
bash before fixing.
The scanner now counts the run of backslashes before the `$` and skips only odd
runs, the same parity rule logicalLineEndAfterContinuations in this file already
applies to line continuations. Six-case parity table added; the three even-run
cases fail against the previous check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): recognize destructured environment reads
Review round 9. `const { API_KEY } = environmentVariables` delivers the value by
name with no property- or element-access node in the AST, so the member-access
walk missed it entirely — and a missed read leaves an emitted value unmasked,
the dangerous direction.
The AST walk now also recognizes the declaration form (shorthand, renames,
defaults, string-literal keys), the assignment form ({ KEY } = env), and a
...rest element — which names no key but takes every value, so it reports every
configured name; the alternative left `const { ...all } = env; return all`
entirely unmasked. A computed key stays unrecognized, the same runtime-name
boundary as a computed subscript, and a receiver that is not the bare identifier
is not attributed.
Nine cases added; the six positive ones fail against the previous walk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): one receiver rule for destructured reads, parentheses included
Review round 10. Two accurate findings, folded into a generalization instead of
two more special cases:
- A parameter default (function f({ API_KEY } = environmentVariables)) and a
binding-element default are the same by-name delivery as a variable
declaration. The detector now keys on the ObjectBindingPattern itself and
checks its parent's initializer, so every declaration position follows one
rule instead of per-kind arms.
- Parentheses group without changing the receiver, so (environmentVariables) is
unwrapped before the identifier check — in the destructuring arm AND the
member-access arm, which had the same hole unreported.
Declined the for-of-over-array-literal finding: the receiver there is a
container, not the environment object, and following data flow through
containers has no fixed point — the same documented boundary as aliasing and
computed keys. A test pins the boundary so it reads as chosen, not missed.
Eight cases added; the seven receiver-rule cases fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): a dot in prose is not a qualifier, and a literal computed key is a subscript
Review round 11. Both findings were implementation-narrower-than-rule, fixed by
consulting authorities the detectors already had rather than adding new ones:
- python.ts: the receiver walk crosses whitespace so a parenthesized `other.` on
a previous line is seen — but it landed on a comment's final period
(`# Load the value.`) and discarded the genuine read on the next line. The
landing position is now checked against the same lexer ranges that filter the
candidates, which is also why the receiver check moves after lexing.
- javascript.ts: `const { ['API_KEY']: key } = environmentVariables` is the
element-access rule in pattern position, so a computed key holding a string
literal resolves like a literal subscript; any other computed key keeps the
runtime-name boundary a computed subscript already has.
Eight cases added; the comment-period case and all three literal-computed-key
cases fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Waleed
|
3ff91f0439 |
improvement(docs): clean up leftovers from the code-block alignment PR (#6825)
* improvement(docs): clear leftovers from the reverted revisions
A cleanup pass over the final state. Every finding was residue from an approach
this PR tried and abandoned, or a claim that stopped being true when it did.
- Delete the copy-button svg sizing rule: a later rule sets `display: none` on
that same element ungated, so sizing it was never observable. Superseded by
the mask approach.
- Drop the paragraph in page.tsx arguing about a custom Shiki factory. The
factory was deleted; nothing configures one now.
- Correct shiki-curl-json.ts, which still claimed the grammar "reaches the
client path too". It does not — that was the justification for choosing a
grammar over a transformer, so leaving it stated the opposite of the truth.
Now records where it applies, where it does not, and why not to retry.
- Correct the global.css section header, which claimed the component owns the
shell while the next rule defines it here.
- Qualify the `--copy-glyph` declarations with `:has(> svg[class*="lucide"])`,
which the group's own comment asserts of every rule in it.
- Correct `getCode`'s TSDoc: the gutter is a `::before`, and pseudo-element
content never reaches `textContent`, so line numbers were never what the
clone guards. It guards transformer-emitted `.nd-copy-ignore` nodes.
- Compose `chipGeometryClass` and emcn's `ChipChevronDown` in the API example
selector instead of restating their literals.
- Merge the duplicated `div[role="region"]` rule. The tablist pair stays split:
biome's `noDuplicateProperties` reads a nested `@variant` setting the same
property as a duplicate and fails the build — recorded so it is not remerged.
- Note that fumadocs ships its own gutter for `lines`-meta fences, which cannot
be suppressed from here and would paint a second column.
* fix(docs): drop a highlighter registration that can never fire
fumadocs-openapi calls `renderCodeBlock` with a hard-coded `"json"` from both of
its call sites (`request-tabs.js:76`, `response-tabs.js:48`), so the docs
`CodeBlock` it routes through never receives a shell language. The
`getHighlighter('js', { langs: [curlJsonBodyGrammar] })` registering the
shell-scoped JSON-body injection therefore did nothing but await on every API
sample render, and the docblock claiming the grammar covers those samples was
wrong.
- Delete the call and its imports.
- State the grammar's real coverage: prose fences only, via `langs`. Both API
reference paths are unreachable — samples are JSON, and the cURL usage tabs
highlight client-side off fumadocs' own factory.
- Correct `code-block.tsx`'s TSDoc, which still said API samples come from
fumadocs' own renderer. They come through this component; `UsageTab` is the
renderer that bypasses it.
- Re-home a comment orphaned when two CSS rules merged — it had drifted onto
the rule below and read as documenting it.
- Drop a `.nd-copy-ignore` claim about transformers emitting those nodes;
nothing here does, and upstream parity is the reason the clone exists.
|
||
|
Vikhyath MondretiandClaude Opus 5
|
3a03774e42 |
fix(forks): stop copying connector-managed knowledge base documents (#6818)
* fix(forks): stop copying connector-managed knowledge base documents A fork copies a KB's documents but never its connectors, so a connector-sourced document arrives with `connector_id` nulled and its `external_id` intact. The sync engine keys every existing/tombstone/ exclusion lookup off `connector_id`, so that copy is invisible to it - never updated, reconciled, or purged - and `doc_connector_external_id_idx` does not constrain it either, since its `connector_id` is NULL. Attaching a connector in the child then re-ingests every page as a NEW row on top of the snapshot. Each fork hop re-copies the previous hop's orphans and adds one more generation, so a prod -> UAT -> staging chain leaves three rows per page and a knowledge search returns the same page three times, one of them serving content frozen at the fork date. Exclude connector-managed documents from all four doors a document can enter a fork through: the whole-KB content copy, the in-transaction placeholder pre-creation, the sync-only copy into an already-mapped KB, and the content fill (guarded for payloads planned by a pre-change worker mid-rollout). The placeholder path matters as much as the copy loop - filtering only the content phase would leave a permanently archived row behind a persisted `knowledge_document` mapping. Skipped on both sides, the reference clears like any other uncopied document's. A document whose connector was deleted already has a null `connector_id` (the FK is ON DELETE SET NULL) and is static in the source too, so it still copies. One count(*) per copied KB logs what was left behind, since a fully connector-synced KB now forks to zero documents. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): keep the skipped-document count from failing a copied KB The connector-managed count feeds a log line, but it sat inside the KB's try block, so a transient failure on a COUNT(*) would roll back a copy that had otherwise succeeded and clear every reference to it. Move it into a helper that swallows its own error. Counting is not copying: only the copy itself may fail a resource. Test proven red by removing the catch - the mutation reports a knowledge-base failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): clean up full-KB placeholders planned before the exclusion The mapped-KB fill guarded a pre-change plan, but the full-KB path did not: a placeholder planned by an old worker for a connector-managed document is simply no longer returned by the page query, so nothing fills it and it stays archived behind a live mapping that a remapped document-selector still resolves to. Report those child ids as failed documents so the shared cleanup clears their references and drops the rows, and delete their persisted identity so a later sync does not resolve to a row cleanup removes. Keyed on the SOURCE being connector-managed, which can never become copyable, so it cannot race a concurrent attempt mid-fill the way a "source is gone" check could. The mapping drop is now one helper shared with the mapped-KB catch. Test proven red by removing the reconciliation block. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(forks): make the stale-plan probe best-effort The probe ran inside the KB try, so a transient SELECT would reach the catch, roll back a complete copy, delete the child base, and clear every reference to it. Weighing it as "load-bearing, so fail closed" was wrong: the probe runs on EVERY copied KB that has referenced documents, while the state it repairs exists only inside a rollout window. Failing closed traded a common-path outage against a rare-squared one. It now swallows its own failure with a loud error log, leaving that pre-existing state in place rather than destroying a good copy. Test proven red by removing the catch - the mutation reports the KB failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Waleed
|
1c69372cba |
feat(cli): follow a run, wait for one, and tail the log (#6813)
* feat(cli): follow a run, wait for one, and tail the log Three commands the surface was missing, each polling or streaming something the generated command layer cannot express. `workflows run --follow` renders the SSE the execute route already emits, so a multi-minute agent run stops printing nothing until it ends. It rides on the generated `run` leaf rather than a sibling command — same operation, one different response encoding — and delegates to the handler it replaced, so every non-follow invocation still runs the generated path. Answer text, thinking and tool calls go to stderr; only the final envelope reaches stdout, so redirecting still yields the result. Reasoning and tool frames need the `X-Sim-Stream-Protocol` header, which is sent only when asked for, because negotiating also switches answer text to live chunks the server may retract. `workflows runs wait` closes the loop `--async` opens. Terminal is completed, failed or cancelled; `redacting` is not, since a run whose output is still being scrubbed is not yet a run you can read. A time pause keeps polling because the server resumes it, and a human pause stops with the resume command rather than burning the bound and calling it a timeout. Distinct exit codes keep cancelled and paused from reading as failure. The bound is `--wait-timeout` and not `--timeout`, because SIM_TIMEOUT_SECONDS already bounds one request and two knobs of the same name hide each other. `logs follow` tails runs as they arrive. Dedup keys on run id, not on the timestamp: a schedule fan-out starts many runs in the same millisecond, so a timestamp watermark either drops the siblings or reprints them. JSON output is one object per line, because a follow never closes an array, and the table header is printed once so columns stay aligned across polls. * fix(cli): disclose a truncated burst, and clear a stale retry notice Two review findings in `logs follow`, both verified against the code first. The page budget bounds one poll so an enormous burst cannot stall the follow, but on reaching it the live cursor was discarded: the remainder is older than everything collected and the next poll restarts at the newest page, so those runs were never printed and nothing said so. The budget stays — draining without one trades a bounded poll for unbounded buffering in a process meant to run for hours — but hitting it now warns on stderr, naming the count and pointing at `sim logs list`. That notice is written even off a terminal, because a piped log is where an unexplained hole is hardest to spot. The retry notice was cleared after the empty-rows check, so a poll that recovered but found nothing left "retrying in Ns…" on screen while the follow was already healthy. Clearing now happens as soon as a poll succeeds. The second test needed two failures to be worth anything: the teardown clears the line either way, so what separates fixed from broken is whether a bare erase lands before the second notice or only at the end. The first version passed against the bug. * test(cli): pin that a mixed page is the watermark, not a truncation A page holding a run already printed proves the follow caught up, so the truncation warning must not fire there — that is how every healthy poll terminates, and warning would report a hole on the ordinary path. The straggler sharing that page is still collected, because the filter takes every unprinted row on it rather than only those above the known one. * fix(cli): say when the requested backlog was larger than a page holds The logs API clamps `limit` into 1–1000 rather than rejecting it, so `logs follow -n 5000` came back with 1000 rows, anchored the floor to that partial page, and said nothing. The seed already knew — it computes whether a live cursor remained — but the caller discarded the answer. Guarded on both halves. Fewer rows than asked for is only a shortfall when more were waiting: a workspace holding ten runs answers `-n 50` with ten and nothing is missing, so warning on the row count alone would fire on every small workspace. The cursor is what separates the two. |
||
|
Waleed
|
c17043a8b7 |
improvement(docs): align code blocks with the platform design system (#6810)
* improvement(docs): align code blocks with the platform design system
Docs code blocks rendered in stock `github-light`/`github-dark` on fumadocs
chrome, sharing no colors, typeface, metrics, or corner radius with the app.
- Add Sim Shiki themes transcribed from emcn's Prism token colors, shared by
the MDX pipeline and fumadocs-openapi (which highlights through its own
instance, so the API reference was left on the GitHub palette).
- Use the mono stack the app actually renders. `tailwind.config.ts` points
`font-mono` at `--font-martian-mono`, but nothing defines that variable, so
every code surface in the product resolves to the system stack.
- Give blocks the platform's field chrome — `rounded-lg`, a `--border-1`
hairline, a `--surface-5`/`--code-bg` fill — and the 13px/21px metrics of
`Code.Viewer`. The rule keys on `figure.shiki` because two renderers emit
these figures and that is the only join point they share.
- Number every line, from the same tokens as the in-app gutter. Padding sits
on `.line` rather than fumadocs' `--padding-left`: that property is
re-declared on the inner `pre` for API samples, which dropped the digits on
top of the code.
- Collapse tabbed fences into one box with the strip as the title row, and
align the inline-code chip with the app's markdown renderer.
- Reuse emcn's `Button`, `useCopyToClipboard`, and chip chrome constants
instead of re-deriving them, and drop ~90 lines of `!important` overrides,
including a rule that could never match.
* fix(docs): stop line numbers overlapping code, unify the copy glyph
The gutter opened its column by setting `padding-left` on `.line`, which never
applied: fumadocs' own rule is `.shiki:not(.not-fumadocs-codeblock *) .line`,
and `:not()` carries its argument's specificity, putting it at (0,3,0). Every
code block rendered its line number on top of the first characters.
- Drive fumadocs' `--padding-left` / `--padding-right` instead of overriding
`.line`. Declared on the figure, the viewport, and any inner `.shiki`,
because the variable is inherited and the nearest declaration wins — the
class sits on the figure alone for prose fences but on the figure and the
inner `pre` for API samples, and `--padding-right` is also written as an
inline style on the viewport.
- Route API request/response samples through the docs `CodeBlock` via
fumadocs-openapi's `renderCodeBlock`, so they carry the emcn copy control
rather than fumadocs' lucide clipboard.
- Mask the emcn glyph over the one block `renderCodeBlock` cannot reach — the
usage tabs hardcode `ClientCodeBlock` and `OperationClientOptions` exposes
only `APIExampleSelector` — so the copy icon is identical everywhere.
* improvement(docs): reserve the gutter column without numbering one-liners
A line number on a single-line shell command has nothing to reference, and the
CLI pages are mostly single-line commands. Dropping the gutter on those blocks
was the original behaviour, but it made adjacent fences start their code 28px
apart wherever a command sat next to its output.
Reserve the column on every block so all code shares a left edge, and paint the
digit only when the fence has more than one line.
* fix(docs): drop the gutter entirely on single-line fences
Reserving the column but leaving it blank gave one-line commands a 44px indent
with nothing in it, which reads as a rendering fault rather than as alignment.
Gate the column and the digit together, so a single-line fence keeps fumadocs'
default padding and a multi-line one gets both.
* fix(docs): stop the copy-button CSS restyling emcn's own Button
The rules added for fumadocs' copy button matched on `aria-label` alone, so
they also hit the emcn `Button` this app renders — re-declaring geometry,
radius, color, and a `background: none` that killed its hover, and pinning docs
to today's `buttonVariants` values with no failure signal if those change.
- Qualify every one with `:has(> svg[class*="lucide"])`, the same scoping the
mask rules already used, so they reach only the block fumadocs renders.
- Stroke the masked glyph at 1.25 to match `Button size='icon'`, which
overrides the icon's authored 1.55. The two copy glyphs were rendering at
different weights — the mismatch the mask exists to remove.
- Drop `.line::after { content: none }`: it cannot outrank fumadocs' (0,4,1)
rule, and no fence in `content/` uses the `lines` meta it guarded against.
- Drop the `!important` and the redundant viewport selector on `--padding-left`;
nothing declares it between the figure and the region, and nothing contests
it at equal specificity. `--padding-right` keeps both — its inline style is
real.
- Use emcn's `cn` where emcn class constants are merged, so they go through the
merger that knows the `text-micro|caption|small|md` scale.
- Correct the comments the review disproved: two claimed the API reference
still renders fumadocs' CodeBlock, which `renderCodeBlock` changed.
* fix(docs): keep the gutter padding override belt-and-braces
A simplify pass removed the `!important` and the viewport selector from
`--padding-left` as provably redundant, and on the numbers they are: fumadocs
declares the property at (0,2,0) while these selectors are (0,3,1) and (0,4,1),
and nothing declares it on the viewport.
Restore both anyway. Getting this wrong paints the line numbers on top of the
code — a regression this PR already shipped once — and the specificity of
`:has()` and `:not()` is easy to miscount in exactly that direction. The comment
now records both that the override is redundant on paper and why it stays.
* feat(docs): highlight the curl JSON request body as JSON
A `curl -d '{…}'` payload is one single-quoted string to a shell, so the same
JSON that renders with colored keys in a response sample rendered as one flat
block of string color in the request sample directly above it.
Fixed with a TextMate injection rather than the two approaches that don't work:
- `{ include: 'source.json' }` attaches the JSON grammar but its object pattern
only assigns `support.type.property-name.json` — the scope that colors keys —
when it owns the opening brace. Entering mid-string, keys stay string-colored,
which is the whole difference. So the key/value/array patterns are written out
and name that scope directly.
- A Shiki transformer tokenizes it correctly but is a function, and the request
tabs highlight in the browser off a `shikiOptions` object passed through RSC,
where functions cannot cross. A grammar is plain data and reaches both sides.
An injection has to be registered on the highlighter, not passed per call, so
the API page moves to `createAPIPage` from `fumadocs-openapi/ui/base` with our
own factory, and `ApiShikiProvider` hands that same factory to the client code
blocks — both public API. The MDX pipeline preloads it through `langs`.
The opening brace requires `}`, a quoted key, or end-of-line after it, which
keeps `awk '{print $1}'` out; the end-of-line case is needed because Oniguruma
matches line by line. Verified against `jq '.[0]'`, `awk '{print $1}'`,
`grep -o 'foo'` and `echo '{}'` — none are re-colored.
* fix(docs): paint the code fill on the viewport, not the figure
The request and response panels on an API reference page rendered on different
backgrounds. Sampled from screenshots: the request panel showed the page
background (#ffffff light, --bg dark) while the response panel showed the code
surface (--surface-5 / --code-bg).
The fill was left to show through from the figure or the tab group, and those
diverge per renderer. fumadocs gives a standalone figure `bg-fd-card` but an
in-tab figure `bg-fd-secondary`, and this app forces
`--color-fd-card: transparent` on API reference pages — so zeroing the in-tab
figure's fill, expecting its group to supply one, left the request panel
transparent while the response panel's `bg-fd-secondary` group kept ours.
Paint it on the scroll viewport instead. That is the innermost box all three
renderers wrap code in, so it cannot diverge, and it no longer matters what any
ancestor sets.
* fix(docs): hide the code tab strip's scrollbar
fumadocs makes the strip `overflow-x-auto`, and an endpoint with ten status
codes overflows it in the API reference's narrow rail — leaving a scrollbar
across the bottom of a 34px header, which reads as the header being clipped
rather than as something scrollable.
Hidden the way the platform hides it on a scrolling tab strip: emcn's `TabStrip`
carries `overflow-x-auto [scrollbar-width:none] [&::-webkit-scrollbar]:hidden`.
The code viewport below keeps its scrollbar. There the overflow is content, and
the platform's own `Code.Container` shows one for the same reason — hiding it
would hide that a line continues.
* fix(docs): keep fumadocs-openapi's server graph out of the client bundle
The Vercel deployment went red at the commit that added `ApiShikiProvider`, and
stayed red for three commits. That component is `'use client'` and imported
`ClientCodeBlockProvider` from `fumadocs-openapi/ui/base` — an entry that also
pulls `remark`, `remark-rehype`, `@fumari/json-schema-ts` and `github-slugger`.
Importing it from a client module forces that whole server graph into the
browser bundle. Measured in `.next/static/chunks`: `json-schema-ts` in 1 chunk,
`github-slugger` in 3, `remark-rehype` in 4. A local build tolerates the weight;
a deployment with size limits does not.
`ClientCodeBlockProvider` lives in a `"use client"` module that the package does
not expose through its `exports` map, so there is no client-safe path to it.
Back to `createAPIPage` from `fumadocs-openapi/ui`, dropping the custom factory
and the provider. After: `json-schema-ts` 0 chunks, `github-slugger` 1,
`remark-rehype` 1 — the remainder is fumadocs' own client-side markdown.
The server path keeps the injection by registering it on the shared highlighter
`highlight` already resolves. What is given up is the API reference's cURL usage
tabs, which highlight in the browser off fumadocs' own factory. Prose fences
keep it, and that is where the `curl -d '{…}'` examples live — getting-started,
authentication, workflows/deployment, passing-files, triggers/webhook, in every
locale.
|
||
|
Waleed
|
edc25aa976 |
docs(helm): document null as the way to remove an inherited env key (#6801)
* docs(helm): document null as the way to remove an inherited env key Setting `app.env.KEY: ""` cannot clear a key that `app.envDefaults` sets: the Secret template drops empty values, and the deployment template treats an empty override as "not overridden" and still inlines the default. Helm's own `KEY: null` deletion is the supported mechanism and already works. The empty-string behavior is load-bearing, not a bug — every key under `app.env` ships as a "" placeholder, and ten collide with a real `envDefaults` value (NEXT_PUBLIC_APP_URL, BETTER_AUTH_URL, ...), so "" has to read as "unspecified" or a default install would blank them out. - README: document `null`, with the --reuse-values and Argo CD valuesObject caveats; correct the claim that `app.env` always wins over `app.envDefaults` - values.yaml + self-hosting docs: same guidance where operators look - sim-helm skill: record why an unset list is the wrong shape here - tests: lock in that null removes a key and "" does not * docs(helm): correct the verify command's chart path and scope the required-secret claim - The verify snippet used a `sim/sim` repo alias that this chart never publishes; every other instruction installs from the local `./helm/sim` path, so the command could not run as written - Nulling a boot-critical key only fails at template time with the chart-managed Secret. `existingSecret` mode skips that validation entirely (the chart cannot read a pre-created Secret), and under ESO the key must instead be mapped in externalSecrets.remoteRefs.app * docs(helm): say null must be applied in every layer that sets a key `null` deletes a key from the map it is applied to, not from the pod. A key set in both `app.env` and `app.envDefaults` survives a null on the app.env entry alone — the deployment then inlines the envDefaults value again. Under ESO a retained `externalSecrets.remoteRefs.app` mapping keeps syncing the key regardless of app.env. - README and self-hosting docs: drop the "works in all three secret modes" shorthand and spell out that every layer setting the key must be nulled, including the ESO remote mapping - tests: cover both halves — nulling only app.env restores the envDefault, nulling both actually removes the key - chart 1.5.4; staging took 1.5.3 in the meantime |
||
|
Waleed
|
9dc828f36a |
fix(email): greet SMTP relays with a qualified hostname instead of [127.0.0.1] (#6799)
* fix(email): greet SMTP relays with a qualified hostname instead of [127.0.0.1] Nodemailer derives the EHLO greeting from os.hostname() and substitutes the address literal [127.0.0.1] whenever that name contains no dot. Kubernetes pod hostnames never contain one, so every k8s deployment introduced itself to the relay as loopback and strict relays refused the session before any mail moved. Send the domain the app is served from instead, as RFC 5321 4.1.4 asks, with SMTP_EHLO_NAME to override it for relays that expect a different identity. * fix(email): parse EHLO address literals and drop a port from the app domain Review round 1. The bracketed branch matched a character class rather than an address, so [::::] and [13] reached the relay as a greeting it would refuse. Parse the address with node:net instead, which also admits the RFC 5321 IPv6: form. getEmailDomain reports a URL host, so a deployment served on a non-default port failed the qualified-name check and fell back to nodemailer's default — [127.0.0.1] again on Kubernetes, the exact failure this change exists to fix. Strip the port before validating. * fix(email): accept any casing of the IPv6 literal tag, and stop owning SMTP_EHLO_NAME in setup Review round 2. RFC 5321 tags the IPv6 address-literal form, and RFC 5234 makes ABNF string literals case-insensitive, so [ipv6:2001:db8::1] is as valid as [IPv6:...]. The exact-prefix check routed it to isIPv4 and discarded it. Drop SMTP_EHLO_NAME from the email capability's optional fields. SMTP_SECURE, the same kind of optional transport knob on the same provider, is not modelled there either, and claiming the field obliged the setup wizard to prompt for it — a field whose entire purpose is to stay unset now that the default is right. |
||
|
Waleed
|
d17a11f29b |
feat(jotform): trigger a workflow on every new form submission (#6802)
* feat(jotform): trigger a workflow on every new form submission
Jotform's only webhook event is a new submission, so the block gets one
trigger. Deploying it registers the callback on the form through the API
and undeploying removes it again.
Two things about this provider needed handling:
Jotform posts submissions as multipart/form-data, which the shared webhook
body parser did not read — the delivery died as a 400 before any handler
saw it. The parser now flattens a multipart body the same way it already
flattens a urlencoded one, reducing an uploaded part to its filename so a
stray file cannot inflate the execution input.
The form's webhooks are identified by their position in the form's webhook
map, so an id captured at registration goes stale the moment any other
webhook on that form is removed. Nothing persists it; cleanup re-resolves
the id by matching the callback URL. Registration checks the same way,
because Jotform answers a rejected request with the unchanged list rather
than an error.
Answers are exposed as the parsed `rawRequest` rather than re-keyed by
question label — the labels are not unique, and the payload shape is only
documented as the raw q{qid}_{slug} map.
The trigger's region field is named `apiRegion` so it does not collide
with the block's own advanced-mode `region`.
* fix(jotform): make webhook registration idempotent and URL matching tolerant
Validated the trigger against Jotform's API reference and a captured
delivery (zulip's multipart fixture), which confirmed every mapped field —
formID, submissionID, formTitle, username, ip, type, pretty, rawRequest —
and turned up three things worth correcting.
Jotform keeps a form's webhooks as a plain list and does not treat the URL
as a key, so posting one it already holds leaves the form delivering every
submission twice. Registration now consults the list first and only posts
when the URL is absent. The documented POST sample returns the new entry as
"0", renumbering the rest, which is further reason nothing persists an id.
URL matching no longer lets a trailing slash decide the outcome. Jotform
stores the URL verbatim in every sample seen, but an exact match failing
would hard-fail deploy, and Pipedream's client normalizes the same way.
The rawRequest description claimed the field holds the submitted answers.
A real payload also carries slug, buildDate, submitSource and
jsExecutionTracker, and a file answer appears under the bare slugified
label as upload URLs rather than under a q{qid}_ key — which is also why
filtering to q-prefixed keys would silently drop file answers.
* fix(jotform): keep the callback when an active deployment still needs it
Redeploying prepares the replacement webhook row alongside the live one and
a workflow keeps its path across deployments, so both rows resolve to a
single callback on a single form. Registration adopts the callback already
present instead of posting a duplicate, which left the retired row's
cleanup deleting the one the new row had just adopted — the trigger went
silent after a redeploy that changed the trigger config.
Teardown now skips when another webhook row belonging to an active
deployment resolves to the same form and callback URL, matching how the
Telegram handler skips deleteWebhook while an active deployment still uses
the same bot. A genuine undeploy has no such row and still cleans up.
|
||
|
Waleed
|
0e84e92d39 |
fix(cli): bound, trace, and explain the requests the CLI makes (#6798)
* fix(cli): bound, trace, and explain the requests the CLI makes Four transport gaps, all of which failed silently. A request had no timeout, so a connection that was accepted and then never answered hung the terminal indefinitely. `SIM_TIMEOUT_SECONDS` now bounds one, defaulting to 3600s — deliberately above every timeout the server itself applies, since a synchronous workflow run is allowed 3000s on a paid plan and a tighter default would abort real work and report it as a transport failure. `0` removes the bound, for a self-hosted deployment that runs executions without one of its own. The caller's abort signal is composed with the timeout rather than replaced, so neither masks the other. Node ignores HTTP(S)_PROXY unless NODE_USE_ENV_PROXY opts in, and only from v22.21 and v24.5, so on a network that reaches the API only through a proxy every command failed to connect while the variable that would have fixed it was already set. The CLI cannot enable that from inside the process — Node reads it at startup — so it says what to do rather than bundling an HTTP stack for a setting the platform now owns. An API key was sent to any http:// endpoint with no signal. Now a warning, not a refusal: http is the documented way to reach a local dev server, and a deployment terminating TLS at a gateway is real. Loopback stays silent. `SIM_DEBUG=1` traces method, URL, status and duration. Bodies and headers are deliberately absent — the request carries the API key, and `secrets set` carries the secret itself. All four write to stderr, so a piped stdout stays parseable. * fix(cli): make the request bound safe on every runtime it supports Two ways the new timeout could fail before the request was made. `AbortSignal.any` arrived in Node 20.3 and this package supports Node 20, so composing a caller's abort signal with the timeout threw a bare TypeError on the earliest 20.x releases. It is now used when present and composed through an AbortController when not. `AbortSignal.timeout` rejects a fractional millisecond outright, and past 2^31-1 ms it does not fail at all — it clamps to 1ms, so the longest timeout anyone asked for became the shortest. The value is now rounded and refused above what Node can actually wait, pointing at 0 for an unbounded wait. Also unstubs env vars between tests: `stubEnv` is not undone by `unstubAllGlobals`, so a SIM_TIMEOUT_SECONDS set for one test configured every test after it. * fix(cli): correct the proxy version table, and classify a timeout mid-body `runtimeCanProxy` treated any release between 22 and 24 as capable, so on Node 23 — which reached end of life before the backport — a configured proxy was ignored and the CLI stayed silent about it, which is the exact failure the warning exists to report. The table is now the two lines that shipped the support, and anything after them. `AbortSignal.timeout` keeps firing after `fetch` resolves, so a bound that elapsed while the body was still being read — a large `files get` — escaped the client's own handling and printed a raw TimeoutError stack. The top-level handler now names it, which covers the streaming path as well as the JSON one. A user's own Ctrl-C raises AbortError and is deliberately left alone. * fix(cli): report a timed-out download as a timeout `files get --output-file` streams the body to disk, and `streamToFile` converted anything the stream threw into a write failure. So a request bound elapsing mid-download read as `Could not write <path>: ...`, sending the reader to check permissions and free space for a timeout they can raise, and hiding the one instruction that resolves it. The predicate and that instruction now live beside the timeout that raises them, so the client, the top-level handler and the download path all say the same thing. The wrapping stays where it is: the staged-download cleanup runs off that failure, and rethrowing past it would leak the temporary directory. * fix(cli): keep a sub-millisecond timeout bounded Zero is how this function says "no bound", so rounding a positive SIM_TIMEOUT_SECONDS down to zero inverted the request: anything under 0.0005s asked for the shortest possible timeout and got none at all, leaving a stalled request to hang. Introduced by the rounding that fixed the fractional-millisecond rejection. Floored at 1ms for every positive value; only a literal 0 still disables. |
||
|
Waleed
|
0b4d34137b |
feat(secrets): add optional descriptions to workspace secrets (#6796)
* feat(secrets): add optional descriptions to workspace secrets Workspace secrets already have a backing credential row with a description column, but nothing surfaced it. Teammates had no way to record what a secret is for. - Add a Description field to the secret detail page, matching the integrations credential page, gated on workspace-secret admin - Fold the value and description editors into one Save/Discard pair and one unsaved-changes guard; two guards cannot coexist, since each seeds its own same-URL history entry - Match descriptions in the secrets settings search - Expose description on GET/PUT /api/v2/secrets and in the CLI Descriptions are workspace-only: env_personal credential rows are per-workspace mirrors of one user-global secret, so one saved there would exist in a single workspace, and a personal secret has no teammates to inform. The API rejects a description on personal scope rather than silently dropping it, and omitting it on PUT leaves any existing description untouched so a value rotation cannot erase it. * fix(secrets): address review findings on secret descriptions - Patch the credential detail cache optimistically on update. `onMutate` cancelled the detail query but only patched the lists, so a detail-backed editor stayed dirty after a successful save until the refetch landed — long enough for Discard to restore the pre-save value over the committed one, and for Back to open the unsaved-changes guard. - Memoize `useSecretValue`'s returned callbacks and object, per the hook convention, so the composed form's save/discard stop churning per render. - Reject a description on a personal secret in the domain layer rather than only at the v2 boundary. The internal credential update path accepted one for any type, writing data every reader hides. - Normalize an empty description to null so the API and UI agree. - Correct the secrets documentation, which described a Display Name field the detail view does not have and omitted the scope rule. - Drop the CLI's copy of the 500-character bound; it can't import the contract, so a copy only drifts from the message the API already returns. - Collapse a redundant save guard and align the description write gate with the render gate. Leaves the integrations credential page byte-identical to staging. * fix(secrets): keep the API docs example and CLI column order stable Backward-compatibility fixes for anyone who never sets a description. - Move the blank-to-null normalization out of the contract and into the route. A Zod `.transform()` on any property drops the whole request schema's OpenAPI examples, which had silently removed the Set Secret request example from the published docs. - Append the CLI `description` column instead of inserting it before `updated`. `--output text` is positional, so inserting would shift every field an existing script cuts. - Reject a description on a personal secret with a message that says so, rather than dropping the field and falling through to the generic "no updatable fields" error. |
||
|
Waleed
|
38075ad977 |
fix(sap_concur): align the integration with SAP Concur's documented API (#6790)
* fix(sap_concur): align the integration with SAP Concur's documented API Validated all 70 tools, the block, and both proxy routes against SAP's published API docs. Auth: - add the password and companyUuid to the token cache key so a request with the wrong password can no longer be served a cached token minted from someone else's - wire the documented company-level flow (username = company UUID, credtype = authtoken) so companyUuid actually scopes a token - expand the datacenter allowlist to the documented set (adds glz, apj1, usg, the impl hosts, and the www- twins) and drop the undocumented cn host; validate the returned geolocation by shape instead of membership - coalesce concurrent token fetches so a fan-out mints one token - forward Retry-After so 429 retries pace off Concur's own hint - handle the errorMessageList, SCIM detail, and legacy Error.Message shapes instead of falling through to a generic HTTP message - pin redirects and cap the response body Block: - collapse six contextType subBlocks that disagreed on their default, so a new block no longer seeds MANAGER for every operation - clamp contextType to each operation's documented set - stop requiring a userId and contextType that the default operation's tool does not accept, and scope the receipt fields to the upload ops - reach six params that had no subBlock, and pass userId on travel request updates so a stale value cannot impersonate Tools: - correct response shapes that resolved to undefined: budget headers, budget categories, allocations, receipts, SCIM nextCursor, and the delete endpoints that return a bare boolean - use the Travel Request Amount schema (currency, not currencyCode) - narrow the four XML-only travel tools to a documented string payload and request application/xml - surface real errors instead of a JSON parse failure when the proxy returns a non-JSON body - cap receipt uploads at the documented sizes before downloading Adds 106 tests covering the token cache, geolocation validation, path traversal, and error extraction. * fix(sap_concur): drop the removed forwardId subblock via a migration Removing the `forwardId` subblock without a migration entry breaks deployed workflows that still carry a value under that key. It fed a `concur-forwardid` request header that is documented nowhere in Concur's Receipts v4 or Image v1 references, so it was never honored. There is no replacement subblock and the value is an opaque caller-chosen string rather than a secret, so it is dropped outright. * fix(sap_concur): stop swallowing upload response-read failures The upload route caught every error from the bounded response read and continued down the success path, so a size-limit breach or a stream failure surfaced as an upstream success with a null or header-only body. Concur returns Content-Length: 0 on a successful image-only upload, and readResponseTextWithLimit already returns an empty string for that without throwing, so dropping the catch keeps the legitimate empty-body case working while letting real read failures reach the route's handler. * fix(sap_concur): unblock company auth and correct the body wand prompt The password grant marked username required, so the company-level flow — which sends the company UUID as the token username and has no user login — could not be configured at all, even though the request schema and token fetch already accept companyUuid without a username. Username is now optional for that grant and the server-side check reports which of the two is missing. Relabels the password and companyUuid fields to say what they carry in the company flow. The shared body wand prompt also still described several payloads the way they looked before this branch: quick expenses in PascalCase rather than v4 camelCase, travel requests and expected expenses using currencyCode where the Request v4 Amount schema uses currency, the standard SCIM SearchRequest URN instead of Concur's, startIndex as a search parameter when it is unsupported, and a cash advance shape that does not match the documented request. A wand-generated body was therefore rejected for most of the create operations it covers. * fix(sap_concur): keep Concur's status when an error body fails to read Removing the blanket catch from the upload read fixed one failure mode and introduced its inverse: a cap breach or stream error while reading a non-success body threw before the route reached the branch that preserves Concur's status, so an upstream 4xx surfaced as a Sim 500 and could trigger a retry the caller should not make. Both routes now split the two cases. On a success status the body is the result, so a read failure still propagates. On an error status the body only supplies the message, so a read failure resolves empty and the upstream status survives, with the message falling back to the generic HTTP-status form. Adds 21 tests covering both helpers over success, error, empty-body and boundary statuses; inverting the status check turns 14 of them red. |
||
|
Waleed
|
60097c89b4 |
fix(cli): default to the host that serves the API (#6791)
`sim.ai` answers /api/** with a 301 to `www.sim.ai`, and the client refuses to follow redirects — a 301 rewrites a POST into a bodyless GET, so following one turns a write into a silent no-op and hands the API key to whatever host Location names. Defaulting to the apex therefore failed every command for anyone who never set an endpoint. Before the refusal shipped it was quieter and worse: reads succeeded while writes did nothing. Also trims the provider catalogue from eleven inferred columns to seven. `docsUrl`, `helpText`, `requiresClientGeneratedCredentialId` and the nested `fields` are what you read once you have chosen a provider, not what you scan to choose one, and they pushed the table well past a terminal. Both ids stay: `credentials connect` names an OAuth provider by `serviceId`, `credentials create` matches a service account on `providerId`. |
||
|
Waleed
|
c8f559ae77 |
fix(workflows,connectors): close pre-merge audit findings (#6783)
* fix(workflows,connectors): close pre-merge audit findings Recover subblock values orphaned by the id renames in this release, and stop truncated knowledge-base listings from reporting themselves complete. - Add operation-scoped subblock id migrations so a saved workflow's stored value survives a rename. Cloudflare create/update DNS record, ServiceNow read record, and Okta deactivate/delete previously lost their stored value: the create path substituted a seeded default (an A record where the user chose CNAME, and unproxied where they chose proxied), and the update path silently no-opped while reporting success. A migration is used rather than a legacy-id fallback so no subblock id carries two value spaces at runtime. - Webflow, Zendesk: a listing that stops for a reason the connector cannot rule out now reports as capped instead of exhausted. A malformed envelope, an unfollowable continuation link, or an absent collection list previously read as a complete listing and let deletion reconciliation hard-delete every document past the truncation point. - Sentry: pin the listing window in the request rather than inheriting the server default, so the range cannot silently narrow into hard deletes. - Fork sync: a parent re-pick no longer writes a blank over a hidden optional dependent's stored target value, and a required field stays on screen once it is filled. Add hook-level coverage for the submitted payload. - Fork file copy: a file whose name is already taken in a reused target folder is de-duplicated instead of dropped. - Delete an orphaned Shopify OAuth route that built a credential from unsigned cookies. It had no writer, no caller, and no inbound link. - Tailwind: drop two content globs that scanned 5.4k files to emit one unused rule, keeping the ones that fix brand tile icon color. - Correct the API route-count baseline, add an Evernote docs redirect, align library copy with the language rules, and fix a stale turbo filter. * fix(connectors,forking): trim the audit fixes to their minimum A legitimacy review found several changes closed no live defect, and two introduced problems of their own. - Zendesk: narrow the cursor fix to a signal change. Treating a missing meta envelope as truncation had also made the walk follow links.next and keep paginating, and the ticket cursor has no page-depth valve, so a source advertising a next page with no meta could loop without terminating. The page-fetch set now matches the previous behavior; only the flag is new. - Zendesk: drop the search next_page branch. The existing count check already caps every case where a missing key could lose documents. - Webflow: drop the empty-collections flag. The sync engine already blocks the first sync on an empty listing and reconciles only when a second sync agrees, which handles a transient fault better and still removes documents when a source is genuinely emptied. The flag short-circuited that and suppressed reconciliation permanently. Restore the previous loud failure on a non-array envelope, and drop the unreachable collection-id filter. - Webflow: soften a docstring that claimed pagination.total is always present. It is documented optional, so its absence proves nothing either way and treating it as unprovable truncation is the fail-safe reading. - Sentry: drop the pinned statsPeriod. Sentry's issue search floors every query at 90 days in the executor regardless of the request, and the endpoint this release moved away from hit the same floor, so there was no window to close. Keep the tests and the docstring recording that. - Fork copy: drop the renamed counter, which no caller reads. - Repair check-block-registry, which stopped exempting migrated subblock ids when the migration map became an array — `in` was testing array indices. - Drop mdx from a Tailwind content glob that emits nothing, and loosen an exact compiled-SQL assertion to the invariant it was pinning. * fix(migrations): keep a ServiceNow write body off the read projection Review findings from the first round. - A legacy ServiceNow block can hold a Create/Update Record JSON body under `fields` while its stored operation is Read Records: the id served both value spaces before the rename, and a subblock value is not cleared when the operation changes. The scoped migration moved that body onto `readFields`, where it would reach the wire as sysparm_fields. Migration entries can now carry a `whenValue` predicate for the case where the stored operation alone cannot separate two value spaces, and the ServiceNow entry uses it to move only a plausible comma-separated projection. - Type the fork copy test harness instead of using `any`, without weakening it: every predicate shape it does not model still throws rather than matching. - Correct the dependent-omission comments. Omitting a parent-invalidated field preserves the target's stored value on Save and across an undo, where the parent nets out unchanged; on a Sync the written state is source-derived, so what it prevents there is an explicit blank reaching the fields the remap's clearing pass does not cover, nested tool params in particular. Okta's migration scope is left as-is: `okta_remove_user_from_app` and the sendEmail split shipped in the same release, so no saved block can hold legacy state for it, and widening the scope would promote an activation-era value onto the deactivation switch. Tests document the boundary. * chore(forking): move the fork-sync changes to their own PR The dependent-omission fix and the fork file-copy de-duplication are reviewed separately in #6787. They are the only changes here that overlap #6776, and they carry their own design tradeoff, so they should not ride along with the unrelated audit fixes in this PR. * fix(migrations): separate a ServiceNow write body from a projection by parsing The guard tested for a `{` or `[` prefix, so a stored scalar body — `true`, `"short_description"`, `42` — read as a field list and was promoted onto `readFields`, where it would go out as sysparm_fields. A Create/Update Record body is JSON and a projection is a bare comma-separated field list, which is never valid JSON, so parsing is the whole test rather than a guess at its opening character. Ambiguity still resolves to "not a projection", leaving the value where the Create/Update control owns it. * test(connectors,credentials): tie two assertions to what they actually prove - Webflow: a non-array collections envelope reaching `for...of` throws, which is the intended loud failure. Assert the spec-mandated TypeError plus a single request and no write-back, rather than matching V8's wording. - Credentials: the second guard test cannot observe "not deleted" — the proxy driver replays canned rows — so name it for what it does verify, that the reference check carries no workspace predicate and an empty RETURNING logs nothing. Making the driver decide the outcome would fake the database. - Drop `vi.importActual`; a plain `drizzle-orm/pg-proxy` import works now that `drizzle-orm` is un-mocked. * fix(migrations): identify a ServiceNow projection by its own shape Recognising a write body was the wrong way round. A saved body is not always well-formed: it can be a half-typed draft or carry an unquoted block reference, so neither "opens with a brace" nor "fails to parse as JSON" identifies one — and a body misread as a projection is moved to readFields with its original key dropped, losing the draft. Match the projection instead: a comma-separated list of ServiceNow field names, which are word characters plus the dot of a dotted walk. A brace, quote, colon, angle bracket or interior space fails that shape. Parsing then removes the bare scalars that satisfy it by accident. |
||
|
Waleed
|
ae2147645c |
fix(cli): resolve findings from a full command-surface audit (#6788)
* fix(cli): resolve findings from a full command-surface audit Exercised all 147 commands against a live deployment. Fixes the defects that surfaced, plus the docs and generator drift they exposed. Transport - Stop following redirects. A bare domain that 301s to www silently converted POST to GET and dropped the body, so reads worked while every write failed with a misleading validation error and login returned 405. Both the client and the device flow now explain the redirect and name the endpoint to configure, rather than carrying credentials off-origin. - Report a non-JSON response as one instead of printing the HTML page. - Name the personal-API-key remedy on a workspace-key refusal, reading the machine-readable code the API actually sends. - Drop union-branch noise from validation errors that contradicted itself. - Show paging progress on stderr for multi-page fetches. Output - Clamp record values for table only. text is the format built for pipes, and it was truncating signed URLs and tool source mid-value. - Infer timestamp, duration, bytes and boolean formatting for API-owned keys so undeclared commands stop printing raw ISO and float ms. Skips user-defined table cells and leaves json/yaml on the raw payload. - Render a declared-but-absent field as an em dash; billing credits were vanishing silently. Paths, naming and validation - Percent-encode folder paths per segment and decode them for display, so a folder reads and types as the name shown in the app. - Reject a malformed endpoint where it is set and where it resolves, instead of crashing with a URL parse trace. - Request the detail level logs list's own columns need; its workflow column could never populate. - Rename three commands that described themselves wrongly and align two flags with their siblings. Old spellings still work: hidden, warned on stderr, and kept out of help and docs. - Verify whoami against the API, separating a bad key from an unreachable endpoint, and report the workspace by name. - Correct the --yes help text, which advertised skipping a prompt that does not exist. Docs - Teach the docs generator that a flag required by the runtime is required, and that hidden commands are not documented. * fix(cli): clear the paging progress line when a page fails Progress is written without a trailing newline so it can be overwritten in place, and both paging loops cleaned it up only on success. A page that threw part-way through left `fetched 1200…` on the line the error was then printed onto, so the two ran together. * fix(cli): name a working API root when an endpoint redirects The suggested endpoint was the redirect target's origin, which drops a path prefix. A self-hosted deployment reached at https://host/sim was told to set https://www.host — not an API root, so following the advice replaced one broken endpoint with another. Derive it by stripping the request's own path from the target instead, so a prefix survives, and say nothing about --set-endpoint when the target resolves to the endpoint already configured: a trailing-slash or path normalization redirect keeps the origin, and naming the value the caller already has explains nothing. The login poll shared both faults and now shares the helper. |
||
|
Waleed
|
746a4496ba |
chore(deps): upgrade next to 16.3.1, its optimizer no longer deletes live code (#6777)
* chore(deps): upgrade next to 16.3.1, its optimizer no longer deletes live code 16.3.0 was reverted in #6242 because its Turbopack optimizer modelled a bare `return <asyncCall>()` tail call inside an async function as returning the promise object, propagated that always-truthy fact through the caller's `await`, and deleted everything after the resulting `if`. That shipped two dead code paths to production: the whole `POST /api/credentials` create path, and the insert inside `upsertAsyncToolCall`. We reported it as vercel/next.js#96595. The fix — "[turbopack] Collapse nested promises in the analyzer" (vercel/next.js#96601) — folds `Promise<Promise<T>>` to `Promise<T>` in the analyzer, and was backported as #96675 and released in 16.3.1. Verified before taking the bump: - The minimal reproduction from the issue no longer reproduces on 16.3.1. All four routes keep their code; on 16.3.0 `/api/broken` lost everything after the `if`. - A production build of `apps/sim` on 16.3.1 still emits the markers whose disappearance was the original signal: `credential_connected` (43 files), `acquireOrganizationUserMutationLocks` (28), and the `upsertAsyncToolCall` insert-path warning (10). The `return await` hardening added to both sites in the revert stays as is, and so does the TypeScript toolchain configuration. 16.3.1 published 2026-08-13, so it is inside the 7-day `minimumReleaseAge` supply-chain window until 2026-08-20 and needs an exclusion to install. The alternative is sitting on 16.2.12, whose successor we already reverted once, so the entries go in dated and come out on the next touch of the file. The mermaid and js-yaml exclusions aged out on 2026-08-11 and 2026-08-07 and are dropped here per that same rule. * fix(deps): keep the musl and win32 SWC binaries in the lockfile The release-age exclusion only listed the four @next/swc platforms that package.json pins, but next declares all eight as its own optionalDependencies, so all eight are normally resolved into bun.lock. A gated optional dependency does not fail the install — bun drops it silently — so the first install stripped both musl variants and both win32 variants from the lockfile. That left the Alpine devcontainer and any Windows machine with no SWC binary to resolve. Adding the remaining four to the exclusion list restores all eight entries at 16.3.1. Worth knowing for the next time this happens: bun.lock is sticky here. Once an optional dependency has been dropped, re-running the install — even with --force, even with the age gate switched off entirely — does not bring it back, because the resolution is not reattempted. The lockfile has to be regenerated from a base that still contains the entries, which is why this restores bun.lock from staging before re-applying the bump. |
||
|
Waleed
|
75718ab39f |
fix(execution): stop a cancelled run reporting success when its wait swallows the cancellation (#6775)
* fix(execution): stop a cancelled run reporting success when its wait swallows the cancellation Cancellation reaches a running execution over Redis pub/sub, which is at-most-once. The engine turns that into `status: 'cancelled'` via `signalCancelled`. But the wait handler also polled the durable Redis cancellation key itself, and on a hit it broke out of its sleep and returned an ordinary successful block output. The engine's `cancelledFlag` stayed false, so a cancelled run finished as `success: true` — and with a block after the wait, kept executing. Whichever detector fired first won. The engine's pub/sub path normally wins by about one round trip; when the wait's own 500ms poll landed inside that window the cancellation was lost. Consolidate detection in the engine, which is the only component that can project run status: extend the once-at-start durable backstop into a poll that runs for the life of the run and routes through `signalCancelled`. The wait handler and loop orchestrator now observe only `ctx.abortSignal`, which the engine aborts, so no leaf can observe a cancellation the engine has not seen. The loop orchestrator additionally used to ignore `abortSignal.aborted` whenever Redis was enabled, so a mid-loop timeout or client disconnect was invisible to it, and it awaited a Redis round trip on every iteration. Handlers that abort their own I/O off `ctx.abortSignal` are unaffected: that surfaces as a throw, which the cancelled branch of `run` already classifies. * docs(wait): correct the in-line wait ceiling to 5 minutes The Wait page claimed a 10-minute cap for a synchronous wait in three places. `MAX_INPROCESS_WAIT_MS`, the block description, the sub-block hint, and the validation error all say 5 minutes. |
||
|
Waleed
|
b38e4e2f91 |
docs(integrations): add missing manual intros; fix light brand tiles rendering white glyphs (#6774)
* docs(integrations): add manual intro sections to eight integration pages * fix(styling): scan blocks and ee in the tailwind content globs * docs(snowflake): correct the unload-data capability to a table source |
||
|
Waleed
|
31521d6abf |
fix(connectors): validate and repair the knowledge-base connector fleet (#6757)
* fix(connectors): validate and repair all 61 knowledge-base connectors
Audits every KB connector against its provider's live API documentation and
fixes what the audit found. The dominant defect class is deletion
reconciliation: the sync engine hard-deletes any stored document absent from a
"full" listing, and most connectors had a path where a truncated or errored
listing failed to set `syncContext.listingCapped`.
Highest-impact fixes:
- linear: `getDocument` was dead code. The query declared `$id: ID!` where the
schema is `issue(id: String!)`, so every call failed variable validation.
- salesforce: `v62.0` was substituted into a `{version}` template that already
contains the `v`, so every REST call 404'd. SOQL `LIMIT` was also used as a
page size, silently capping every sync at 200 records.
- notion: only the first level of blocks was fetched, so tables indexed empty.
- microsoft-teams: `/messages` returns messages without replies, so no threaded
content was ever indexed.
- gmail: an empty page discarded `nextPageToken`, which reads as a complete
empty listing and hard-deletes every stored thread.
- confluence: the CQL path paginated with `start` and `totalSize`, neither of
which exists on that endpoint, so label-filtered syncs stopped after one page.
- box: `supportsRefreshTokenRotation` was unset, so Box's rotated refresh token
was discarded and every credential died on its second refresh.
Removes the Evernote integration entirely: the classic EDAM API is deprecated,
its sandbox is decommissioned, and developer tokens are no longer obtainable.
Makes SFTP host-key verification mandatory, and adds an attendee-PII opt-out to
google-calendar and google-meet (default on, so existing sources are unchanged).
Bumps the contentHash namespace for notion, google-docs and hubspot so existing
documents re-hydrate once and actually receive the content fixes above.
* fix(connectors): close swallow-into-empty and cursor-taper regressions
Ship-gate pass over the connector audit. Every finding was re-verified
against the provider's live documentation or machine-readable spec before
being acted on; several pass-3 edits were reverted rather than extended.
Correctness fixes:
- fireflies: a 2xx with an unparseable body returned an empty listing
instead of throwing. fireflies runs a full sync every time, so a fault
persisting across two syncs would have tombstoned all indexed docs.
- linear: same shape via `data.issues || {}` on a non-nullable connection.
- greenhouse: a 403 from a key without scorecard permission was treated as
transient, appending `:partial` to the hash. That never matches the list
stub, forcing full re-hydration of every candidate on every sync forever.
- google-meet: `fetchParticipants` carried a 404 swallow copied from its
transcript siblings, freezing every speaker as "Unknown".
- airtable, asana, ashby: reverted page-size tapers applied over opaque
cursor tokens. The cap was already enforced server-side.
- google-docs: response byte cap resolved to 800MB and could never fire.
- google-forms, google-vault, notion, sharepoint, dropbox: `getDocument`
now throws on transient failure instead of returning null, which the
engine reads as absence.
Security:
- Retry headers are attached non-enumerably. TypeScript `private` is
compile-time only, so `SecureFetchHeaders.setCookies` was an own
enumerable property that the logger serialized into sync logs.
Docs and dead code:
- Corrected six fabricated doc citations (github, jira, jsm, linear,
google-meet, dropbox) and removed the Evernote integration entirely.
* refactor(jira): type the ADF node helpers with unknown instead of any
* fix(connectors): throw on misconfigured typeform/zendesk sources instead of returning null
A null from getDocument reads as documented absence, so on an add the
document is dropped with neither a failure counter nor a log. Both
listDocuments paths already throw on the same missing config.
* fix(connectors): keep confluence CQL page size constant; unify monday API version
The CQL search endpoint paginates by opaque cursor, and Atlassian does not
document that a cursor issued against one limit survives a request asking
for a different one. Narrowing limit to the remaining budget was the same
pattern reverted on airtable, asana, and ashby. The page size is now
constant and the cap is applied by trimming the returned page, which keeps
the cap exact without varying the request.
Monday OAuth getUserInfo hardcoded API-Version 2024-10 while every other
monday surface reads MONDAY_API_VERSION, defeating the single-source pin.
* fix(connectors): act on the final validation sweep
Findings from a read-only /validate-connector pass over all 59 changed
connectors, verified against provider specs before acting.
Silent-drop fixes (a fulfilled null from getDocument records no failure and
no log, so the document vanishes):
- ashby: candidate.info returning success with an unusable payload. Ashby
sets contentDeferred, so this path is live.
- azure-devops: an unresolvable branch, likewise live.
- dropbox: 409 covers the whole LookupError union, and restricted_content
and locked both mean the file still exists. Only not_found is absence.
- docusign: fetchFormValues swallowed every non-OK status, baking a
permanently incomplete document since the hash is metadata-only.
typeform: 'all' sent response_type=started,partial,completed, but Typeform
documents only partial and completed. An unknown enum member risks a 400
that fails the whole sync, and staging omitted the parameter entirely, so
this shipped as a regression. Now requests the widest documented set.
github: removes a utf-8 blob branch justified by a misattributed quote —
that sentence describes the encoding REQUEST parameter of Create a blob;
the GET response is documented as always base64. Also corrects two comments
that hid a real drop: >1 MB files under vnd.github+json 403 rather than
returning encoding: none.
hubspot: routes HTML detection through a shared anchored helper. The loose
pattern matched angle-bracketed prose such as an email address, and
htmlToPlainText deletes the span and collapses line structure. This matters
now because the hubspot:v2: bump rewrites every live document once.
youtube: drops an invented channel-ID format quote.
* fix(connectors): converge incidentio partial hash on settled statuses
A 403 from a key without incident_updates permission, or a 404, returns on
every sync. Marking those incomplete appended :partial to a hash that then
never matched the listing stub, so the incident re-hydrated forever without
converging. Only a transient failure may mark content incomplete now, which
matches how greenhouse already treats the same class.
* fix(connectors): flag WIQL truncation unconditionally; stop swallowing docusign form-data failures
azure-devops: the 20,000-item WIQL ceiling was probed by asking for a matching
item with an id beyond the largest returned. That probe is unsound — buildWiql
orders by ChangedDate DESC while ids are assigned in creation order, so the
highest-id match is almost always inside the returned window. The probe came
back empty for genuinely truncated projects, left the listing unflagged, and
let deletion reconciliation remove every indexed item outside it. Flag
unconditionally instead; the cost is a project sitting exactly at the ceiling
not reconciling deletions until a full resync.
docusign: fetchFormValues threw on a non-404 status and then caught its own
throw, returning []. The earlier fix was a no-op. The catch now rethrows, so a
transient failure produces a failed row instead of a permanently incomplete
document under a metadata-only hash.
* fix(connectors): restore airtable AI Text indexing; floor sentry maxIssues
airtable: staging rendered object cells with a JSON.stringify fallback, so AI
Text values and nested lookup arrays reached the index. This branch replaced
that with a fixed key-probe list to stop attachment-URL hash churn, but the
probe list has no fallback — aiText ({state,isStale,value}) and nested arrays
rendered to the empty string and vanished from every document, and the
content-derived hash never moved when the text regenerated. Read `value` last,
after the existing probes, and recurse on nested arrays. The generated text is
stable rather than an expiring signed URL, so this does not reintroduce churn.
sentry: maxIssues now feeds the request limit, and Sentry rejects a non-integer.
validateConfig accepts a fractional entry, so a config that saved cleanly would
fail every sync at listing time. Staging was immune only because it sent a
hardcoded page size.
|
||
|
Waleed
|
cc1a278d73 |
fix(integrations): close defects found by an independent cold audit (#6767)
* fix(integrations): repair Update SLO and advanced OData filters
An independent audit — eight cold readers, one per integration, given no
prior findings — checked the eight integrations merged to staging today.
Two defects broke an operation outright; both are fixed here.
datadog: Update SLO rewrote every non-metric SLO to `metric`. The SLO Type
dropdown carried a `metric` default and its condition covered both create and
update, so an untouched control reached mergeSloUpdatePayload as an edit. A
metric SLO requires `query`, and the merged body carries monitor_ids or
sli_specification instead, so Datadog rejected it — Update SLO was unusable on
monitor-based and time-slice SLOs, with no way to express "keep the current
type". Update now has its own control defaulting to "Keep current".
microsoft_ad: list_users, list_groups and list_service_principals emitted
$count=true only alongside $search, so any $filter using an advanced operator
(ne, not, endsWith, startsWith on non-indexed properties) returned 400. Graph
requires $count=true with ConsistencyLevel: eventual for those. list_devices
already did this correctly; the other three now match it.
Also from the same audit: Datadog path IDs are trimmed before encoding in all
20 URL builders rather than 2, matching the existing get_monitor test.
* fix(integrations): cloudflare, crowdstrike, and mssql audit findings
From the independent cold audit. Cloudflare: DNS analytics no longer emits
fabricated min/max telemetry (Cloudflare documents both as always empty);
purge_everything defaults to specific-purge and errors when combined with
target lists; three unsourced description claims corrected; Array.isArray
guards on four older list transforms.
CrowdStrike: IOC sort placeholder corrected to the dot form (created_on.desc,
not the nonexistent created_timestamp); the 500-indicator cap relabelled as a
Sim bound rather than a CrowdStrike one; credential failures return 401 rather
than 500; prevent_no_ui noted as unenumerated.
MSSQL: introspect no longer lets the model choose the database; row and byte
caps on reads; introspection collapsed from 4N+2 to 6 fixed queries; WHERE and
identifier guards run before the connection opens so rejections are 400 not
500; SAVE TRANSACTION, OPEN/CLOSE key, DEALLOCATE and ADD SIGNATURE added to
the statement screen as two-token phrases; encrypt wording corrected to say
TDS 7.4 encryption is negotiated, not guaranteed.
* fix(splunk): publish the real output tables and read the errors Splunk sends
The docs generator parses tool source text and resolves a shared `outputs`
const only from the family's `types.ts`, so Splunk's helpers in `utils.ts` were
invisible to it: seven operations published the block's union of every output
instead of their own. Run Search and Get Search Results each shipped a ~50-row
table naming savedSearches, alerts, indexes, and apps they never return, Cancel
Search Job lost `messages`, and the four list tools lost `total`/`offset`.
Inline the four helpers into each consuming tool and delete them, since
relocating a shared const only moves the trap.
Also:
- Add a `splunk-errors` extractor for the documented `{messages: [{type, text}]}`
envelope and set it on all twelve tools. A rejected SPL string, the most common
failure, previously fell through to the status text and reported "Bad Request".
- Read `searchEarliestTime`/`searchLatestTime` with `asNumber`. The job entry
documents them as bare epoch numbers, so `asString` returned null for every
`output_mode=json` response.
- Project the `<messages>` block of the XML job-control response. It is the only
payload that endpoint returns, so `cancel_search_job.messages` was always empty.
- Mark the nullable job outputs optional, matching the transform.
- Default `run_search` to `max_count=1000`. A oneshot search has no paging escape
hatch and Splunk's own default is 10000 rows in one buffered response.
- Add suggested skills to `SplunkBlockMeta`, grounded in `tools.access`.
The regenerated tool metadata also picks up the Cloudflare and MSSQL output
changes from the previous commit, which were never synced.
* fix(okta,servicenow): apply integration audit findings
Cherry-picked from fix/okta-servicenow-audit-followups (6db0f54), whose base
predated the earlier audit round; the duplicate isOktaFlagEnabled that produced
is resolved in favour of the existing richer helper, which already accepts
'yes'/1/'on' as well as true/'true'.
okta: get_logs no longer advertises hasMore forever. A System Log query with no
'until' is a polling query, and Okta always returns a next link for one, even on
an empty page — so any loop driven by hasMore never terminated, including the
one our own shipped skill instructs the agent to run. errorCauses is now
surfaced, so a failed write reports the real reason instead of the useless
'Api validation failed: profile'. sendEmail routes through one coercion helper
across all four lifecycle tools. update_group's declarative fallback throws
rather than silently truncating an extensible group profile.
servicenow: attachmentLimit and limit no longer overwrite each other. Neither
assignment was scoped to an operation, so all 12 paginated operations could
silently return a row count the user never asked for — defeating the block's own
design, which gave attachmentLimit a unique id precisely to avoid this. All
seven approval states are published by ServiceNow and are now reachable from the
filter, with the space-vs-underscore punctuation documented. The five legacy
generic tools route through the shared response helpers, and the folder's only
'any' is gone. Block skills now name the semantic operations.
* chore(integrations): regenerate catalog and docs artifacts
* fix(integrations): disclose MSSQL truncation and keep Okta's poll cursor
Three defects the review round found in the audit fixes themselves.
MSSQL capped a recordset and then reported it as complete: `executeQuery`
computed `truncated`/`truncationReason` but all five statement routes returned
only `message`, `rows`, and `rowCount`, so a caller could not tell paging was
required. A shared `toRowsResponseBody` now folds the reason into `message`
for an agent reading the status line and exposes the two fields for a caller
that branches on them.
The byte ceiling also admitted a single oversized row as a lone exception, so
one `nvarchar(max)` value serialized an unbounded body — the ceiling bounded
everything except the case it exists for. A row is now admitted only when it
still fits, and the drop is disclosed rather than read as an empty table.
Okta's `get_logs` nulled `nextCursor` alongside `hasMore` on an empty polling
page. Terminating the loop is right, but the cursor is the resume handle Okta
tells callers to persist, so a scheduled workflow that hit one quiet interval
restarted from `since` and re-delivered events it had already processed. The
two answer different questions and now diverge.
Cloudflare's purge block no longer lets the invalid combination be built: the
four target fields are hidden once Purge Everything is selected, so the tool's
guard is a backstop rather than a reachable hard error.
* fix(okta,servicenow): stop sending requests the APIs reject
Okta documents `since` and `after` on the System Log as mutually
exclusive, so `get_logs` lets the cursor win rather than sending both —
the shape a scheduled poll that persists the cursor would otherwise send.
Seven boolean query params reached Okta interpolated raw, so an agent
tool call supplying `yes` was rejected. Each now routes through
`isOktaFlagEnabled`, keeping its existing send-or-omit behavior.
A cleared ServiceNow limit/offset/quantity stayed `''` through the block
mapper and was appended as a valueless `sysparm_limit=`. The mapper now
resolves a blank to undefined, and the tools skip a blank as well.
* fix(integrations): mssql guard gaps and Entra query, scope, and output findings
MSSQL read-only screen
- Screen RENAME, documented T-SQL DDL for Azure Synapse dedicated SQL pools and
Analytics Platform System, which are reachable over TDS with exactly the
connection fields this block exposes. `SELECT 1 RENAME OBJECT dbo.t TO t2`
was a schema change passing an operation advertised as read-only.
- Screen the Service Broker family: RECEIVE as a word, and END/MOVE/GET
CONVERSATION and SEND ON CONVERSATION as two-token phrases, since END closes
every CASE. RECEIVE is a destructive read and END CONVERSATION WITH CLEANUP
drops a conversation's messages.
MSSQL routes and block
- Build the insert statement before connecting, matching update and delete, so a
bad identifier answers 400 instead of burning a TLS+login and returning 500.
- Declare `truncated`/`truncationReason` on the block, which the tools declare
and the routes emit but the block left unreferenceable.
Microsoft Entra ID
- Pair `$count=true` with `ConsistencyLevel: eventual` conditionally. Graph
documents `hasMembersWithLicenseErrors`, `isLicenseReconciliationNeeded`, and
`identities/any(i:i/issuer)` as filterable only *without* advanced query
parameters, and documents advanced queries as unsupported in Azure AD B2C
tenants, so the unconditional pair broke filters that previously worked. When
continuing from a nextLink the pairing is read off the link itself.
- Request `LicenseAssignment.Read.All` instead of `Directory.Read.All`. The
latter was needed by `GET /subscribedSkus` alone, whose permission table names
the former as least privileged and does not list the ReadWrite scope we hold.
- Enumerate the block's real output keys instead of a single `response` object
no tool emits.
* fix(splunk,datadog): stop truncating searches and send mute/unmute as query params
Splunk run_search: revert the `max_count=1000` default added last pass. It was
wrong on both halves. Splunk documents the parameter as "the number of events
that can be accessible in any given status bucket. Also, in transforming mode,
the maximum number of results to store" — so for a non-transforming oneshot it
bounds status buckets, not the response, and for a transforming search (`| stats`,
`| timechart`, which is what the block's own skills generate) it capped results
at 1000 where Splunk would have stored 10000, silently. The block's `maxCount`
placeholder already read `10000`, contradicting the code. Send `max_count` only
when the caller sets it and restate the description in Splunk's own terms,
matching create_search_job. The real guidance — a oneshot buffers the whole
result set, so use Create Search Job + Get Search Results for anything large —
moves into the tool description and the search-splunk-logs skill.
Datadog mute/unmute: send `scope`, `end`, and `all_scopes` as query parameters.
`MuteMonitor` and `UnmuteMonitor` declare no `requestBody` in the authoritative
spec (docs.datadoghq.com/resources/json/full_spec_v1.json — the generated
datadog-api-client-go v1 schema omits both operations and is a subset, not the
authority); all three parameters are `in: query`. Sent as a JSON body they are
dropped, so a scoped, time-boxed mute becomes an indefinite mute across every
scope and unmute's "all scopes" never applies — answered with a 200 and the full
monitor object, so nothing surfaces.
Datadog list_monitors: imply `page=0` when a page size is set without a page.
Datadog "returns all monitors without a `page_size` limit" when `page` is absent,
so Page Size was inert from a control that reads as a bound. `page` is not
defaulted when neither is set — that would silently truncate a caller relying on
the documented return-everything behavior.
Also:
- Note in get_fired_alerts that `name=-` returns every saved search's fired
alerts and the endpoint documents "Request parameters: None", so there is no
count/offset to bound it.
- Fix the Splunk block's `messages` output blurb: `[{type, text}]` holds for the
search and job-control operations, but get_search_job returns an object.
- Generalize the Datadog block's numeric coercion (`datadogPageNumber` →
`datadogNumber`) over all 32 bare `Number()` mappings, so a typo or unresolved
reference is omitted rather than sent as `NaN`/`null`, and an explicit `0`
survives the old truthiness guard.
- Disclose create_event's documented 18-hour `date_happened` ceiling, and that
send_logs' `ddsource: "custom"` is a Sim default rather than a Datadog one.
* chore(integrations): regenerate tool metadata and docs
* fix(integrations): resolve confirmed findings from cold block audit
Cloudflare: clear purge_cache advanced targets across operations; send
action_parameters/ref/logging on rate-limit rule updates; migrate off the
deprecated batch zone-settings endpoint; correct MX/URI priority wording;
stop coercing blank numerics to 0.
CrowdStrike: seed includeHidden to match Falcon's documented default.
Microsoft Entra ID: resolve a UPN to an object ID for app role assignment;
wrap 21 array outputs in items.properties so nested paths resolve.
Okta: route assign_user_role's notification flag through isOktaFlagEnabled.
ServiceNow: drop the triage skill's claim of a default limit that does not exist.
Splunk: always assign coerced numerics so raw values cannot leak through the
executor's raw-input merge.
* fix(editor,credential-group): mask secrets outside short-input and stop a per-option abort from failing a shared query
config.password only reached the short-input renderer, so eight credential
fields rendered in plaintext: private keys on ssh/sftp/pi/kalshi, the
Secrets Manager payload, the STS web-identity and SAML assertions, and the
Browser Use variables table. long-input, code, and table now honor the flag.
Code fields mask through the highlighter because react-simple-code-editor
paints its textarea transparent; the table masks every column but the first
so key/value rows stay distinguishable. A registry-walking audit test fails
both on a password flag sitting on a type that cannot honor it and on any of
the eight fields losing its flag.
credential-group threaded a per-option AbortSignal into the fetch registered
under the workspace-wide credential group list key, so closing one option
panel rejected every co-observer with an AbortError that is not a React
Query cancellation. The shared fetch now runs on its own lifecycle signal.
* fix(mssql,editor): measure the response cap in UTF-8 and stop search from unmasking secrets
capRecordset sized rows with JSON.stringify(row).length, which counts UTF-16
code units while the emitted body carries raw UTF-8. CJK is the worst case at
3 bytes per unit, so a recordset admitted as 10 MB serialized to 28 MB. Rows
are now measured with Buffer.byteLength, serialized once each, with array
punctuation charged exactly and a reserve held back for the response envelope.
Workflow search revealed masked credentials without the user touching the
field: the search panel keeps focus in its own input and only scrolls the
match into view, so typing a guess painted a private key on screen. The index
is built client-side from values already in page memory, so this was never a
privilege boundary, but masking exists to prevent incidental display and a
screenshare-visible reveal defeats it. Focus is now the only reveal, applied
through one shared policy across all four renderers.
* test(editor): drop PEM-shaped fixtures from the masking tests
The masking fixtures carried a literal OPENSSH private key header, which
GitGuardian flags as a committed secret even though the body was only the
base64 of "openssh-key-v1". The fixtures now use an obvious marker string,
and the assertions derive their match text and dot counts from the fixture
instead of restating its bytes.
* refactor(editor): drop the dead isSearchHighlighted prop
No renderer consumed it. The editor computed it at two call sites and
sub-block passed a hardcoded false into renderLabel's slot for it, so even
the one function that declared a parameter never saw the real value. Its
only live effect was in the memo comparator, where an unconsumed value
changing forced a re-render for nothing.
The name stays in the masking audit's forbidden-inputs list, which guards
against a search signal being wired back into a masking decision.
|
||
|
Waleed
|
0844d4166b |
feat(jotform): add Jotform integration (#6772)
* feat(jotform): add Jotform integration
Adds 43 tools covering forms, questions, submissions, reports, webhooks,
labels, and account operations, plus the block, icon, and generated docs.
Request shapes are pinned against the API's own curl samples and the
official SDKs: PUT /form/{id}/properties and PUT /form/{id}/questions each
take a named envelope while PUT /form and the bulk-submission PUT take
their payload bare, and submission answers accept both the nested object
and the documented {qid}_{subfield} shorthand.
Skips the deprecated folder endpoints in favor of labels, and leaves out
endpoints whose response shape the docs do not publish.
* fix(jotform): harden the error envelope against quoted codes and non-JSON bodies
Jotform quotes `responseCode` on some endpoints and not others, so a
typeof-number test skipped the check on the quoted ones and turned an auth
failure into a successful tool result with empty output. Also caps the raw
body fallback, since an upstream gateway can answer with an HTML page
instead of the documented envelope.
* fix(jotform): stop duplicate question labels overwriting derived answers
Question labels are not unique — a form can carry two questions both
labelled "Email" — so keying the derived `values` map on the label alone
dropped all but the last and handed downstream workflows a confidently
wrong answer.
Every occurrence of a repeated label is now suffixed with its question ID,
rather than only the later ones, so the result does not depend on answer
order and a newly duplicated label reads as absent instead of as an
arbitrary winner. The id-keyed `answers` record was already complete and
is unchanged.
* fix(jotform): make the label-keyed answer map collision-proof
Question labels are free text, so the disambiguation key added in
|
||
|
Waleed
|
aeb5624cc8 |
fix(integrations): close regressions found in the final validation sweep (#6764)
* fix(integrations): close regressions found in the final validation sweep
An independent read-only audit of the eight integrations merged to staging
today found defects in every one, most of them side effects of the surgery
those PRs performed on already-shipped code.
Data loss and destructive paths:
- cloudflare: restore the shipped subBlock ids on read filters so existing
workflows keep their DNS/zone/purge filters. Losing them made
list_dns_records return the entire zone with success: true, which a
downstream delete fan-out would then target. The colliding write controls
are renamed instead, chosen by blast radius.
- cloudflare: refuse an update_ruleset_rule that would tear down the rule it
edits. PATCH is a replace, so an omitted action_parameters unbound the WAF
managed ruleset and every override under it.
- cloudflare: split the hidden `enabled` control so a value set while drafting
can no longer disable a live WAF or rate-limiting rule.
- cloudflare: stop `name` leaking into update_dns_record and renaming a live record.
- okta: stop a blank name overwriting a stored group name via the LLM path.
The block guard covered only the UI.
Broken on the default path:
- microsoft_ad: update_user sent accountEnabled: "" on its own default, so
every call left at "No Change" failed. Same tri-state defect already fixed
for forceChangePasswordNextSignInWithMfa; `visibility` fixed alongside it.
- cloudflare: `domain` is required for self_hosted (the default app type),
ssh, vnc and rdp; add saas_app/target_criteria and drop dash_sso, which has
no request variant.
Silent wrong results:
- datadog: list_monitors inherited Create Monitor's tag filter and returned a
filtered list as if complete.
- servicenow: `fields` carried both a JSON body and a projection on the three
legacy generic operations. The regression test for this fed already-JSON and
could not fail; it now feeds a real projection.
- splunk: cancel_search_job reported failure on success by parsing an XML body
as JSON; readSplunkJson now tolerates it.
- okta: sendEmail === true dropped a string 'true', silently skipping the
deactivation email.
Security:
- mssql: add writetext/updatetext/readtext to the statement screen. \bupdate\b
cannot match UPDATETEXT, so both were reachable through the read-only path.
- crowdstrike: chunk repeated-query ids. At the published caps a single request
built a ~68 KB query string, past typical proxy limits.
Also: splunk count=0 unbounded read, splunk pagination totals, the `nobody`
placeholder that reintroduced the namespace bug by copy-paste, okta cursor and
activate controls split per operation, servicenow sysparm_having syntax and two
required controls no longer pre-seeded with consequential values, datadog block
outputs reconciled with tool outputs, and 16 escaped apostrophes that corrupted
the published Entra docs.
One scope removed from microsoft_ad (User.Read.All). Directory.Read.All and
GroupMember.ReadWrite.All were proposed for removal and verified still required;
a test now asserts they stay.
* fix(integrations): surface corrupt Splunk bodies and partial CrowdStrike deletes
Narrows readSplunkJson's non-JSON tolerance to XML. The dispatching and
job-control endpoints answer in XML, but a body that is neither empty nor XML
was meant to be JSON, so swallowing its parse failure handed get_search_results
an empty envelope and reported a lost result set as a search with zero events.
Annotates a batched CrowdStrike delete that fails partway with the IDs its
earlier batches already removed. Falcon cannot roll those back, so a bare
failure left the caller unable to tell what was gone and a blind retry
re-targeted IDs that no longer existed.
Registers the Cloudflare subblock-ID migration the registry-stability check
requires. The suffixed read-filter IDs never shipped in a release and every
block already materializes the restored IDs, so they are dropped rather than
renamed onto values the collision guard would discard anyway.
* fix(integrations): close the three defects Bugbot found in the sweep
An execute rule sent an explicit empty action_parameters object past the new
guard, because presence was checked rather than emptiness. `{}` is the same
payload Cloudflare's schema default produces, so it unbound the managed ruleset
the guard exists to protect.
Datadog's new List Monitors pagination used a bare `Number()`, so a typo or an
unresolved reference in either advanced field reached Datadog as a literal NaN
— the pattern this same sweep fixed for Entra `top` and the Splunk numerics.
Okta's block still marked the group name required on update, blocking a
description-only update that the tool, its merge helper, and the API all accept.
* fix(integrations): confine the Splunk XML tolerance and the CrowdStrike commit list
Splitting the XML tolerance out of readSplunkJson into readSplunkDispatchJson
puts it only on the three dispatching and job-control tools that need it. The
results path can no longer read any non-JSON body as an empty envelope, so a 2xx
HTML interstitial surfaces instead of reporting a search that matched nothing.
The dispatch reader anchors on the one documented `<response>` root, so an
interstitial fails there too.
A batched delete now records the IDs Falcon echoed in `resources` rather than
the IDs that were requested. A batch can answer 200 while reporting per-ID
failures, and naming those as deleted told the caller to drop still-live
indicators from the retry.
* test(crowdstrike): pin batched partial-delete parity with an unbatched request
A 2xx envelope carrying per-ID errors is a partial success, not a failure —
failedWithoutResources fails the operation only when nothing came back at all.
The batched path already reports it exactly as a single request does, with
deletedIds naming what Falcon confirmed and errors naming what it refused. Pin
that so the contract is not mistaken for a swallowed failure.
* fix(splunk): read the dispatch XML envelope instead of discarding it
A dispatch answering in the documented XML form was replaced with an empty
object, so create_search_job and dispatch_saved_search threw a missing-sid error
after the remote job had already been created — stranding a job the caller could
no longer poll or cancel. The envelope is now projected onto the same `{ sid }`
shape output_mode=json produces, so the search ID survives.
Matching only the opening tag also accepted a body cut off mid-transfer, which
on a cancellation reported a truncated response as a successful cancel. The
pattern now spans the closing tag, so a truncated envelope falls through to
JSON.parse and throws.
|
||
|
Waleed
|
025ea4d2bd |
fix(docs): serve JSON-LD in the HTML, fix sidebar spacing, and tighten the CLI guides (#6763)
* docs(cli): use -g for the install, and cut the prose that was not pulling weight
`--global` is valid but `-g` is what every comparable CLI documents, and the
long form only came from the package README. Also drops the yarn tab: it read
`yarn global add sim`, which works on Yarn 1 only — Yarn 2 removed global
installs, so that command fails for anyone on a modern Yarn. Adds `npx sim` for
running without installing.
The guides had accumulated design rationale that belongs in code comments rather
than user docs — why the filter grammar is JSON, why the config section naming
is asymmetric, why an unexpected error keeps its stack trace. Surveying how gh,
Vercel, Turborepo, Deno, Bun and Supabase write theirs, none carry that kind of
justification, and callouts are reserved for content whose absence produces a
wrong result rather than for general asides.
So: 1016 lines to 763, and 12 callouts to 3. The three that remain are the
pairing-code check, that `sim logout` does not revoke the key, and the
`--limit 100` default on `batch-delete`/`batch-update`, which silently truncates
a larger match. Troubleshooting drops the entries whose error message already
contained its own fix and keeps the seven whose cause is not obvious.
* fix(docs): render JSON-LD as native script tags so it reaches the HTML
All four structured-data blocks — WebSite, TechArticle, BreadcrumbList,
SoftwareApplication — were rendered with `next/script`, which never emitted a
script tag. Measured on a production build, `/api-reference/getting-started`
contained zero `<script type="application/ld+json">` elements; the payload
existed only in the `__next_s` client-injection queue and the RSC flight data,
so anything reading the served HTML saw no structured data at all. React was
also logging "Encountered a script tag while rendering React component" on every
page.
`next/script` is for loading and executing JavaScript. JSON-LD is data, and
Next's own guidance is a native `<script>` in the component. `serializeJsonLd`
already escapes the `<` character to its unicode form, which is the
sanitization that guidance calls for, so only the element changes.
Same build, after: three valid tags per page with `WebSite` in `<head>`, and the
injection queue gone entirely.
* fix(docs): scope the flush-separator rule to a container's first separator
`[data-separator]:not([data-separator] ~ [data-separator])` was meant to keep the
first sidebar group flush against the top padding, but `~` only reaches siblings,
so it also matched the first separator inside every expanded folder. Under
Self-Hosting, "Install" lost its top margin and crowded the "Architecture" link
above it — 25px of gap where "Configure" and "Operate" below it had 40px.
`:first-child` expresses the intent directly. Only the four sidebar roots open
with a separator; every nested folder starts with a page, so the intended case
still goes flush and nothing else changes.
* fix(docs): move the flush-separator rule onto the separator component
Keeps the styling with the component that owns it, per the repo standard, and
lets the global rule be deleted outright rather than corrected — `global.css`
now only loses a rule in this PR. Tailwind's `first:` variant compiles to the
same `:first-child` selector, so behavior is unchanged: the build emits
`.first\:mt-0:first-child{margin-top:0}` and the prerendered HTML carries the
class on the separator.
|
||
|
Waleed
|
257029a60c |
feat(microsoft_ad): licensing, security, audit, role, and device operations (#6742)
* feat(microsoft_ad): licensing, security, audit, role, and device operations
Deepens the Microsoft Entra ID block from 12 to 36 tools against the Microsoft
Graph v1.0 reference: license assignment and tenant SKUs, password set/reset,
sign-in session revocation, authentication methods, sign-in and directory audit
logs, app role and directory role assignments, service principals, device reads,
and conditional access policy reads.
Device write (device-update, device-delete) is deliberately excluded. Both
document Directory.AccessAsUser.All as their only delegated scope, with the
higher-privileged read documented as unavailable, so supporting them would mean
requesting tenant-wide act-as-the-user directory access for two operations that
additionally require the caller to hold Intune Administrator.
Also drops an undocumented ?$select= from create_user that was silently nulling
department and accountEnabled in the response.
* fix(microsoft_ad): resolve OData filter and search by owning operation
The params mapper assigned result.filter from each filter subBlock in turn, so
the last non-empty one won regardless of the selected operation. Because a
subBlock keeps its value after the operation changes, a filter written for one
endpoint was sent to every other collection operation — invalid OData against a
different Graph resource, or a silently wrong page.
Resolves the filter and search terms from an explicit operation-to-field map
instead, so each operation reads only the field it owns.
* fix(microsoft_ad): clear non-owning filter and search on the merged inputs
The executor merges { ...inputs, ...transformedParams }, so declining to copy a
stale filter is not enough — the serialized value survives the merge and still
reaches the tool. Advanced-mode subBlocks are serialized on non-emptiness alone
and never have their condition evaluated, so the value is present even when the
field is hidden.
Write filter and search on every operation, as undefined when the operation owns
neither, so the merge clears them.
* fix(microsoft_ad): clear the MFA flag and let paged user operations continue without a User ID
The set_password MFA dropdown only wrote its key when non-empty, so the "No Change"
empty string survived `{ ...inputs, ...transformedParams }` and reached Graph in place
of a boolean. Assign it explicitly, including as `undefined`, the same way `filter` and
`search` are handled.
`list_user_app_role_assignments` and `list_user_devices` page by `@odata.nextLink`, and
both tools already treat `userId` as optional once a continuation URL is supplied. Drop
them from the required set when Next Page is filled in so pagination-only runs pass block
validation.
Also note on the reset_password output that a generated password reaches workflow outputs,
run history, and the model, matching how other tools that return secrets document exposure.
* fix(microsoft_ad): require the service principal ID only on the first page
Every other single-resource ID field pairs its condition with a matching required
rule; servicePrincipalId had none, so a first-page run could pass block validation
with an empty ID and fail inside the tool instead. Require it unless a continuation
URL is supplied, matching the paged per-user operations.
* fix(microsoft_ad): reject a continuation URL from a different collection
Every paged operation reads the one shared Next Page field, and a subBlock keeps
its value after the operation changes. Paging /users and then switching the block
to /devices short-circuited back to the user page, silently returning the previous
collection instead of the selected one.
Assert the continuation URL's terminal path segment against the collection the tool
actually reads, which also rejects a nextLink pasted from an unrelated response.
|