mirror of
https://github.com/simstudioai/sim.git
synced 2026-10-31 03:47:12 +08:00
ea70f8dcf119a2b8e48a70d85b406d44d6076dfd
549
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
ea70f8dcf1 |
feat(harmonic): add contact workflow integration (#6902)
* feat(harmonic): add contact workflow integration * fix(harmonic): sync docs manifest * fix(harmonic): address integration review findings * feat(harmonic): add the missing people endpoints and fix two error paths Extends the integration from 4 to 13 tools, covering every non-deprecated people-scoped Harmonic endpoint, and repairs two defects found by validating the existing tools against Harmonic's OpenAPI and API reference. New tools: - Enrich Person (POST /persons) — the only path from a LinkedIn URL or email a workflow already holds to a Harmonic contact. - Get Person, Get Company Employees — account-based sourcing; employees returns URNs that chain into Batch Get People. - Saved-search net-new results and their acknowledgement, so a monitor stops reprocessing the entire result set on every poll. - Bulk email enrichment: submit, poll, and quota, plus Get Enrichment Status. Fixes: - The error extractor dropped Harmonic's string and object `detail` envelopes. A tool that names an extractor gets no fallback chain, so every FastAPI abort surfaced as "Request failed with status 403". The enrichment 404 also carries the scheduled `enrichment_urn`, which was being discarded — that URN is the only handle on the job, so it is now kept in the message. - The saved-search selector failed the whole dropdown instead of degrading: the response cap was half the sibling value on an endpoint that is unpaginated and returns every saved search with its full query object, and the option ceiling threw rather than truncating. Raised to 1MB and switched to truncate-and-warn, matching the other data-driven selectors. Clearing net-new results now requires an explicit scope. Harmonic treats an absent `entity_urns` as "clear everything", so an empty field would have silently discarded the backlog. Scope deliberately excludes company-side, deal, typeahead, network, and Scout streaming endpoints, and every endpoint retiring on 2026-11-05. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Waleed
|
865f8173ab |
feat(affinity): add Affinity CRM integration (#6908)
* feat(affinity): add Affinity CRM integration Adds the Affinity v2 API as a block with 70 tools, covering 86 of the 87 documented endpoints. Only Send Feedback is omitted — it reports product feedback to Affinity rather than doing workflow work. Endpoint families that differ only by an entity segment are one tool with an entityType param, so companies/persons field, list, row, and relationship reads, the company/person merge endpoints, and entity notes each collapse into a single operation. * chore(affinity): regenerate the docs manifest for the new integration page |
||
|
Theodore Li
|
a99f61bee9 |
feat(api): add v2 resource management endpoints (#6900)
* feat(api): add v2 resource management endpoints * fix(cli): gate destructive v2 commands * fix(test): update v2 request-slice count * feat(cli): add shared workspace profiles |
||
|
Vikhyath MondretiandClaude Opus 5
|
85290693c4 |
refactor(search): one definition of what a search occurrence is (#6905)
Follow-up to #6901, closing two places where the workflow search index and the Note card that mirrors it could drift apart. Neither is a live bug; both are the shape that produced one — the card silently disagreeing with the panel about which hit is which, counted in one place and painted in another. THE SCAN. #6901 shared `foldSearchWhitespace` but left the scan around it duplicated: normalize, then non-overlapping `indexOf` stepping by `max(len, 1)`, written out once in the indexer and once in the renderer package. They agree today. They would stop agreeing the moment either grew whole-word matching, diacritic folding, or a regex mode, and the failure is silent. Both now call one `forEachSearchOccurrence` in `@sim/utils/string` — the only place either package can share, since the card renders from a package that cannot import from `apps/*`. THE DECLARATION. The indexer projects markdown escapes only for a field declaring `searchTextFormat: 'markdown'`; the card projects unconditionally, because it cannot read the block registry. Dropping that one line from the Note config would leave them disagreeing with nothing to catch it, so a test now pins it and explains why. Net negative in lines: this deletes a duplicated loop rather than adding a layer. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vikhyath MondretiandClaude Opus 5
|
2e111f615c |
fix(search): answer a Note match on the canvas card (#6901)
* fix(search): answer a Note match on the canvas card A workflow search match inside a Note counted towards the result total and then highlighted nowhere: the editor panel renders nothing for a Note, and `clearCurrentBlock` — which the panel calls to refuse one — also cleared the shared `activeSearchTarget`, destroying the very target the card was about to paint. Searching a 15k-character note reported "1 of 6" and moved nothing. The card's read view is now the surface that answers: - A rehype plugin marks every rendered occurrence; the current one is picked by an ordinal counted with the same scan the indexer uses, and travels by context so cycling matches does not re-parse the document. - `<Streamdown>` is keyed on the query. Its memo comparator ignores `rehypePlugins`/`components`, so a plugin change alone cannot re-render it — marks appeared only when something else remounted the card, and then outlived the query that produced them. - The canvas selects, centres and expands the Note, because a compact card resets its scroll region to the top and cannot hold a position deep in its own body. Scrolling to the mark is `scrollTop` arithmetic, never `scrollIntoView`, which would drag ReactFlow's transformed viewport off-frame. - Title matches mark the name too. `activeSearchTarget` is re-published with a fresh identity on most of the search panel's renders, so subscribers take primitives. Holding the object in `WorkflowContent` — the panel's own ancestor — closed an unbounded update loop. Separately, the serializer escaped every underscore, writing `SB\_ACTION\_ROUTER\_SECRET` into the document. CommonMark's intraword rule means that backslash carries no meaning, and search matches the stored markdown, so it made anything with an underscore unfindable in a note that plainly showed it. Dropped outside code regions, where the serializer emits verbatim and a backslash is the author's own character. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(search): honour fences and folded whitespace in Note highlighting Two review findings, both real. The intraword-underscore cleanup guarded code with a pattern that recognised only the shortest delimiter forms — a bare ``` pair and a single-backtick span. A ````-fenced block, a tilde fence, or a ``multi-backtick`` span ended the region early and handed the rest of the author's code to the rewrite, turning `a\_b` into `a_b` inside their code sample. Fenced blocks are now walked a line at a time, tracking the opening delimiter exactly the way stripEmptyListItemLines already does (three or more, closed only by a run at least as long), and the inline branch matches a backtick RUN closed by one of equal length. The note scanner claimed to be the same scan as the indexer's `findTextRanges` but did not fold whitespace, which staging added since this branch was written. The indexer folds every `\s` to a space, so a phrase matches across a soft line break — which `remark-breaks` renders as a `<br>`, splitting the phrase over two text nodes that a per-node scan could never see. The hit counted in the panel and highlighted nowhere, the exact bug this branch exists to fix. The plugin now scans runs of continuously-readable text rather than single nodes, so a match spanning an inline boundary (a soft break, a bold word) is wrapped as several marks sharing one ordinal. Runs end at any non-inline element, so two paragraphs are never joined into a phrase the reader cannot see. `foldSearchWhitespace` moved to `@sim/utils/string`: the canvas card renders from a package, which cannot import from `apps/*`, and two copies of that rule silently disagreeing is precisely what produced the second finding. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(markdown): close a fence only on a bare delimiter run A closing fence carries nothing but its delimiter run; a line that merely starts with one is content. The guard matched the prefix alone, so an interior line like ` ````example ` inside a same-length fence ended the block, and every cleanup below then processed the author's remaining code as prose — dropping the backslashes from their `a\_b`. Both fence walks in this file shared that flaw, so both now go through one `closesFence`, which requires the run to be followed by nothing but whitespace. Strictly more conservative: a fence stays open longer, so more content is left verbatim. Scope, stated plainly: the serializer always opens a block with one more delimiter than the longest run inside it, so its own output cannot reach this shape today, and `postProcessSerializedMarkdown` only ever sees serializer output. This is a correctness fix that removes an unstated coupling to that choice, not a live corruption path. The tests therefore exercise `postProcessSerializedMarkdown` directly — a round-trip test of the same input would pass either way, which is exactly the vacuous check worth avoiding. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(markdown): skip quoted fences, and stop joining runs across inline tags Two review findings, both real, both the same shape: a rule that looked at the rendered form and forgot what the source actually says. QUOTED FENCES. The fence walk only recognised a bare delimiter run, but the serializer writes a fence inside a blockquote or a `[!NOTE]` callout with a `>` on every line. Code state was therefore never entered there and the block's interior was cleaned up as prose: `> x = a\_b` round-tripped to `> x = a_b`, losing the author's backslash. Unlike the fence-length cases this one is reachable today — verified against the real serializer before and after. Both fence walks now unquote the line first. INLINE JOINS. Runs concatenated the visible text of every inline tag, so `a<strong>b</strong>c` read as `abc` — a hit that cannot exist in the markdown the indexer scans, where `**` sits between the words. That is worse than a spurious mark: `occurrenceIndex` counts SOURCE occurrences, so a fabricated hit earlier in the document steals the current ordinal and paints the mark on text the search never matched. Only `<br>` continues a run now, because it alone stands for a character the source really has (a `\n`, folded to a space). Everything else stands for syntax the render drops. Nothing real is lost: a match spanning `a**b**c` would have to contain the asterisks to exist at all, and a match wholly inside an element is still found — the element simply starts its own run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(search): match a Note body as it renders, instead of rewriting the file Replaces the serializer change with one that writes nothing. The editor backslash-escapes every markdown-significant character in prose, so a Note the reader sees as `{{TE_SERET}}` is stored as `{{TE\_SERET}}` and search — which matches the stored value — could not find it. The previous approach undid that escape in `postProcessSerializedMarkdown`, which meant re-deriving markdown structure from the serialized string with regexes so it knew what was code. That is a losing game: three review rounds, each finding another construct it did not model (longer fences, then delimiter-prefixed lines, then quoted fences), and each miss REWROTE somebody's code. `markdown-fidelity.ts` is back to staging, byte for byte. The escape is now undone on the matching side only. A field declares `searchTextFormat: 'markdown'` (the Note body is the only one), and the indexer matches it against `projectEscapedMarkdownForSearch(value)` — a total, structure-free function that returns the rendered text plus an index back into the source. Ranges stay in source coordinates, so replace still rewrites the whole `\_` and never strands a backslash. The asymmetry is the whole point: a matcher that de-escapes something a fence would have kept literal changes only which text highlights, and no caller writes it back. A rewriter making the identical mistake corrupts the file. So there is nothing here that needs to know about fences at all. Two consequences worth having: existing notes are searchable immediately rather than after their next edit, and no stored byte changes, so no document the editor has ever written can be affected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Waleed
|
a27f376164 |
feat(connectors): add Bitbucket, Databricks, Google Chat, and Workday Help KB connectors (#6895)
* feat(connectors): add Bitbucket, Databricks, Google Chat, and Workday Help KB connectors
Adds four knowledge base connectors, closing the gap where Sim shipped tool
blocks for these services but could not index their content.
- Bitbucket: repository source files and pull request descriptions over the
existing Bitbucket OAuth credential
- Databricks: notebooks (Workspace API) and saved SQL queries, PAT auth
- Google Chat: spaces indexed as message transcripts, new google-chat OAuth
service under the shared Google client
- Workday Help: knowledge article versions via the public helpArticle/v1 API
* fix(connectors): second-pass validation fixes and test coverage
Adversarial re-validation of all four connectors plus a combined-change
regression audit.
- bitbucket: stop declaring incremental sync (deletion reconciliation is
disabled for incremental runs, so deleted files were never removed on the
default code configuration); drop a wasted listing round-trip after the
frontier drains; add 33 tests
- databricks: reject an explicit maxDocuments of 0, which meant unlimited;
add 34 tests
- google-chat: correct the sender displayName documentation (user auth
populates only name and type) and emit second-precision RFC-3339 in the
message filter; 15 -> 24 tests
- workday: fix a crash when maxVersions is persisted as a number; refuse a
configuration whose status filter Workday did not honor; cap the
unresolved-name error; 18 -> 27 tests
- document the Google Chat service-account omission
- docs: list all four connectors and correct the connector count
* fix(connectors): index Google Chat spaces with no messages in the window
Review round 1.
- orderBy takes a full ordering expression, not a bare direction. The reference
documents the default as `createTime ASC`, so send `createTime DESC`; a bare
`DESC` either 400s every hydration or is ignored, which would make the cap keep
the oldest traffic and the later reverse render the transcript backwards.
- getDocument no longer returns null when the message window is empty. A space
with no messages is still a live space, and null is the "document is gone"
signal the engine treats as last-known-good: returning it dropped spaces whose
only prose is their description or guidelines, and left a stale transcript
indexed after a space was cleared or lookbackDays was tightened past every
message.
- The transcript header is omitted when no message contributed text.
* fix(connectors): only flag a Bitbucket listing capped when the cap withheld something
Review round 2.
takeIndexableWithinCap reports capReached as soon as the running total equals
maxItems, which is also true of a listing that ended at exactly that count.
Setting listingCapped there suppressed deletion reconciliation for a complete
listing, so upstream-deleted files and pull requests could stay in the knowledge
base indefinitely. applyMaxItemsCap now takes whether Bitbucket had more content
beyond the page -- a next link, or directories still queued on the frontier --
and flags the listing only when the cap actually withheld something, matching the
Databricks, Google Chat, and Workday connectors.
* fix(connectors): keep the Bitbucket cap flag set when it skips the pull request phase
Review round 3. Fixes a regression from
|
||
|
|
d9cfd7c68e |
improvement(mothership): v0.9 (#6815)
* checkpoint
* Checkpoint
* dot fixes
* Make async tool resume delivery recoverable
* Support split table tools and option recovery
* Harden VFS mutation handling
* feat(platform): platform subagent support — docs corpus VFS, search_docs, account context
Squash of the feat/platform-agent branch (sim side): mounts the Sim docs
corpus in the copilot VFS, wires search_docs and retires the legacy docs
search tools, and syncs the generated tool catalog and trace contracts for
the platform subagent.
* Align Copilot tools and resource handling
* Expand workflow log query support
* checkpoint
* Port desktop-improvements-0 desktop and browser-agent work
* fix(desktop): keep browser-agent input alive through live-SPA re-renders
* Harden workflow sanitization and Slack setup
* feat(desktop): coordinate clicks, caret insertion, and drag for the browser agent
* Revert subagent group eager auto-collapse
* fix(chat): keep sends FIFO across the streaming-to-idle drain gap
* Add the steering backend surface for mid-turn sends
* Sync generated contracts for async subagent orchestration
Pulls the mothership tool catalog (wait_agents / tail_agent / steer_agent /
interrupt_agent), trace spans (chat.async_subagent.*, chat.orchestrate.*), and
trace attributes (copilot.async_subagent.*) into the generated TS contracts.
* Add display titles for the async subagent orchestration tools
wait_agents / tail_agent / steer_agent / interrupt_agent get natural-language
running titles (naming the agent id being waited on, tailed, steered, or
stopped) and a Steering→Steered completed-verb rewrite.
* Show orchestrator-chosen subagent names on agent groups
A subagent_start whose payload data carries a name (the orchestrator's new
name trigger parameter) now labels the agent group with that mission name —
the agent-type icon stays. The name flows through the live stream path, the
turn model (AgentNode.displayName) and its serialize/rebuild round-trip, and
persisted transcripts (PersistedContentBlock.name), so reloads keep the label.
* Improve Copilot error handling and logging
* Backfill the subagent display name from the second start event
The dispatch-time subagent_start fires before the trigger args (and therefore
the name parameter) have streamed; the phase-3 start re-announces the lane with
the name. The block builder was dropping that duplicate wholesale, losing the
name on streaming providers — now it backfills subagentName onto the existing
block instead. (The home turn-model path already reconciled this case.)
* Support Slack bot connection flow
* Harden Copilot error and VFS handling
* Harden VFS resource operations
* Show 'Waiting for the first of N agents' for mode-any waits
The wait_agents title ignored the mode argument, so an any-mode wait over
three agents read 'Waiting for 3 agents' while the model narrated waiting for
the first — contradicting the transcript.
* Collapsed-by-default agent cards with live intent status lines
Subagents now narrate their work through <intent>3-5 words</intent> tags (a
fleet-wide prompt protocol on the mothership side). The turn model streams
each subagent's text through a split-safe tag parser: complete tags update the
agent's currentIntent and disappear from the prose, tags split across deltas
are carried until their close arrives, and a tag that never closes flushes
back as plain text.
The agent card renders as one line — display name (or agent label) plus the
latest intent, replaced inline as the agent shifts gears — and never
auto-expands; expanding to the full tool log is a deliberate click. Only an
outstanding permission prompt or a browser hand-back forces a group open.
Intents persist on the subagent block (and through the legacy persisted-
message paths) so reloads keep the last status, and a renamed reinvocation
now takes the latest name instead of pinning the first.
* Add the internal in-band tool execution route for live mothership turns
POST /api/copilot/tools/execute (INTERNAL_API_SECRET, Go→Sim) runs one
sim-server tool through the same server tool router the resume driver uses
and returns the result synchronously — no checkpoint. This is what lets
background (async) subagents write files/tables/knowledge, and lets the main
lane keep streaming (instead of checkpoint-pausing and killing every
background run) while async agents are live.
* Persist resource side effects for in-band tool execution
Files/tables created through the internal execute route now register on the
chat's resources exactly like the resume driver's executions — the route runs
the same handleResourceSideEffects pass (persistence only; an out-of-band
route has no live event sink, so mid-turn chip pushes are a follow-up).
* Extract intents from group text on every path, sync and async
The turn-model intent filter only fires for span-scoped subagent lanes, but
this surface also delivers subagent text through the legacy block path — so
<intent> tags flowed through unparsed and rendered as prose rows. Groups now
extract intents from their accumulated text at append time: the last complete
tag becomes the card's status line and every complete tag is stripped from
the rendered prose. Covers span-scoped, legacy, and persisted-reload paths
for both synchronous and background delegations.
* Fall back to the live tool title for the agent card status line
Persisted data proved tool-first subagents (grok search agents) emit zero
prose, so intent tags never stream no matter what the prompt says. The
collapsed card now always narrates: the agent's own <intent> tag when present,
else the latest tool's display title while the lane is live.
* Catch subagent <intent> tags in the server relay
The relay's subagent text handler now runs the split-safe intent extraction
as chunks stream: the latest complete tag is stamped onto the lane's persisted
subagent block (subagentIntent) and stripped from the stored prose, so live,
persisted, and replayed views all agree. Per-lane carry handles tags split
across chunks; a never-closing tag flushes back as plain text.
* Drop the tool-title fallback: the status line is the agent's intent
With the intent protocol now injected into every spawn's task message, agents
open with an <intent> tag; the card shows that narration or nothing.
* Replace intents with live tool-title status lines on agent cards
Intent parsing is fully removed (turn model, relay handler, persistence
fields, group extraction). The collapsed card's status is the latest tool
call in its RUNNING phrasing — never the completed rewrite, which stays in
the expanded log. Parallel tools show the most recently started still-running
title with a +N for concurrent siblings; between rounds the last title stays
frozen; a closed lane shows the bare name. Nested agent cards compute their
own status recursively from their own items.
* Keep the main Sim lane live-expanded; collapse only real subagent cards
The mothership group is the turn's own narration, not a delegation card —
collapsing it hid main-lane text and tools until manual expand, which read as
mis-ordered streaming while async subagents interleaved. It keeps the
original live-expand behavior and no status suffix.
* Persist subagent lane lifecycle blocks from the span handler
Lane-scoped span events route to the span handler, which only recorded trace
side effects — no subagent start block was ever persisted (verified: a
seven-agent run stored 104 blocks with zero starts). Grouping then fell back
to keying lane content by agent NAME, so a respawned agent of the same type
merged invisibly into the first one's card until it resolved. The handler now
persists the start block (spanId-keyed and deduped, carrying the display
name) and stamps endedAt on close, giving every invocation its own card.
* Name agents in orchestration titles; '+ n more' overflow format
wait/tail/steer/interrupt titles humanize the slugified agent ids back to
their display names ('Waiting for the first of Digest Workflow Build + 4
more'), and the agent card's parallel-tool suffix uses the same '+ n more'
format.
* Harden in-band tool execution and resources
* Route in-band execution through the comprehensive tool dispatcher
The internal execute route used the bare server-tool router, which rejects
VFS tools with 'Unknown server tool: read/glob/grep' — so nearly every
background agent's first discovery call failed (102 in-band calls in one run,
dozens rejected). It now uses the relay's executeTool dispatcher: registered
handlers (VFS, function execute) with permission checks and param
normalization, falling back to the app tool router — the same surface
foreground execution gets.
* Harden chat stream transition handling
* Harden VFS provenance and resource writes
* Standardize tool environment references
* Harden browser panel and chat cleanup
* Descriptive, user-language tool titles across the board
House rules applied everywhere: use every argument the call carries, never
name internal machinery, and never lead with Getting (the Got rewrite is
deleted so it cannot return).
- Deployments name the workflow: Deploying {workflow} as API/chat app/MCP tool
- Workflow reads name the part: Reading {workflow} meta/state/deployment/notes;
generic reads always name the file (Reading {leaf}), never bare Reading file
- Block runs name block and workflow: Running {block} in {workflow}, Running
from {block} in {workflow}, Running {workflow} until {block}, and
Enabling/Disabling {block} in {workflow}
- The six split-table tools get per-operation verbs (Adding column {name},
Updating rows, Wiring automation, Creating view {name}) instead of a wall
of Querying table
- The manage quartet drops X-action system-speak for gerunds
- get_* internal names become user language (Checking run settings, Tracing
block inputs, Reading the deployed version); web_fetch says Fetching
- Scheduled-task titles removed entirely (feature deleted from the Go catalog)
- New verb rewrites: Fetched, Traced, Wired, Configured, Looked, Rotated
* Deploying {workflow} as chat, not as chat app
* Loader gerunds; mv names both ends; mkdir names the folder
search_integration_tools -> Finding the right integration;
load_integration_tool -> Loading {integration} tools; load_skill ->
Loading skill {name}; run_enrichment -> Looking up {subject}. mv prefers the
model's phrasing, else reads 'Moving {files} to {destination}'; mkdir reads
'Creating folder {name}' from the path.
* Overflow counts read '+ n', dropping 'more'
* Unify workspace find and search
* Scale desktop title bar with page zoom
* Serialize account and organization truth into the copilot VFS
Workspace standing, membership, billing, org role, access-control
restrictions, published-block provenance, and fork topology were reachable
only through three parameterless tools (or not at all). They are ambient
read-only facts, so they belong in the VFS where they are greppable, cost no
tool round-trip, and every agent that can read gets them — the same move that
retired get_blocks_and_tools and list_user_workflows.
Adds account/{workspace,workspaces,members,billing}.json (always mounted) and
organization/{organization,access-control,custom-blocks,forks}.json (only when
the workspace is org-hosted). Every file projects an existing use case or util
after getOrMaterializeVFS's access assert — no new queries, no new
authorization. One relation per file, cross-referenced by id-and-name stub, so
overlapping facts cannot disagree. Volatile content (billing, access control,
forks) is lazy, so numbers are read-time fresh and unasked-for reads cost
nothing.
Projection follows the viewer: member emails are admin-only, fork detail
requires workspace admin on a forking-enabled org, and the whole organization/
namespace is absent for a personal workspace — which is itself the answer.
Retires get_account_billing, get_enterprise_context, and list_user_workspaces
along with their handlers; display titles stay for transcript replay.
* Fix insert_text refusing an editable field focused inside a frame
describeFocusedEditable descended shadow roots but not frames, while
activeElementReadback descends both. Focus inside a same-origin frame therefore
surfaced to the first as the FRAME element — not an input, not contentEditable,
not a canvas, no textbox role — so it fell through to 'not-editable' and
insert_text refused a field that press_key had just typed a character into.
Two functions answering 'what is focused' with different answers is the bug;
the descent loops now match exactly.
The refusal also names what actually held focus (tag, role, contenteditable).
A bare 'not-editable' gave the agent nothing to act on, so it guessed at the
cause — a real run spent twenty rounds on the wrong theory and had to be
stopped by the user.
* Keep retired browser takeover renderable in history
The tool is gone from the catalog, so its generated constant went with it and
every path that referenced it stopped compiling. Deleting those paths instead
would have silently downgraded every past transcript containing a takeover card
to a generic tool row, and dropped the no-timeout budget that an in-flight
takeover still needs while a rolling deploy finishes.
retired-tools.ts gives the literal a documented home that says what it is and
why it survives its tool.
* Follow the agent into a tab it opened to work in
browser_open_tab created the page with activate: false, so the agent worked in
a tab the user could not see while the panel sat on a page where nothing was
happening. The panel now follows a tab the agent deliberately opened.
Scoped to that tool only. A page spawning its own tab (popup, target=_blank) is
the site grabbing the view rather than the agent choosing a workspace, and
stays in the background as before — two existing tests pin that and caught the
first version of this change, which moved both.
A tab the user claimed still wins over both: the work starts in the background
instead of pulling the page out from under them mid-read.
* Make the browser tools agree with each other
An audit of the module found the frame-descent bug was one instance of a
pattern: six independent definitions of 'is this editable' and seven of 'what
is focused', disagreeing with each other. A tool refusing what its sibling
accepts on identical page state is invisible at runtime — the agent follows a
snapshot that says one thing into a tool that says another.
- browser_type now accepts role="textbox" like browser_insert_text does. The
snapshot advertises those elements as [textbox] with a ref, so refusing them
meant rejecting exactly what the outline told the model to type into. Both
the native and synthetic paths, and their descendant scans.
- pressKeyOnPage descends shadow roots and frames like every other focus
reader. It was dispatching synthetic keys at the shadow host or <iframe>
element, where they bubble but never reach the editor, while reporting
success — and contradicting the activeElement reported beside it.
- not-editable and ambiguous-editable name what was found: the element's tag
and role, and the candidate fields. Both had the data and discarded it, which
is what turns one blocked step into twenty rounds of guessing.
- obstructedAfterNavigation requires a dialog that ARRIVED with the
navigation. It compared against nothing, so every SPA route change under a
persistent role=dialog reported a successful click as obstructed. The test
that covered this asserted the false positive; it now pins both directions.
- browser_insert_text observes the top document when typing inside a frame,
like every other input tool. A submit that navigates the top page was
invisible to its frame-scoped observation.
* Let hover actually see what it mounted
Four independent defects made browser_hover blind to the most common thing a
hover produces — a row's action bar — so it reported no effect on a hover that
worked, and the agent fell back to clicking pixels off screenshots.
- The popup scan matched only role=tooltip/menu/listbox. Slack's message
shortcuts bar is a labelled toolbar/group, so it registered as nothing at
all. Added toolbar, menubar, labelled group, and [popover].
- The baseline was captured BEFORE prepareElementSurface scrolled the target
into view, so scrollChanged was always set by the tool's own probe. That
pinned every unproductive hover to 'background DOM churn' instead of the
honest 'nothing happened', and hid scrolling the hover really caused.
Re-baselined once the scroll settles and before the pointer moves.
- The MutationObserver attached only on the first observation, while the roots
list is rebuilt every call and grows as shadow roots mount. Components that
appeared later were never observed, so their DOM changes raised no revision.
Roots are now observed as they show up.
- observationTruncated was computed and never read, so a scan capped at 12k
nodes reported 'nothing appeared' with the same confidence as a complete
one — and portalled overlays live at the end of <body>, exactly what the cap
drops. Hover now says the page was too large to scan and to confirm visually.
* Stop the browser agent acting on the wrong element, and say why it refused
Four findings from the module audit, the first of which could silently do the
wrong thing rather than merely fail.
- A ref whose node is gone is re-adopted by structural resemblance, matching on
ORIGIN only so a pushState between snapshot and act does not kill every ref.
That leniency also let a ref to a row control in one view rebind to the
identical control in a view the app had since navigated to — acting on the
wrong message, signalled by nothing louder than recovered: true. Adoption now
requires the same path; a view swap reports the ref stale, and the caller
re-snapshots. Revalidating a still-connected node stays lenient, because that
is literally the node the model chose.
- A hit INSIDE the requested element is its own nested control, not an overlay.
Both produced 'covered by X — close or move the overlay', advice that cannot
be followed because there is nothing to close. Nested hits now say so and
point at retargeting.
- browser_click_at, browser_insert_text, and browser_drag listed targetChanged
in their effect formulas, but none passes an elementId, so no targetState is
ever captured and the term was always false — coverage that read as real.
Removed, with a test pinning the dependency.
- The seven effect formulas are deliberately NOT collapsed into one predicate:
drag must trust domChanged where others must not, hover must ignore field and
focus changes, click counts focus only for editables. Forcing one would make
each tool wrong differently. The differences are now documented in one place
next to the shared computation, so divergence is a declared policy rather
than an accident.
* Let edit_workflow configure block retries
* Updates
* Always focus the resource the agent is working on, and its browser tab
The resource panel had a carve-out: an already-open browser session declined
to replace another selection and only got an attention marker, so agent
browser work happened off-screen. The panel now follows the agent to whatever
it touches — browser included — and an event can still opt out explicitly.
The browser panel also follows the agent BETWEEN tabs: the store already
tracked automationTabId (and the strip marked it), but the visible tab never
changed. It now switches when the agent's target tab changes, so watching the
agent never means hunting for the tab it moved to. Keyed on the target
changing rather than on it being set, so a user who browses elsewhere
mid-run is only pulled along when the agent itself moves.
* Never paint a browser snapshot at stale geometry (the modal-open flash)
Opening a modal locks scroll, which removes the window scrollbar and reflows
the panel — so a capture taken before the lock describes a rect the panel no
longer occupies. The handshake painted that frame anyway and only then
retried, so the replacement landed visibly offset from the page it stands in
for: the flash. A capture is now checked against the host's live rect before
it is painted; a mismatched frame is skipped and re-captured at the settled
layout instead (modal retries go 2 -> 3 to absorb the extra settle).
* Name the workflow in deployment and workflow-scoped tool titles
'Checked deployment status' never said which workflow — nor did the deployed-
state read, run settings, block outputs/inputs, redeploy, promote, or the
global-variable write. These tools carry a workflowId (often defaulting to the
current workflow), so only the client can resolve a name: the enrichment layer
now resolves it for the whole workflow-scoped family and passes it as
workflowName, which every workflow title already reads.
Titles: Checking {workflow} deployment status, Reading deployed {workflow},
Checking {workflow} run settings, Reading {workflow} block outputs, Tracing
{workflow} block inputs, Redeploying {workflow}, Promoting {workflow} version
{n} to live, and 'Adding workflow variable {name} in {workflow}' — each
falling back to its unnamed form when no workflow resolves.
* Name the block that ran; never fall back to a raw block id
run_block and set_block_enabled carry only a blockId, so their titles would
have printed an opaque UUID ('Running 7f3a2b91-… in Invoice Sync'). The
enrichment layer now resolves blockId against the workflow store the same way
it already did for run_from_block's startBlockId, and the base titles no
longer accept an id as a name — an unresolved block reads 'Running block'
rather than a UUID.
* Add the missing Removing -> Removed rewrite
The table work introduced 'Removing automation'/'Removing enrichment' with no
past form, so those rows kept their present tense after completing.
* Name the target resource in the remaining tool titles
Table tools keep their operands nested under args and identify the table by
id, so their rows said 'Adding rows' with no hint where: enrichment now lifts
the nested args and resolves tableId against the cached workspace table list,
giving 'Adding rows to Runtimes', 'Adding column status in Runtimes',
'Reading views of Runtimes'.
Also: a block-schema read names the block instead of the file ('Loading
Slack', 'Loading Google Sheets tips'); browser type/insert show the text they
send, middle-ellipsized; downloads name the file; library-docs searches name
the library and query; knowledge-base searches include the query; generated
media names its output file; and diff_workflows, list_deployment_versions,
and publish_custom_block joined the workflow-name enrichment set.
* Bubble nested agents' tool calls into the parent's status line
The collapsed status only scanned a group's OWN tool items and skipped nested
agent groups, so a parent that had delegated froze on its last own tool while
its child did the actual work — the line described nothing that was running.
Status now walks the whole subtree: any tool at any depth counts, the most
recently started running one is shown, and the rest become the same '+ n'
overflow. With nothing running it falls back to the last tool at any depth,
so an idle parent still reflects where its subtree got to.
* Align nested tool call status rows
* Revert branch-local KB connector error-message edits
Restores apps/sim/connectors/ to staging state. Two copilot-focused commits
on this branch (
|
||
|
Vikhyath MondretiandClaude Opus 5
|
97c1688c49 |
feat(modal): add Modal Labs integration (#6896)
* feat(modal): add Modal Labs integration Modal has no public REST control plane — the Python/JS/Go SDKs all speak gRPC — so this covers the two surfaces that are reachable over HTTP: deployed Web Functions/Servers, and the OpenAI-compatible Endpoints API. Three operations: call a deployed function with proxy-token auth, generate a chat completion on an Endpoint, and list the models a token can reach. Auth sends the token pair as Modal-Key/Modal-Secret rather than the combined bearer form, so a Web Function that validates its own bearer token keeps the Authorization header free. Both URL fields require https since Modal terminates TLS everywhere, and a cleartext URL would leak the token. Chat completion declares request.modelInput so the system prompt and user message project to canonical placeholders before egress. Call Function deliberately does not — a Web Function runs arbitrary user code, and nothing proves its body reaches a model. /v1/models fields beyond `id` are inferred from OpenAI compatibility rather than printed in Modal's docs, so they are marked optional. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(modal): type the wire payloads and default chat to the shared endpoint Chat Completion required an endpoint URL and passed a blank one straight into modalOpenAiUrl, which throws — while List Models already fell back to the shared inference host and the generate-on-modal-endpoint skill tells agents to leave the field empty for Shared Endpoints. Skill-driven chat calls against the shared host failed instead of using that default. Chat now falls back the same way and the block field is no longer required. Replaces every `any` in the Modal tools with declared wire types for the OpenAI-compatible /v1 payloads. Fields stay optional because the shape comes from whichever inference engine backs the endpoint, so the readers keep their defensive `??` guards — the types exist so a future change to that mapping fails the compiler instead of shipping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
3b4d9e9149 |
fix(schedule): reconcile interrupted schedule executions (#6780)
* fix schedule execution recovery * simplify schedule recovery provider handling * fix(schedules): stop reconciliation rewriting healthy carriers and index its scan Follow-up hardening on the schedule recovery reconciliation. Reconciliation settled every carrier it examined, including ones that had already reached a terminal status and recorded their own outcome. Because the normal completion path never stamps the reconciled marker, each successful run's carrier was picked up on the next tick and rewritten: `completedAt` bumped, `error` nulled and `output` replaced with a recovery stub, all of which `GET /api/jobs/[jobId]` surfaces. A completed carrier whose execution log had aged out was additionally flipped to failed. Settle only carriers still in flight; a terminal one is owed schedule accounting and the marker, nothing more. The recovery scan matched no index. There was none on `async_jobs.updated_at`, and the unreconciled-terminal branch tested a jsonb extraction, so the whole OR fell back to a sequential scan and sort of `async_jobs` on every tick. Add the partial index, and spell the branch's status list and metadata key as SQL literals: Postgres cannot prove a parameterised predicate implies a literal index predicate, so a bound key would have left the new index unused. Irrecoverable carrier tombstones were exempted from retention with no secondary expiry, so the one class of row that can never reconcile grew without bound. Give them a longer bounded window instead. `WORKFLOW_EXECUTION_MAX_ROWS_PER_RUN` had become a per-status budget when the stale-execution sweep gained its `redacting` pass, silently doubling the per-run cap. Share the budget across both passes, and add the matching partial indexes so the second pass keeps an index rather than seq-scanning. Also consolidates the carrier metadata keys, their predicates and the jsonb merge into one module -- the two routes were the writer and reader of the same keys with no shared symbol, so a rename type-checked clean while silently breaking retention. * fix(schedules): rotate deferred carriers and stop the redacting sweep failing live runs Addresses the review findings on the reconciliation hardening. A carrier whose accounting was deferred got no write at all, so it kept its `updatedAt` and stayed at the head of the `updatedAt`-ordered recovery batch on every tick, starving every other claimed carrier and never becoming eligible for retention. This was a regression from the previous commit: settling used to bump the timestamp for every examined row, and skipping the settle for already-terminal carriers removed the bump with it. A payload with no `scheduledFor` can never reconcile, so such a row pinned a batch slot permanently. Bump `updatedAt` unconditionally and keep only the reconciled marker conditional. The stale-execution sweep terminalized `redacting` logs on the execution deadline. That deadline bounds execution, while `redacting` covers payload masking after the run already finished -- so a run that used most of its budget entered redaction with the deadline due, and the sweep failed it five minutes later while the worker was still masking. Schedule recovery then read the log as a failed occurrence and counted a failure that never happened, even though the worker's terminal write later restored `completed`. Sweep `redacting` on the generic stale window only. `getScheduleNextRunAt` falls back to a daily cadence when a schedule has no cron expression. Deployment cannot persist such a schedule, so the branch is unreachable, but this change widened its use from failure recovery to every outcome -- log a warning when it fires rather than silently guessing a cadence. `executionDeadlineAt` was missing from the shared `workflowExecutionLogs` schema mock, so it read as `undefined` and assertions comparing against that column were trivially true. Add it. * refactor(schedules): drop vestigial recovery code and close a builder gap Follow-ups from a full re-read of the change. No behavior change except the removed dead code paths. The metadata merge stripped a `scheduleRecoveryBlocked` key on every write. That key has never been written by any shipped code -- it appears nowhere in staging and nowhere in history outside this branch -- so the strip guarded against a state that cannot exist, at the cost of an extra jsonb operation and a bind parameter on every reconciliation write. `processScheduleItem` set `carrierObservedOrLookupUncertain` immediately before returning on an ambiguous enqueue. The flag is only read from the surrounding catch block, which a normal return skips, so the assignment was dead and read as though it were load-bearing. Replaced with a comment stating why the occurrence is preserved. The stale-execution sweep carried two near-identical `jsonb_set` templates that differed only in their error expression, kept flat because the test mock renders nested SQL fragments as placeholders. The suite now has a recursive renderer, so the error expression is a named per-status value and there is one `jsonb_set`. Success was the only schedule outcome without a named update builder, which left `executeScheduleJob` using two idioms for the same guarded write and left the update shape untested. Add `buildScheduleSuccessUpdate` beside its cancellation and failure siblings, use it from both call sites, and cover it the way `buildScheduleCancellationUpdate` is covered -- a mutation of its `failedCount` reset previously passed every suite. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Justin BlumencranzandClaude Fable 5
|
38630ff46d |
feat(tables): autosave persisted default views (#6724)
* feat(tables): autosave persisted default views * fix(tables): close persisted view lifecycle gaps * fix(tables): preserve edits through view hydration * fix(tables): preserve the persisted default owner * fix(tables): preserve legacy layout through view adoption * fix(tables): close final view autosave races * fix(tables): preserve valid saved sort on stale links * fix(tables): make the first saved view the default Creating the first view left isDefault false, so the legacy All fallback stayed in the menu instead of handing off to the new view. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tables): refuse to delete a table's last saved view The sibling check and the delete share the views advisory lock, so racing deletes cannot drop a live table to zero views and regress it to the legacy "All"-only state. Views of a hard-deleted table are removed by the FK cascade, which this guard never sees. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Theodore Li
|
4214a891f4 |
fix(setup): publish unscoped setup package (#6886)
* fix(setup): publish unscoped setup package * fix(setup): strip renamed status command |
||
|
Waleed
|
f6a9f0dd87 |
fix(integrations): white CB Insights tile and a borderless Crunchbase mark (#6884)
CB Insights moves from a dark navy tile to white, matching Jira, Confluence, and Bitbucket. Its icon carries its own fills, so it stays legible on the lighter tile. The Crunchbase icon drops the white rounded-square plate and its border, leaving just the `cb` mark on `currentColor` so the block's bgColor supplies the tile. The viewBox is retargeted to the glyph's true curve extrema with padding that keeps it at the same optical weight as the surrounding brand marks. |
||
|
Justin Blumencranz
|
0b71717241 |
perf(icons): reduce and guard SVG path precision (#6839)
* perf(react): reduce SVG path precision * fix(react): preserve Sim wordmark precision * fix(icons): preserve Quartr scale * test(icons): ratchet SVG path precision * fix(icons): make precision exceptions local * perf(icons): enforce three-decimal paths |
||
|
|
e3a4874ece |
feat(integrations): add Bitbucket Cloud (#6860)
* feat(integrations): add Bitbucket Cloud * fix(bitbucket): enforce selector workspace slugs * fix(bitbucket): overfetch small pipeline log tails * fix(bitbucket): harden provider edge cases * fix(bitbucket): accept provider diff redirect specs * fix(bitbucket): stop advanced-field leakage and harden log, status, and selector paths Splits the `closeSourceBranch` advanced subBlock into per-operation ids. Advanced fields serialize without evaluating their condition, so a value set on Create Pull Request reached Merge Pull Request and closed the source branch unprompted. Also: - read step logs through the byte-capped server transport and map an empty-log 416 to an empty result, keeping a genuine 416 an error - trim a step log's partial leading line after the character cap rather than before, and never return an empty log when the retained window held content - surface Bitbucket's `error.detail` alongside `error.message` - treat commit-status `key`/`state` as nullable so one malformed row cannot drop a page - match repository `full_name` case-insensitively and reject dot segments in a workspace slug before the outbound request - type `reviewerAccountIds` as the comma-separated string it is - trim optional Bitbucket query strings; correct the token lifetime to two hours --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
Waleed
|
9f346765fe |
feat(granola): complete API coverage, note triggers, and connector validation (#6880)
* feat(granola): complete API coverage, note triggers, and validation fixes
Granola's public API exposes nine endpoints; Sim implemented three. Adds the
remaining six and wires the new programmatic webhook-endpoint lifecycle into a
managed trigger.
Tools (6 new, 9 total):
- get_transcript, list_audit_events
- create/list/update/delete_webhook_endpoint
Triggers: note.generated, note.edited, note.access_granted, plus an all-events
trigger. The provider handler registers the Granola endpoint on deploy and
deletes it on undeploy, scoped to the trigger's own event names, and verifies
every delivery with the Standard Webhooks HMAC-SHA256 signature Granola returns
on creation. event_id is the idempotency key, which Granola reuses across
retries.
Validation fixes to the shipped tools:
- get_note dropped speaker.attribution ("me"/"them"); now surfaced
- a 413 on get_note now explains that the transcript is too large inline and
points at get_transcript, instead of surfacing a bare status code
- note IDs are URL-encoded rather than interpolated raw
- base URL, auth headers, and status-aware error handling are shared runtime
helpers; params/outputs stay literal per file so the docs generator still
reads them
Tests cover signature verification (including replay and body-tamper
rejection), event matching, subscription create/delete, and the block/tool
contract — plus a guard that ids shared between the tool and trigger surfaces
seed the same default, since block state is keyed by id and last-wins.
The knowledge-base connector was validated against the spec and needed no
changes.
* fix(granola): correct array output schemas, listing-truncation signal, and docs
Findings from validation passes over the tools, trigger, and connector.
Tools — array outputs were declared as `type: 'json'` with `properties`, which
describes an object, not an array. Agents and the output picker therefore saw
`notes.title` instead of `notes[i].title`. All 15 array outputs (including the
pre-existing three tools) now use `type: 'array'` with `items`, matching the
2000+ other tool files. The audit event `data` field stays `json`; it is
genuinely free-form per the spec.
Connector — `hasMore` was ANDed with the cursor, so a `hasMore: true` response
with no cursor was reported as a complete listing. The sync engine treats
exactly that shape as truncated and sets `listingTruncated` to block deletion
reconciliation; masking it meant a partial first page could be taken for the
whole corpus and reconciliation would hard-delete every note past it. Granola
would have to violate its own contract to emit that shape, but the engine
already handles it and the connector was hiding the signal. Also aligns
mimeType with the `.txt`/text-plain bytes the engine actually writes (it was
the only connector of 101 claiming text/markdown).
Trigger — the setup instructions named a Granola settings path that does not
exist; the help center says Settings > Connectors > API keys in the desktop app.
Both list parsers now split commas inside array entries, so an array-wrapped
free-text value cannot be sent as one malformed identifier.
Block — `id`, `events`, and `hasMore` are produced by several operations but
their descriptions named only one, unlike `folders` which already documented
both meanings.
Adds connector tests pinning all four listingCapped quadrants and the
truncation signal, and tool tests for the list parser and the PATCH body's
per-field "omit means unchanged" semantics.
* fix(granola): clean up webhook endpoints created by a failed registration
Raised independently by both reviewers. The registration service only rolls
external state back when createSubscription *returns* — its rollback is guarded
on `preparedProviderConfig`, so a handler that throws is assumed to have left
nothing behind. Granola's handler broke that contract: when Granola accepted the
POST but the success body was missing `id` or `signing_secret` (including a body
that failed to parse and became `{}`), it threw with the endpoint already live.
Nothing then recorded an external id, so undeploy could not remove it, and
Granola kept delivering to a callback whose signature could never be verified —
duplicating on every deploy retry.
The handler now removes what it created before rethrowing, matching the pattern
grain's multi-hook create already uses. It deletes by id when Granola returned
one, and otherwise recovers the endpoint by matching the callback URL, which
also covers a connection that fails after the request reached Granola.
Endpoints whose URL was redacted to its origin are never matched — that
comparison could delete another workflow's endpoint on the same host. Cleanup is
best effort and never masks the original failure. A non-2xx is left alone, since
no endpoint was created.
Also folds the delete call shared with deleteSubscription into one helper.
* fix(granola): never recover an orphaned endpoint by callback URL
The previous commit's URL-based recovery was unsafe. A redeploy reuses the live
registration's `path`, so the candidate and the currently serving endpoint share
a callback URL — listing by that URL and deleting every match would remove the
live deployment's endpoint and silently stop a working trigger, which is worse
than the leak it was trying to prevent.
Cleanup is now keyed solely on the id Granola returned. When the success body
carries no id there is no way to tell the candidate's endpoint from the live
one, so it is left in place: a leaked endpoint produces unverifiable deliveries
that Granola disables on its own, whereas deleting the wrong one takes down live
traffic with no signal.
The 2xx-missing-signing-secret case this originally fixed still cleans up, since
that response does carry an id.
Adds a test asserting no lookup or delete is attempted when the response has no
id, so URL matching cannot be reintroduced unnoticed.
|
||
|
Waleed
|
02ae2b4c33 |
feat(sidebar): add Tables and Files flyouts to the collapsed rail (#6882)
* feat(sidebar): add Tables and Files flyouts to the collapsed rail Chats and Workflows already open a hover flyout on the collapsed rail; Tables and Files were plain links. Both now list their contents, with folders as submenus and the open resource marked. The chip stays a real link, so clicking still opens the list page and right-click still reaches the nav context menu. Each flyout owns its queries and mounts only when the menu opens: a hook on the sidebar keeps its cache subscription on every workspace route even when disabled, so an unrelated writer would re-render the whole sidebar for a closed flyout. Rows are ordered by the shared sortResources, so pinned rows float and the flyout reads in the same order as the page it links into. Also removes two dead components (CollapsedFileFolderItems, FileList) that were exported but never rendered, and extracts SidebarNavChip so the rail chip has one definition. * fix(emcn): stop ordinary menus scrolling at the shared height cap Every DropdownMenuContent was capped at a flat 240px. A menu is 28px per row, 13px per separator, plus 12px padding, so a 7-row action menu with 3 separators measures 247px and scrolled for 7px while the 7-row menu beside it with 1 separator did not. Raises the cap to 420px, which clears every hand-authored action menu, and clamps it with min() against the space Radix measures so a menu near a viewport edge stays on screen — which the flat value never did. The cap still exists so a long data-driven list scrolls instead of running the height of the screen. * fix(sidebar): hold the rail flyout until its lists resolve for this workspace Both the resource and folder queries keep the previous workspace's rows as placeholder data across a switch. Gating only on isPending let the flyout build a tree from one workspace's resources against another's folders, where no folder id resolves — which the builder reads as "archived out from under it" and files the whole list at the root. Gate on isPlaceholderData too, matching foldersResolved in use-folder-ancestors. An error settles a query without resolving it and is deliberately not held: the flyout then renders flat, which still reaches every row. * improvement(sidebar): mark pinned rows in the rail flyout The flyout sorts pinned rows to the top via the shared sortResources, but rendered no indicator, so that ordering read as arbitrary — the exact pairing Resource's own label cell documents. Carry `pinned` on each row and render the same non-interactive glyph, on folders as well as resources. Adds folder-structure coverage alongside it: per-level ordering, the full depth of a nested chain, and an empty folder staying in the tree. |
||
|
Waleed
|
f17938c09e |
feat(cbinsights): add CB Insights API v2 integration (#6879)
* feat(cbinsights): add CB Insights API v2 integration Covers every non-streaming v2 endpoint across 25 tools: free organization lookup, firmographics search, funding rounds and cap tables, investments, portfolio exits, business relationships, management and board, the Mosaic / Commercial Maturity / Exit Probability outlooks and their histories, funding windows, revenue, strategy maps, Scouting Reports, ChatCBI, and RAG context. CB Insights authorizes by client-credential exchange rather than a static key, so the tools run through directExecution: the shared executor trades the credentials for a bearer token, caches it briefly, and re-authorizes once on a 401 — the token lifetime is undocumented, so expiry is discovered rather than predicted. ChatCBI and RAG declare request.modelInput so an activated Sim secret in the message is projected to its canonical label before reaching a third party's model. directExecution still runs projectToolModelInputParams, so the two are compatible. The two streaming endpoints are deliberately excluded; they deliver incremental JSON chunks and their non-streaming counterparts return the same content in one piece. * fix(cbinsights): reject malformed ID lists and bound the token cache - Reject an organization ID list containing an invalid entry instead of dropping it. Silently filtering meant a typo ran the request against a narrower set — spending credits on the wrong organizations, or quietly widening a filtered search — and still reported success. - Apply the same rule to the optional firmographics ID filters, where a dropped filter broadens the search rather than narrowing it. - Bound the process-wide token cache so a long-lived worker serving many CB Insights accounts does not grow with the cumulative number of accounts seen. Expired entries are swept on write, then the oldest evicted. * fix(cbinsights): stop paging and blank input bypassing the search guards - Measure the firmographics empty-search guard against the filters alone. limit, nextPageToken, and sort were in the same object, so a request carrying only paging slipped past it and issued an unfiltered search over the whole database — which still spends credits. - Reject a mistyped numeric bound instead of dropping it. A bad headcount, funding, or valuation filter silently widened the search, the same failure mode already fixed for ID lists. - Treat an empty comma segment identically on the required and optional paths. A trailing or doubled comma is a separator artifact that cannot change which records are requested, so both paths now discard it; every other malformed entry is still rejected. * fix(cbinsights): accept only plain decimal organization IDs Number reads "0x10" as 16 and "1e2" as 100, so either notation resolved to a real but unintended organization and the request spent credits on it. Both the path-scoped and the bulk validators now require a plain run of digits, and use Number.isSafeInteger so an ID past the precision limit cannot round to a neighbouring one. * fix(cbinsights): bound a numeric organization ID to the safe-integer range The string path already required a safe integer; the numeric path still used Number.isInteger, which accepts a value past the precision limit. JSON parsing has already rounded such a value, so the request would target a different organization than the caller supplied. |
||
|
Justin Blumencranz
|
8aab6d573e | chore: remove confirmed unused files (#6863) | ||
|
Waleed
|
a9cf760c0f | feat(pitchbook): add PitchBook integration (#6876) | ||
|
Waleed
|
40aa8ad5eb |
feat(crunchbase): add Crunchbase Data API integration (#6875)
* feat(crunchbase): add Crunchbase Data API integration Covers the v4 Data API end to end: dedicated search and lookup operations for organizations, people, funding rounds, and acquisitions, plus generic collection-parameterized search and lookup reaching the remaining 39 collections, single-card paging, autocomplete, the deleted-entity feed, and fields metadata. Adds a crunchbase-errors extractor: the API answers failures with a bare JSON array, which no existing extractor reads, so an auth or predicate failure would have reported only its HTTP status. * fix(crunchbase): honor card paging limits and cursor exclusivity - Cap a card page at the documented 100-item maximum instead of Search's 1000, which the shared Limit field made easy to carry over - Always request the card's identifier so a narrowed cardFieldIds cannot return a full page with a null cursor and stall a paging loop - Reject the mutually-exclusive afterId/beforeId pair on the card and deleted-entity endpoints, not just on search - Report an unexpected card shape as empty rather than wrapping the envelope as a one-row page |
||
|
Theodore Li
|
1372977d07 |
feat(setup): publish standalone self-hosting package (#6849)
* feat(setup): publish standalone self-hosting package * fix(setup): refresh discovered compose installs * improvement(setup): unify repository command * fix(setup): harden standalone package launch * Update README.md * fix(setup): isolate standalone compose installs * fix(setup): restore default stopped installs |
||
|
Vikhyath Mondreti
|
fcea50deb0 |
fix(provenance): stop size limits silently dropping secret provenance (#6867)
* fix(provenance): stop size limits silently dropping secret provenance A bundle-selection cap counted cells rather than rows, so a 25-column table insert lost secret provenance for every row past 400 — the whole batch was stamped unknown with nothing logged. The same number lived in the sender, the runtime type guard, and the route contract. Consolidate every provenance limit into one definition: an 8MB serialized envelope and 10,000 distinct secrets. The pair had been copied into seven modules under fourteen names, and several copies had drifted into bounding inputs — rows, cells, files, chunks — rather than the envelope. Remove every limit that could refuse a legal payload, add write-side cause logging and a workspace-visible audit entry when a read proceeds on unrecorded provenance, and repair the existing unknown rows. * fix(provenance): close a repair race and stop memory double-reporting The repair matched sidecars by the id its page captured, so a provenance-aware write committing between the snapshot and the delete had its fresh exact sidecar removed and its marker cleared behind it — a secret-bearing row left reading as legacy. The delete now re-checks status, which under READ COMMITTED re-evaluates against the writer's committed row so it no longer matches. Walk the candidate set by keyset over row_id. A page whose rows were all repaired concurrently clears nothing, and terminating on "cleared nothing" ended the walk with the rest of the backlog untouched. Memory reported unrecorded provenance twice, and counted records even when the surface was enforced — auditing a fail-open read that had actually failed closed. * fix(provenance): take the repair's locks in the writer's order The repair deleted the sidecar and only then updated its parent row, while mutateTableRowsWithSecretProvenance locks user_table_rows up front and upserts the sidecar inside the same transaction. Opposite orders, so an overlapping write deadlocked and Postgres resolved it by aborting either the deployment or somebody's table write. Lock the parent first, in id order, matching lockTableRows. Holding that lock is also what makes the status re-check decisive rather than racy: the writer commits its sidecar and its marker under the same lock, so once it is held the write is either wholly done or has not begun. |
||
|
Vikhyath Mondreti
|
483ff12b6f |
feat(billing): align enterprise reporting periods (#6851)
* feat(billing): align enterprise reporting periods * fix(billing): harden enterprise reporting flow * fix(outbox): preserve handler compatibility * fix(billing): bound enterprise provisioning reads |
||
|
Vikhyath MondretiandClaude Opus 5
|
9ee1b81b41 |
feat(custom-blocks): join cross-workspace runs into the caller's trace, and map blocks per environment (#6857)
* feat(custom-blocks): join cross-workspace runs into the caller's trace, and map blocks per environment Teams that orchestrate work across workspaces have two gaps that keep them on HTTP blocks instead of custom blocks: they cannot see what a custom block actually did, and a forked environment silently keeps calling the environment it was forked from. Debugging. A custom block is an invocation boundary — a published block is org-wide, so its internals must not reach every consumer by default. The child already writes its own log row in the source workspace, correlated to the invoking run; the trace existed, it just was not joined. The parent's span now carries only the child's opaque execution id, and `hydrateChildTraces` joins the child's spans at READ time, after authorizing the person reading against the child's workspace. Authorization follows the viewer rather than a flag set at publish time, re-evaluates on every read, and needs no second copy of the spans. Each hop of a nested chain is authorized against its own workspace. Boundaries left unexpanded — no access, no data, past a cap — say so, because a childless boundary span otherwise renders exactly like a leaf and a partial trace reads as a complete one. Live runs stream too, gated on `liveTraceViewerUserId`, which only surfaces with a single known authenticated viewer set. Chat deployments stream through the same callbacks and their consumer may be anonymous, so anything that does not opt in keeps the boundary shut. Child spans handed to such a viewer are projected through the CHILD's session: the invoking run's registry knows nothing about the publisher's secrets, so projecting there would leave a source-owner credential unmasked. They reach the live stream and stop — `createSpanFromLog` still refuses to persist them, which is what keeps read-time hydration the single authorization point. Environments. A fork inherits its parent's organization and `custom_block` is keyed `(organization_id, type)`, so a uat fork resolved to the same row and ran the prod workflow. Custom blocks become a fork-mappable resource, keyed by BLOCK TYPE — the rule every kind follows: key by whatever the workflow references, as `file` does with storage keys and `env-var` with names. A custom block is the only resource referenced by the canvas block's own type rather than a sub-block value, so the rewrite gets its own channel. Unmapped blocks keep the source type, because a type cannot be emptied without deleting the node; they surface as unmapped and block the promote, which is what stops uat from quietly invoking prod. Same-named environment copies now carry their source workspace, so an Access Control allowlist decision between three identical "Invoice Parser" rows is no longer a guess. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(workspace-forking): let an explicit identity custom-block mapping resolve `remapForkBlockType` reported a mapping whose target equalled the source as unresolved, conflating "a mapping exists" with "the type string changed". Those are opposite states that produce an identical output `type`, and every caller uses the flag for the former — to decide whether the reference blocks a promote. The org-wide candidate list includes the source block, so binding an environment to the shared block is a normal pick. Under the old flag it raised `unmapped-custom-block` and refused the sync over a choice the user had explicitly made. The flag is now named `resolved` and reports mapping existence; whether the type moved is already visible from `type`. Reported by Cursor Bugbot on #6857. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(custom-blocks): keep a streamed child's spans and markers off the parent's log Two leaks in the live-stream path, both the same mistake: treating a channel as viewer-scoped when it is actually persisted, so gating the stream on an authorized viewer bought nothing. `childTraceSpans` rode the block output to reach the stream. `filterOutputForLog` only dropped a hidden key when the block's own config declared it `hiddenFromDisplay` — true of the workflow block, never of a custom block, whose outputs are publisher-curated. The source run's spans therefore persisted into the parent's `span.output`, readable by anyone with parent-workspace access and never re-checked by `hydrateChildTraces`. A globally hidden key is now dropped at the top level, not only when nested, and `extractDisplayOutput` strips it again so no other producer can reintroduce it. The fan-out also called the invoking run's `onBlockStart`/`onBlockComplete`, which are persist-then-emit composites: they write block names and I/O into the parent's LoggingSession before reaching the stream. Those markers are keyed by the parent execution and outlive the per-viewer check entirely. Custom-block children now go through `liveStreamCallbacks`, the raw emit-only pair, and fail closed when a surface supplies none. Same-workspace workflow children keep the composites — they belong to the same run and their markers are legitimately the parent's. Reported by Cursor Bugbot on #6857. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(custom-blocks): carry the emit-only stream sink into nested executions Routing custom-block events through `liveStreamCallbacks` forwarded the viewer id to the child but not the sink itself, so a nested hop cleared `canStreamCustomBlockToViewer` off the inherited id and then had nothing to stream through — `parentStreamSink` fell back to `{}` and live traces stopped at the first sub-executor. That hit a custom block nested inside a workflow block as readily as one inside another custom block. The sink now travels with the viewer id, and both are withheld together when streaming is not permitted. It is always the INHERITED chain, never `parentStreamSink`: for a same-workspace workflow block that is the persisting composite, so forwarding it would put a custom block nested inside one straight back onto the parent's progress markers — the leak the previous commit closed. Reported by Cursor Bugbot on #6857. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Waleed
|
4f9d5f33b0 |
improvement(search): search every folder, and document real API error bodies (#6861)
* improvement(search): search every folder, and document real API error bodies Search on Files, Tables, and Knowledge was ANDed with the open folder, so a query only ever matched that folder's direct children — and the query was not cleared when you entered a folder, filtering the folder you just opened down to the same matches. A non-empty query now searches the whole workspace, a Location column names each result's folder, and opening a folder ends the search. Also gives GET /api/v2/files a `recursive` flag, and replaces the single shared OpenAPI error example — which showed `BAD_REQUEST` under every status tab — with one real body per status. * fix(search): discard the search term on clear instead of masking it `useSearchFilterValue` returned the debounced term whenever the input was non-empty, so clearing only hid the settled needle. The mask lifted on the next keystroke while the debounce still held the pre-clear term — opening a folder and typing within the window searched the whole workspace for the query the user had just abandoned. A clear now resets the settled term rather than hiding it, adjusted during render so the reset is visible to the render that follows the clear. The initial state is seeded from the first value so a deep-linked `?search=` still filters on the first render. |
||
|
Vikhyath MondretiandClaude Opus 5
|
521348b529 |
feat(secrets): record which secrets each run resolves, and surface it per secret (#6823)
* feat(secrets): record which secrets each run resolves and surface it per secret
Redaction stops a value at a boundary but cannot stop code that never emits it —
a Function block can print a key one character at a time and nothing ever matches
the secret. That is undecidable in general, so this adds the other half of the
posture: attribution.
Every run now records which configured secrets it actually resolved, under whose
identity, through which surface (workflow, Sim agent, MCP). The data already
existed in ResolvedSecretTraceRegistry.addActiveEntry and was persisted only for
paused runs; this persists it for every terminal path.
Execution logs cannot answer this. They store the whole available encrypted
environment rather than what a run referenced, they evidence a secret only where
value-matching redaction happened to fire, and they expire under
logRetentionHours — while "who has touched this key" outlives any single run.
- secret_usage: per-UTC-day rollup keyed by workspace, secret, scope, owner,
source, workflow, actor. A one-minute schedule touching three secrets would
otherwise write thousands of rows a day, which is also why this is not
audit_log. workflow_id/actor_user_id use '' sentinels rather than null so the
unique key works on Postgres 14 without NULLS NOT DISTINCT, and are not FKs:
they are historical facts, and an onDelete would rewrite a key column.
- secret_owner_user_id is part of the key. Two people can hold a personal secret
under one name and a shared personal secret resolves for a caller who does not
own it, so name and scope alone do not identify a secret. It is NOT the actor:
a scheduled run resolves the workflow owner's personal slice under the
workspace's execution actor.
- Direct environment reads are now detected in JS (TypeScript AST), Python
(tokenizer-checked) and shell (quote/heredoc-scanned), so a secret read as
environmentVariables['K'] or $K enters the run's provenance instead of going
unredacted. Each detector prescans for names that are actually configured
secrets before paying for a lex or quote-frame pass.
- Copilot integration tool calls are covered: resolveCopilotEnvReferences
substitutes {{SECRET}} into user-only params, which is a real use.
- See usage lives behind a credential-admin gate, using the same predicate that
reveals the value; members get a disabled chip explaining why.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(audit): register the secret-usage route in the validation baseline
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): keep rollup metadata with its run, and stop shadowed bindings faking usage
Review round 1.
- record.ts: last_execution_id/last_trigger were assigned unconditionally while
last_used_at was chosen by greatest(), so two runs completing out of order split
one row between them — the newer run's timestamp beside the older run's execution
id, making "View log" open a run the row does not describe. Both are now guarded
on the timestamp actually advancing, so the row's metadata always belongs to the
run that owns its timestamp.
- javascript.ts: a local binding named environmentVariables (declaration, parameter,
destructured binding, or bare reassignment) made reads off the user's own object
look like mounted-secret reads. Any such binding now disables detection for the
file; the AST already had parent pointers, so this is a kind check during the
existing walk.
- python.ts: same class of bug with no parser available, so the rule is an allowlist
— every mention of the binding must be a literal subscript or .get(), otherwise
detection is off for the file. This also subsumes the cross-line attribute case
(other.\n environmentVariables['K']), which the previous space-and-tab look-behind
missed.
Under-reporting is the safe direction here: a trail that claims a use that never
happened is worse than one that misses a use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(db): format the generated migration snapshot
CI runs lint:check across every workspace; the drizzle-kit output in packages/db
had never been through biome, so the branch was green locally (where lint had
only been run inside apps/sim) and red on CI. Whitespace only — both files are
byte-for-byte identical once parsed, and drizzle-kit still reports no pending
schema diff against the reformatted snapshot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): detect every rebinding of the environment identifier, not just declarations
Review round 2. A bare `for (environmentVariables of rows)` has no declaration to
key off, so the previous check missed it and reads of the loop value were still
recorded as secret usage.
Rather than extend the hand-rolled node-kind list, this reuses the pair the same
file already applies to reject a placeholder in a write position:
isDeclarationIdentifier covers declarations, parameters, destructured bindings and
imports, and isWriteIdentifier covers every assignment operator, ++/--,
destructuring targets, and for-in / for-of initializers.
That also closes four forms neither the review nor the original check named:
logical (||=) and nullish (??=) assignment, and object and array destructuring
assignment. Six of the eight added cases fail against the previous check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): apply the rebinding rule to shell, and say when a run's log is gone
Review round 3, plus the docs that were left claiming the old behavior.
- shell.ts: a script that writes a configured name (API_KEY=local, export/local/
readonly, read, for, unset) expands its own value from that point on, not the
mounted secret, so recording it claimed a use that never happened. Every mention
of the name must now be a `$NAME` / `${NAME}` expansion, matching the allowlist
shape the Python detector already uses. Applied per name rather than per file:
JavaScript and Python shadow one object holding every secret, whereas rebinding
one shell variable says nothing about the rest.
- The usage trail deliberately outlives execution logs, so a row routinely names a
run whose log has been pruned. The read now left-joins workflow_execution_logs on
its unique execution_id and reports availability, and the panel renders the chip
disabled with the platform tooltip instead of linking into an empty Logs view.
Three states: no run to link, a run whose log is gone, and a live link.
- Docs said a direct environmentVariables/$KEY read does not activate masking,
which this branch changes. Corrected in credentials.mdx, function.mdx and the
logging FAQ, and the recognition limits are now written down: runtime-built
names, reassigned bindings, and reads that cannot be told apart from text.
Added a "See usage" section covering who can see it and why an empty trail
means "nothing recognized" rather than "never used".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): writing a name is not reading it, and a bare mention is not a rebinding
Review round 4.
- javascript.ts / python.ts: `environmentVariables.API_KEY = 'x'` and
`delete environmentVariables.API_KEY` touch the name without ever reading the
mounted value, but the detectors matched the member access and recorded a use
that never happened. JavaScript now asks the same isWriteIdentifier the
placeholder rewriter uses (its parameter is widened to ts.Node — the body
already walked generic nodes, so this is a type change, not a behaviour one)
plus a delete check; Python excludes a subscript followed by `=` and a `del`
target.
- shell.ts: requiring every mention of a name to be an expansion also fired on
text that binds nothing — a comment naming the key, or `echo "API_KEY=$API_KEY"`
where the literal is an argument rather than an assignment — and dropping those
cost masking on a genuine read. It now looks for actual writes: an assignment at
command-word position, a binding builtin, `printf -v`, or a `for` target.
The two directions are not symmetric, which is why this errs toward detecting
the read: missing a write records a use of a secret the script only had in its
environment, a misleading audit row and nothing more, since masking still
searches for the real value and will not find it. Over-detecting a write
suppresses masking on a value that does reach the log.
This also makes the code match what the docs already described — skipping after
a rebinding, not after any mention.
13 tests added; 11 fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): an update reads before it stores, and a del target may be parenthesized
Review round 5. The first of these is a regression from round 4.
- javascript.ts: reusing isWriteIdentifier to answer "is this a read" was wrong.
That predicate answers the rewriter's question — is this a target the
substitution must refuse — so it treats every assignment operator alike, which
is correct there and wrong here: `+=`, `||=`, `??=`, `++` and `--` all load the
current value before storing, so they are genuine reads and were silently
losing their masking. Only a plain `=` stores without reading. Replaced with a
purpose-named predicate, and isWriteIdentifier's parameter is narrowed back to
ts.Identifier now that nothing else needs it widened.
A test committed last round asserted the wrong behaviour for `+=`; it has been
corrected rather than left to pin the bug.
- python.ts: `del (environmentVariables['K'])` slipped past a check that looked
only at the characters immediately before the match. It now isolates the
enclosing logical line and tests whether that is a del statement, which also
covers `del((x))`, `del(x)`, `del a, x`, and a del after a semicolon.
12 tests added or corrected; 10 fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): stop excluding Python writes, which kept leaking in the unsafe direction
Review round 6. Greptile found that `del environmentVariables[environmentVariables['K']]`
had its inner access — which computes a key, so it is a genuine read — skipped
along with the delete, leaving that value unmasked.
The narrow fix was another textual rule. Instead this removes the write and delete
exclusions from the Python detector entirely, because they were optimizing the
wrong direction.
`resolvedSecretNames` feeds `outputSecretMatcher`, an exact-value matcher over the
output. Naming a secret the code never read costs nothing there: the matcher scans
for a value that does not appear. Failing to name one that was read leaves it
unmasked. The two error directions are therefore not comparable, and the
exclusions bought only audit-trail tidiness while every heuristic they needed has
so far leaked into the dangerous side — first a parenthesized target, now a nested
read. A `del` or an assignment is reported like any other access.
JavaScript keeps its exclusion: a real AST answers the question per node, with no
text to misread, and it has produced no such hole.
Net 30 lines removed from python.ts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): report recognized reads instead of proving they are not reads
Review round 7. Greptile flagged both directions at once — false usage from
reporting a write target, and unmasked secrets from the file-wide shadow flag —
so I traced what the signal actually drives before choosing.
The chain: the compiler's names feed outputSecretPlaintextsByName and the
exact-value matcher, NOT context.resolvedSecretNames, which starts empty. After
execution activateOutputSecretProvenance scans the output and adds only names
whose plaintext actually appeared; those become __resolvedSecretNames, which
tools/index.ts turns into recordResolved calls, which is what the usage trail
reads.
So a compile-time false positive produces no usage row on the ordinary path — it
only hands the matcher a value the code never emits. It does produce one on the
!projection.safe fallback, where the system already over-approximates by design.
A false negative, by contrast, keeps the value out of the matcher entirely, so a
genuinely read secret is never masked on any path.
That asymmetry decides it, so every "prove this is not a read" mechanism is gone:
- javascript.ts: the file-wide shadow flag. A helper declaring its own
environmentVariables discarded genuine reads of the mounted binding everywhere
else in the file — Greptile's security finding, and real.
- python.ts: the allowlist requiring every mention to be a subscript or .get().
Same hole: passing the dict to a function suppressed unrelated reads.
- shell.ts: the rebinding check. It had the same hole in a form nobody flagged —
`echo "$API_KEY"; API_KEY=local` dropped the first read, which is of the real
secret.
What stays is the question of whether the text is code at all — strings, comments,
single quotes, quoted heredocs — plus the receiver check that `other.environment
Variables['K']` is a different object, and JavaScript's node-precise write/delete
exclusion, which cannot suppress a read elsewhere.
Net 215 lines removed across the three detectors and their tests. Docs updated:
the rule is now stated as reporting rather than proving, and that See usage may
occasionally list a secret the code had available but did not read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(secrets): drop the last write-vs-read special case
`environmentVariables` is a plain object deserialized from the run payload
(route.ts:206), not a handle on the stored secret. Assigning to it changes
nothing outside the sandbox and is discarded when the run ends, so separating a
write from a read bought almost nothing while leaving JavaScript as the one
language still trying to prove a read is not a read.
Every language now follows the same rule: report a recognized read of a
configured secret name. The only exclusions left are facts rather than
inferences — the text is not executable (string, comment, single quote, quoted
heredoc), the receiver is a different object, or the name is not statically
knowable.
Docs note that assigning to the binding does not edit the secret.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(secrets): ship only the fields the trail actually shows
Five fields crossed the API and reached no reader: usageDate, firstUsedAt,
actorEmail, workflowId and actorUserId. The panel renders the timestamp, the
trigger, what used the secret, the actor's name, the run count and the run link;
everything else was projected, serialized and discarded.
first_used_at is dropped from the table as well. Nothing read it, and inside a
per-day bucket "first used that day" says nothing next to "last used that day" —
so it was a column written on every run for no question anyone asks. The upsert
loses its least() with it. Migration regenerated; the identifier columns behind
the joins stay, they simply are not returned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): report referenced code secrets, not only ones that surface in output
The Function route activated a secret's provenance — and therefore its usage row
and downstream masking — only when the exact value appeared in the result,
stdout, or error. That gate made the trail miss silent use entirely: a key that
authenticates an API call and is never echoed reported nothing, and so did the
founding scenario of this feature, a key exfiltrated character by character. The
innocent run that echoed a key got a row; the run worth catching did not.
Activation now follows the referenced set the compiler already computes: resolved
{{KEY}} bindings plus recognized direct reads, filtered to configured values —
the same set the unsafe-projection fallback already activated. An extra name only
hands the output matcher a value that never appears; configured-but-unreferenced
values are still never included. The output-scan activation path and its surface
helper are deleted rather than kept alongside.
One old test pinned the gate ("does not activate a referenced secret that does
not cross the Function result"); it now asserts the reverse, with the reasoning
attached. Two new tests pin the char-split exfiltration and the silent API-call
case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): shell escaping is backslash parity, not adjacency
Review round 8. `\\$API_KEY` is an escaped backslash followed by a LIVE expansion
— bash prints `\` plus the value — while `\$API_KEY` is an escaped dollar and
stays literal. Checking only the character adjacent to `$` read every even run as
escaped, dropping a real read from usage and masking alike; verified against
bash before fixing.
The scanner now counts the run of backslashes before the `$` and skips only odd
runs, the same parity rule logicalLineEndAfterContinuations in this file already
applies to line continuations. Six-case parity table added; the three even-run
cases fail against the previous check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): recognize destructured environment reads
Review round 9. `const { API_KEY } = environmentVariables` delivers the value by
name with no property- or element-access node in the AST, so the member-access
walk missed it entirely — and a missed read leaves an emitted value unmasked,
the dangerous direction.
The AST walk now also recognizes the declaration form (shorthand, renames,
defaults, string-literal keys), the assignment form ({ KEY } = env), and a
...rest element — which names no key but takes every value, so it reports every
configured name; the alternative left `const { ...all } = env; return all`
entirely unmasked. A computed key stays unrecognized, the same runtime-name
boundary as a computed subscript, and a receiver that is not the bare identifier
is not attributed.
Nine cases added; the six positive ones fail against the previous walk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): one receiver rule for destructured reads, parentheses included
Review round 10. Two accurate findings, folded into a generalization instead of
two more special cases:
- A parameter default (function f({ API_KEY } = environmentVariables)) and a
binding-element default are the same by-name delivery as a variable
declaration. The detector now keys on the ObjectBindingPattern itself and
checks its parent's initializer, so every declaration position follows one
rule instead of per-kind arms.
- Parentheses group without changing the receiver, so (environmentVariables) is
unwrapped before the identifier check — in the destructuring arm AND the
member-access arm, which had the same hole unreported.
Declined the for-of-over-array-literal finding: the receiver there is a
container, not the environment object, and following data flow through
containers has no fixed point — the same documented boundary as aliasing and
computed keys. A test pins the boundary so it reads as chosen, not missed.
Eight cases added; the seven receiver-rule cases fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): a dot in prose is not a qualifier, and a literal computed key is a subscript
Review round 11. Both findings were implementation-narrower-than-rule, fixed by
consulting authorities the detectors already had rather than adding new ones:
- python.ts: the receiver walk crosses whitespace so a parenthesized `other.` on
a previous line is seen — but it landed on a comment's final period
(`# Load the value.`) and discarded the genuine read on the next line. The
landing position is now checked against the same lexer ranges that filter the
candidates, which is also why the receiver check moves after lexing.
- javascript.ts: `const { ['API_KEY']: key } = environmentVariables` is the
element-access rule in pattern position, so a computed key holding a string
literal resolves like a literal subscript; any other computed key keeps the
runtime-name boundary a computed subscript already has.
Eight cases added; the comment-period case and all three literal-computed-key
cases fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Waleed
|
9328e66d69 |
feat(account): let users delete their own account (#6831)
* feat(account): let users delete their own account Adds a GDPR self-serve account deletion path: a preflight that reports what deletion would remove and every reason it would be refused, and a confirmed delete that erases the account and everything only it can reach. Deletion refuses while the account is still entangled rather than reassigning its content. Most tables reference user.id with ON DELETE CASCADE, and those cascades do not distinguish content in the account's own workspace from content it created inside somebody else's, so each blocker names the existing action that untangles it (leave the workspace, leave the organization, cancel the plan) — all of which already hand work over on their own tested paths. * fix(account): make deletion atomic and re-check privacy at delete time Reorders the teardown so nothing irreversible happens before the deletion is certain: anchors are handed over first (the fallible step, while everything is still recoverable), the workspace and user deletes now share one transaction, and the object-storage purge runs only after that commits. The workspace delete also re-checks inside the transaction that each workspace is still private, so a membership granted between the preview and the delete aborts the whole thing instead of destroying the new member's access. * fix(account): close deletion gaps found in review - Run the whole teardown in one transaction. The billing and ownership handovers now take the caller's transaction, so a refused deletion can no longer leave a workspace reassigned for a deletion that never happened. - Fail closed on a subscription read error. getHighestPriorityPersonalSubscription defaulted to returning null, which read as "no plan" and would have erased an account Stripe was still billing. - Erase the account's profile picture. It is personal data under our own storage prefix; an external provider avatar is left alone. - Enforce the storage purge cap while collecting keys rather than after, so an oversized account cannot exhaust memory before the cap applies. |
||
|
Justin Blumencranz
|
b6ca0d787f | fix(emcn): prevent leading glyph clipping in chip inputs (#6833) | ||
|
Waleed
|
374c1064ea |
chore(cli): release 2.1.0 (#6826)
The publish workflow derives the npm version from this field, and on `main` it uses it verbatim: `2.0.0` was already on the registry, so the release step skipped and every change since then stayed unpublished. Staging and dev never showed it because they append `-preview.N` and `-dev.N`, which are always new. So the merged CLI work — the redirect refusal, the default endpoint, folder path encoding, the request bound, the User-Agent, `run --follow`, `runs wait` and `logs follow` — is on `main` but not on npm, and `sim@latest` is still the build whose every write is dropped by the apex redirect. Minor rather than patch: three commands are new and the default endpoint changed. |
||
|
Waleed
|
1c69372cba |
feat(cli): follow a run, wait for one, and tail the log (#6813)
* feat(cli): follow a run, wait for one, and tail the log Three commands the surface was missing, each polling or streaming something the generated command layer cannot express. `workflows run --follow` renders the SSE the execute route already emits, so a multi-minute agent run stops printing nothing until it ends. It rides on the generated `run` leaf rather than a sibling command — same operation, one different response encoding — and delegates to the handler it replaced, so every non-follow invocation still runs the generated path. Answer text, thinking and tool calls go to stderr; only the final envelope reaches stdout, so redirecting still yields the result. Reasoning and tool frames need the `X-Sim-Stream-Protocol` header, which is sent only when asked for, because negotiating also switches answer text to live chunks the server may retract. `workflows runs wait` closes the loop `--async` opens. Terminal is completed, failed or cancelled; `redacting` is not, since a run whose output is still being scrubbed is not yet a run you can read. A time pause keeps polling because the server resumes it, and a human pause stops with the resume command rather than burning the bound and calling it a timeout. Distinct exit codes keep cancelled and paused from reading as failure. The bound is `--wait-timeout` and not `--timeout`, because SIM_TIMEOUT_SECONDS already bounds one request and two knobs of the same name hide each other. `logs follow` tails runs as they arrive. Dedup keys on run id, not on the timestamp: a schedule fan-out starts many runs in the same millisecond, so a timestamp watermark either drops the siblings or reprints them. JSON output is one object per line, because a follow never closes an array, and the table header is printed once so columns stay aligned across polls. * fix(cli): disclose a truncated burst, and clear a stale retry notice Two review findings in `logs follow`, both verified against the code first. The page budget bounds one poll so an enormous burst cannot stall the follow, but on reaching it the live cursor was discarded: the remainder is older than everything collected and the next poll restarts at the newest page, so those runs were never printed and nothing said so. The budget stays — draining without one trades a bounded poll for unbounded buffering in a process meant to run for hours — but hitting it now warns on stderr, naming the count and pointing at `sim logs list`. That notice is written even off a terminal, because a piped log is where an unexplained hole is hardest to spot. The retry notice was cleared after the empty-rows check, so a poll that recovered but found nothing left "retrying in Ns…" on screen while the follow was already healthy. Clearing now happens as soon as a poll succeeds. The second test needed two failures to be worth anything: the teardown clears the line either way, so what separates fixed from broken is whether a bare erase lands before the second notice or only at the end. The first version passed against the bug. * test(cli): pin that a mixed page is the watermark, not a truncation A page holding a run already printed proves the follow caught up, so the truncation warning must not fire there — that is how every healthy poll terminates, and warning would report a hole on the ordinary path. The straggler sharing that page is still collected, because the filter takes every unprinted row on it rather than only those above the known one. * fix(cli): say when the requested backlog was larger than a page holds The logs API clamps `limit` into 1–1000 rather than rejecting it, so `logs follow -n 5000` came back with 1000 rows, anchored the floor to that partial page, and said nothing. The seed already knew — it computes whether a live cursor remained — but the caller discarded the answer. Guarded on both halves. Fewer rows than asked for is only a shortfall when more were waiting: a workspace holding ten runs answers `-n 50` with ten and nothing is missing, so warning on the row count alone would fire on every small workspace. The cursor is what separates the two. |
||
|
Waleed
|
51f1d45eff |
perf(table): collapse sequential round trips on the table read and write paths (#6808)
* test(table): characterize the single-row route before migrating it
The single-row surface (GET/PATCH/DELETE) had no route-level tests despite
being the hottest table write path. Pin the behavior it emits today so the
migration onto the shared internal route builder is verifiable rather than
hopeful.
Covers status codes, body shapes, ISO-8601 timestamp serialization, the
access level each method demands, collaborator invocation, and the
dual-caller wire keying — session callers pass column ids through untouched
while internal-JWT callers translate names to ids in both directions.
Verified to fail: mutating the wire translator, the deleted count, and the
workspace-ownership guard each turn the corresponding tests red.
* feat(table): model wire keying and actor attribution on row write use cases
The row write use cases assumed every caller speaks column names. That holds
for /api/v2, /api/v1 and the Copilot tools, but not for the first-party grid
or the internal /api/table routes, which address cells by stable storage id.
Feeding id-keyed data through the name remap drops every key it does not
recognise — a storage id names no column name — so the write would store
nothing and still report success.
Make the wire an explicit, required property of the input rather than an
assumption. `dataKeying: 'names' | 'ids'` sits alongside `strictWrite` and is
required for the same reason: a new write surface must state which contract it
publishes. Strictness now means the same thing on either wire — an unknown
column id is refused exactly as an unknown column name already was.
Single-row writes also gain optional actor attribution, so the acting tab can
skip refetching its own write. It is optional and absent by default, so every
existing caller keeps broadcasting to all subscribers as before. Only the
single-row create, update and delete paths accept it; a batch write is not
reconciled locally by the actor and must still refetch.
The attribution pin moves with the behaviour: what selects the actor-scoped
signal is no longer which file calls it but which surface supplies an actor,
so that is now what the test pins.
Verified to fail: ignoring the keying discriminator, and dropping actor
attribution, each turn the corresponding tests red.
* perf(table): remove two round trips from every single-row update
A single-cell PATCH spends far more time in sequential round trips to a remote
Postgres than in the UPDATE it issues. Prepared statements are disabled for
PgBouncer transaction mode, so every await is a full Parse/Bind/Execute.
Two of them were avoidable.
getRowById issued the row lookup and its executions sidecar in series, but the
sidecar is keyed on the row id the caller already supplied, so it never
depended on the lookup. Issuing both together makes it one round trip. A miss
now pays one redundant sidecar read, which is the rare path and costs no extra
wall time.
The uniqueness probe ran whenever the table had any unique column, passing the
fully merged row, so editing an unrelated cell re-probed every unique column —
its own transaction plus one query per column. It is now scoped to the columns
the patch actually writes. A merge cannot newly violate uniqueness on a column
it leaves alone: that value is the one already stored, and it satisfied the
constraint when it was written.
Sized before changing: few tables declare a unique column, but write traffic
concentrates in the ones that do, so this is the larger of the two savings in
practice.
Verified to fail: reverting the probe scoping turns the covering test red.
* perf(table): start the workspace load with the table load when a workspace is asserted
resolveActiveTableContext ran two sequential round trips: load the table,
then load the workspace it turned out to live in. When the caller asserts a
workspace the second input is already in hand, so both can start together.
What makes that safe is unchanged: requireTable still compares the table's
canonical workspaceId against the assertion and reports a mismatch as
not_found. The table outcome is inspected first and unconditionally, so any
path that returns has proven the assertion equal to the canonical id, and a
failing workspace load can never replace the concealing not_found. A final
identity check on the loaded context restates the invariant at the point of
return, so even with the first check removed the function cannot hand back a
foreign workspace. Promise.allSettled keeps the discarded branch from
surfacing as an unhandled rejection.
With no asserted workspace the path stays sequential — the table load is what
reveals which workspace to load, so there is nothing to start early.
One existing assertion changed: it required the workspace load not to have
been issued yet on a mismatched assertion, which is internal sequencing rather
than caller-observable behaviour and is definitionally untrue once the loads
overlap. The observable half is kept, and two timing tests now cover the
sequencing directly.
Verified to fail: removing either mismatch check, reading the workspace
outcome first, and swapping allSettled for bare awaits each turn the
corresponding tests red.
* perf(table): read a table and its latest job in one round trip
getTableById issued the table SELECT and then awaited latestJobForTable, so
every table request paid two sequential round trips. With prepared statements
disabled for PgBouncer transaction mode every await is a full round trip, and
this loader is on essentially every table route.
The job read cannot be skipped: a table's reported rowCount is the stored
count minus the job's pendingDeleteRemaining, so dropping it would overstate
the count during a pending delete and could wrongly reject inserts as over
capacity. An opt-out flag would have made that a caller's trap. Instead the
job is read in the same statement, as a correlated jsonb subquery in the
select list — the select-list form of a LEFT JOIN LATERAL, which is what
drizzle can type here. Output is unchanged for every input.
latestJobForTable is deleted rather than left dangling: getTableById was its
only caller, and keeping it would have carried a third copy of the
exports-excluded / newest-started_at / limit-one rule. mapJobRow is now
exported so the batch path and the lateral share one implementation of the
doomedCount and pendingDeleteRemaining logic. The batch DISTINCT ON path used
by the list endpoint is untouched.
Verified to fail: dropping the export filter, reversing or re-keying the sort,
dropping the limit, dropping the correlation, loosening either doomedCount
condition, and removing the rowCount subtraction each turn tests red. Dropping
the lateral from the projection initially survived, because the shared db mock
returns queued rows regardless of predicate; a projection assertion now
covers it.
* chore(table): drop the unused single-row GET contract
Authored as groundwork for migrating the internal row routes onto the shared
builder, which is not in this change. An exported contract nothing consumes is
dead code, so it lands with the migration that needs it instead.
* fix(table): resolve strict id-keyed columns through getColumnId
assertKnownColumnIds read column.id directly, but a column id is optional —
pre-backfill columns have none and are stored under their name, which is why
getColumnId exists and is what every other consumer of the schema uses. Such columns still
exist, so a strict id-keyed write naming one would have been refused as
unknown.
Latent today: no surface yet combines dataKeying 'ids' with strictWrite. Fixed
before one does.
Verified to fail: reading column.id turns the covering test red.
* refactor(table): apply review findings from the quality pass
Four parallel reviews (reuse, simplification, efficiency, altitude) converged
on the same set. Applied:
Removed actorClientId entirely. It had no supplier anywhere in the repo, so
every call reached signalTableRowsChangedByActor(id, undefined), which is
byte-identical to the broadcast it replaced — three optional fields, three
verbatim doc blocks and a pin test asserting the empty set, all inert. It
belongs with the route migration that supplies an actor. The attribution pin
is restored to its original form.
The uniqueness probe was only half-narrowed: the patched column list was
computed and then discarded, and the probe re-derived every unique column from
the full schema. It now receives only the columns the patch touched, so a
table with several unique columns runs one query instead of all of them.
assertKnownColumnIds hand-rolled an id index and duplicated the sibling
assert's message verbatim. It now reuses buildColumnNameById — which already
keys by getColumnId, so the legacy pre-backfill column case is handled by the
shared helper rather than by a special case here — and both asserts share one
message builder.
The job field list had become two copies, one drizzle-checked and one an
unchecked sql<T> cast that could silently return undefined for a renamed
field. The lateral now derives its jsonb pairs from JOB_PROJECTION, which
satisfies Record<keyof LatestJobRow, Column>. That compile-time guarantee
replaces the runtime drift test it makes redundant.
Also: hoisted the id index out of the batch loop to match the names path,
dropped a never-supplied parameter, narrowed an over-broad parameter type,
removed a dead timer cleanup and its now-unused import, replaced dynamic
re-imports with the static one already present, hoisted a repeated stub, and
documented why filters need no keying counterpart and how laxness differs
between the two wires.
Verified to fail: removing a field from JOB_PROJECTION breaks the build in two
places.
* test(table): share one table-definition fixture factory
buildTable was copy-pasted into 13 route test files under app/api/table, each
a near-identical TableDefinition literal. A required field added to that type
would have failed 13 files individually.
packages/testing already owned createTableColumn and createTableRow but no
definition factory, and — as it turns out — did not export any of them from
the barrel, so they were unreachable from @sim/testing. Adds
createTableDefinition beside them and exports all three.
The fixture type is a structural stand-in rather than TableDefinition itself,
because packages/* must not import from apps/* — the same approach the
existing factories in that file already take.
No assertion changed. Call sites that varied a field pass it as an override;
the four files with several call sites hoist a shared options const and spread
it so each call still gets a fresh object.
* perf(table): project only the job field the row count needs
Both job reads selected the whole payload jsonb, but mapJobRow reads exactly
one number out of it, and only for a running delete. The payload also carries
the delete job's filter and an unbounded excludeRowIds array, and the latest
non-export job is read on essentially every table request — a table that once
ran a large delete would ship that id list on every read, forever.
LatestJobRow.payload becomes doomedCount, extracted in SQL. Both readers share
JOB_PROJECTION so one edit reaches the batch DISTINCT ON and the correlated
subquery alike; the compile-time constraint widens to Column | SQL rather than
being dropped.
Behaviour is identical. `->` keeps the value jsonb, which postgres-js decodes
through its built-in JSON.parse handler, so it arrives as a number with no
boundary coercion. A null payload, a payload without the key, a non-object
payload and an explicit JSON null all collapse to the same `?? 0` the previous
optional chain produced.
Sized honestly before claiming a win: payloads are small in practice today, so
this is defensive rather than impactful — it removes an unbounded growth path,
not a measured cost.
Verified against a real Postgres, not just the mocked driver: the
generated correlated subquery returns doomedCount 12 for a delete job, null
for an import job, and a null row for a table with no job.
Also from the review pass: re-homes the strictWrite explanation onto
rowWriteOptions, where six {@link} references now point; records why
replaceProjectedWireRows carries no keying discriminator; notes the one case
the uniqueness-narrowing invariant does not cover; pins the lax id-wire
passthrough with a test; and renames a parameter that misled once only its
keys were read.
* chore(table): drop two comments the code already says
One restated the identifier below it (checkUniqueConstraintsDb), and one was a
section-divider banner in the factories barrel, which the repo's comment
convention rules out. Everything else that survives is a why the code cannot
express: round-trip rationale, sql.raw input safety, the narrowing invariant
and the one case it does not cover.
* refactor(table): move the internal row routes onto the application boundary (#6809)
* refactor(table): move the single-row route onto the application boundary
The hottest table write path authorized in its own handler and queried the
database from the adapter — the two things a surface adapter must never do. It
now declares itself with defineInternalJsonRoute against the readRow, updateRow
and deleteRow use cases: 127 lines instead of 274, with no db import, no
drizzle import and no checkAccess.
Doing that surfaced why the violation existed. Write-provenance resolution
needs the canonical schema to map a caller's column key to the storage column
it certifies, and the adapter could only do that because it was already loading
the table illegally. The envelope is now split along the real seam: the adapter
reads the header and payload field, which is transport, and the use case
resolves the selections against the canonical table, which is domain.
That split has to preserve a distinction the defaulting logic would erase. An
internal caller that sends no envelope stays deliberately untracked; defaulting
it to an exact-empty stamp would certify "this write introduced no secrets" on
a runtime write that may well have introduced some. Only an interactive caller
certifies exact-empty, over the storage columns its write actually persists.
Two further changes fell out of it:
present() now receives the same { principal, input } pair its sibling hooks
responseHeaders and finalizeResponse already got. This route serves a session
and a workflow execution on one path and owes them different column keyings, so
rendering per caller kind is presentation rather than domain. That was a gap in
the builder, not a special case for this route.
tableRowWireSchema describes what the single-row routes actually return. The
contract claimed a full TableRow, carrying the executions sidecar and Date
objects — true of the list and query routes, and never true here. The hand-
rolled handler was never checked against its own contract, so the drift was
invisible until the builder started validating it.
Wire changes, both deliberate and both narrower than before: a cross-tenant
table now conceals as 404 where the old blanket handler answered 403, while an
in-workspace role denial still answers 403. Nothing in hooks/queries/tables.ts
branches on either.
Verified to fail: forcing one keying, dropping the actor, pre-resolving the
envelope, certifying an untracked internal write, and skipping the bundle
completeness check each turn the covering tests red.
* refactor(table): move the upsert route onto the application boundary
Same shape as the single-row route: declares itself against upsertTableRow,
hands the provenance envelope over unresolved, and derives its column keying
from the principal rather than assuming one.
The keying and presentation helpers the two routes shared are now in row-wire
beside the translators they wrap, so a third route does not restate them.
Two response details changed on purpose. The row now carries `position`, which
the use case always had and this route alone omitted — every other single-row
response already returned it, and the contract now describes one shape instead
of three. The upsert result also carries read-back provenance, which the route
previously assembled for itself.
The surface had no route-level tests; it has six now, covering both caller
keyings, both operations, and the envelope handover.
* refactor(table): move the enrichment-detail route onto the application boundary
The last internal adapter that queried the database for itself. It now runs
through readTableRowEnrichmentDetail, a new use case that shares
tableOperations.readRow — reading a cell's cascade breakdown is a projection of
the same row under the same role, not a second semantic operation.
Its tests move to the same seam and gain one the old suite could not express:
a cross-tenant table now conceals rather than confirming it exists.
* fix(table): mirror the storage rule when keying write provenance
storageKeyByWireKey mapped an unrecognised id-keyed key to null, but
rowDataToStorage persists that key when the caller is not writing strictly. A
cell would have been written with no provenance recorded, under a stamp still
marked complete — the same failure the bundle completeness check exists to
prevent, arriving through the keying map instead of the selection set.
The two wires genuinely differ and the code now says so: the name path drops an
unrecognised key, so it maps to null; the id path stores what it is given, so
every key it sends is a storage key.
Unreachable today, since no delegated surface uses id keying and a session
bundle is refused earlier. Fixed because the function's stated invariant — that
it mirrors how the row data itself is normalized — was not true.
Also corrects the delegated principal fixture in these tests, which used a kind
that is not in the Principal union, so the subject-id branch was never actually
exercised. It is now, and the scope check is asserted to receive the acting
principal's own subject id.
Verified to fail: restoring the schema-based lookup turns the covering test red.
* fix(table): restore executor access to the migrated row routes
The migration swapped checkSessionOrInternalAuth for the delegation policy,
and that broke every Table block call to these endpoints in two ways at once.
The old policy accepted a legacy internal token. The new one requires a
delegation token, which the executor only mints when the tool asks for it —
and none of the four table row tools did, so get/update/delete/upsert row
would each have failed with a 401. The knowledge tools already declare it,
because their routes migrated first.
Even with a valid token the operations denied the caller: readRow, updateRow,
deleteRow and upsertRow ran under a policy whose delegatedServices is
['copilot'], so the executor got a 403. They now use the tool-facing policy
that already existed for the group operations.
Neither was visible to the route tests, which mock the auth policy wholesale —
so the gap is closed at the layer that actually decides: one test pinning that
each tool requests delegation and that its operation admits the executor,
mutation-verified against both failure modes.
Also fixes findings from the review pass: the read surfaces no longer load an
executions sidecar none of them put on the wire (two readers rather than a
flag, so a caller cannot silently read an empty one); the provenance name
index is built once per batch instead of once per row; the uniqueness comment
now names the concurrent-insert race as well as the retro-added constraint;
and the presenter context gets NoInfer plus a note that the v2 builder passes
something different.
* fix(table): keep the lock on a 423 and pin the remaining wire changes
The rows error policy was built on the concealment base rather than the
lock-aware one, so a TableLockedError fell through to the generic handler and
the response lost its `lock` field — the only thing that tells a client which
lock to clear. A row write is exactly as lockable as a group mutation, so it
now shares that base.
Also pins the two wire changes the review found undocumented: a mismatched
workspace assertion answers 404 rather than 400, which is a superset of the
cross-tenant concealment already intended, and an unclassified failure answers
the builder's shared "Internal server error" rather than the old per-route
text. Both are consistent with the ~80 routes already on this builder; they are
asserted so they read as decisions rather than drift.
Verified to fail: reverting the policy base turns the lock test red.
* refactor(table): apply the quality pass
Four parallel reviews (reuse, simplification, efficiency, altitude). The
highest-value finding cut against the branch's own purpose: the rows error
policy sat in the barrel-exported route-policies module, so its import of the
1,200-line row use-case graph was paid by every one of the ~28 table routes
that can never throw a row error — the barrel's import cost went from ~1.1s to
~1.7s. row-route-policies.ts already existed for exactly this and is
deliberately not re-exported; the policy now lives there with its v2 sibling.
The upsert path still loaded an executions sidecar no surface puts on the wire,
and did it inside the write transaction, holding it open for a discarded
result. The read path got that fix earlier; the write path next to it did not.
rowKeyingForPrincipal fell through to name keying for anything that was not a
session. The operation policy admits API-key principals, so the first one to
reach these routes would have had every id-keyed cell dropped and the write
reported as successful. It is now an exhaustive switch over the two kinds the
auth policy yields — which immediately failed three tests using a principal
kind that exists nowhere in the repo, so those fixtures are real now too.
Also: reuses toWireTimestamp and createUnknownTableRowSecretProvenance instead
of re-inlining them; shares one helper for the provenance choice the update and
upsert use cases both make; keeps one canonical actorClientId doc with two
cross-references; merges two maps keyed by the same string into one; stops
round-tripping the principal through the legacy AuthType enum; hoists the
presenter function type out of both conditional branches; drops a subsumed
test; and freezes the shared locks fixture so a mutating test cannot poison its
siblings.
|
||
|
Waleed
|
0e84e92d39 |
fix(cli): bound, trace, and explain the requests the CLI makes (#6798)
* fix(cli): bound, trace, and explain the requests the CLI makes Four transport gaps, all of which failed silently. A request had no timeout, so a connection that was accepted and then never answered hung the terminal indefinitely. `SIM_TIMEOUT_SECONDS` now bounds one, defaulting to 3600s — deliberately above every timeout the server itself applies, since a synchronous workflow run is allowed 3000s on a paid plan and a tighter default would abort real work and report it as a transport failure. `0` removes the bound, for a self-hosted deployment that runs executions without one of its own. The caller's abort signal is composed with the timeout rather than replaced, so neither masks the other. Node ignores HTTP(S)_PROXY unless NODE_USE_ENV_PROXY opts in, and only from v22.21 and v24.5, so on a network that reaches the API only through a proxy every command failed to connect while the variable that would have fixed it was already set. The CLI cannot enable that from inside the process — Node reads it at startup — so it says what to do rather than bundling an HTTP stack for a setting the platform now owns. An API key was sent to any http:// endpoint with no signal. Now a warning, not a refusal: http is the documented way to reach a local dev server, and a deployment terminating TLS at a gateway is real. Loopback stays silent. `SIM_DEBUG=1` traces method, URL, status and duration. Bodies and headers are deliberately absent — the request carries the API key, and `secrets set` carries the secret itself. All four write to stderr, so a piped stdout stays parseable. * fix(cli): make the request bound safe on every runtime it supports Two ways the new timeout could fail before the request was made. `AbortSignal.any` arrived in Node 20.3 and this package supports Node 20, so composing a caller's abort signal with the timeout threw a bare TypeError on the earliest 20.x releases. It is now used when present and composed through an AbortController when not. `AbortSignal.timeout` rejects a fractional millisecond outright, and past 2^31-1 ms it does not fail at all — it clamps to 1ms, so the longest timeout anyone asked for became the shortest. The value is now rounded and refused above what Node can actually wait, pointing at 0 for an unbounded wait. Also unstubs env vars between tests: `stubEnv` is not undone by `unstubAllGlobals`, so a SIM_TIMEOUT_SECONDS set for one test configured every test after it. * fix(cli): correct the proxy version table, and classify a timeout mid-body `runtimeCanProxy` treated any release between 22 and 24 as capable, so on Node 23 — which reached end of life before the backport — a configured proxy was ignored and the CLI stayed silent about it, which is the exact failure the warning exists to report. The table is now the two lines that shipped the support, and anything after them. `AbortSignal.timeout` keeps firing after `fetch` resolves, so a bound that elapsed while the body was still being read — a large `files get` — escaped the client's own handling and printed a raw TimeoutError stack. The top-level handler now names it, which covers the streaming path as well as the JSON one. A user's own Ctrl-C raises AbortError and is deliberately left alone. * fix(cli): report a timed-out download as a timeout `files get --output-file` streams the body to disk, and `streamToFile` converted anything the stream threw into a write failure. So a request bound elapsing mid-download read as `Could not write <path>: ...`, sending the reader to check permissions and free space for a timeout they can raise, and hiding the one instruction that resolves it. The predicate and that instruction now live beside the timeout that raises them, so the client, the top-level handler and the download path all say the same thing. The wrapping stays where it is: the staged-download cleanup runs off that failure, and rethrowing past it would leak the temporary directory. * fix(cli): keep a sub-millisecond timeout bounded Zero is how this function says "no bound", so rounding a positive SIM_TIMEOUT_SECONDS down to zero inverted the request: anything under 0.0005s asked for the shortest possible timeout and got none at all, leaving a stalled request to hang. Introduced by the rounding that fixed the fractional-millisecond rejection. Floored at 1ms for every positive value; only a literal 0 still disables. |
||
|
Waleed
|
0b4d34137b |
feat(secrets): add optional descriptions to workspace secrets (#6796)
* feat(secrets): add optional descriptions to workspace secrets Workspace secrets already have a backing credential row with a description column, but nothing surfaced it. Teammates had no way to record what a secret is for. - Add a Description field to the secret detail page, matching the integrations credential page, gated on workspace-secret admin - Fold the value and description editors into one Save/Discard pair and one unsaved-changes guard; two guards cannot coexist, since each seeds its own same-URL history entry - Match descriptions in the secrets settings search - Expose description on GET/PUT /api/v2/secrets and in the CLI Descriptions are workspace-only: env_personal credential rows are per-workspace mirrors of one user-global secret, so one saved there would exist in a single workspace, and a personal secret has no teammates to inform. The API rejects a description on personal scope rather than silently dropping it, and omitting it on PUT leaves any existing description untouched so a value rotation cannot erase it. * fix(secrets): address review findings on secret descriptions - Patch the credential detail cache optimistically on update. `onMutate` cancelled the detail query but only patched the lists, so a detail-backed editor stayed dirty after a successful save until the refetch landed — long enough for Discard to restore the pre-save value over the committed one, and for Back to open the unsaved-changes guard. - Memoize `useSecretValue`'s returned callbacks and object, per the hook convention, so the composed form's save/discard stop churning per render. - Reject a description on a personal secret in the domain layer rather than only at the v2 boundary. The internal credential update path accepted one for any type, writing data every reader hides. - Normalize an empty description to null so the API and UI agree. - Correct the secrets documentation, which described a Display Name field the detail view does not have and omitted the scope rule. - Drop the CLI's copy of the 500-character bound; it can't import the contract, so a copy only drifts from the message the API already returns. - Collapse a redundant save guard and align the description write gate with the render gate. Leaves the integrations credential page byte-identical to staging. * fix(secrets): keep the API docs example and CLI column order stable Backward-compatibility fixes for anyone who never sets a description. - Move the blank-to-null normalization out of the contract and into the route. A Zod `.transform()` on any property drops the whole request schema's OpenAPI examples, which had silently removed the Set Secret request example from the published docs. - Append the CLI `description` column instead of inserting it before `updated`. `--output text` is positional, so inserting would shift every field an existing script cuts. - Reject a description on a personal secret with a message that says so, rather than dropping the field and falling through to the generic "no updatable fields" error. |
||
|
Waleed
|
3abac09dc3 |
fix(cli): name SIM_CONFIG_DIR in help, and fix the similarity score width (#6795)
Two loose ends from the command-surface audit. `--help` stated where the profile files live and named only `~/.sim`, so it was wrong for anyone who had set `SIM_CONFIG_DIR` — which every CI job pointing the CLI at a scratch directory has. The variable is documented in the README and the guides; help was the one place that omitted it. `knowledge search` printed the raw similarity double, `0.2818957269585687`: a nineteen-character column whose last dozen digits cannot separate one result from another. A `score` format fixes it to four decimals, the width `cost` already uses, so the column stays put down the page. `json` and `yaml` still carry the full double, which is what a script compares. |
||
|
Waleed
|
fe4480d3f7 |
fix(cli): keep a path prefix in the login URLs (#6793)
* fix(cli): keep a path prefix in the login URLs
`sim login` built both of its URLs with `new URL('/path', endpoint)`. A
leading-slash path is absolute, so it resolves against the endpoint's origin
and drops any path the endpoint carries: a deployment served at
`https://host/sim` sent the browser to `https://host/cli/auth` and polled
`https://host/api/cli/auth/poll`, neither of which exists there.
Every other command builds its URL by concatenation and was unaffected, so
the endpoint looked correct and login alone failed.
Both now go through the client's `buildUrl`, which is exported for it rather
than duplicated — one URL builder for the whole CLI is the point, since two
of them is how the halves drifted apart. Its TSDoc now names the trap.
* test(cli): restore fetch with spyOn so the stub cannot outlive its test
vi.stubGlobal is not undone by restoreAllMocks, so the completed-auth
response would have leaked into whatever ran next. The rest of this file
already spies on globalThis.fetch, which the existing teardown restores.
|
||
|
Waleed
|
5c37778bbd |
fix(cli): identify the CLI to the API and name the remedy on every key refusal (#6792)
Sends a `User-Agent` on both request paths. Without one a CLI request is indistinguishable from any other API traffic, so a bug that reproduces on a single CLI version cannot be found in the server's own logs; the runtime and platform ride along because they are the first things asked about a transport failure only some users hit. The version moves into its own module so the HTTP client can read it. It could not import `program.ts` — program builds the commands, which reach the client — and duplicating the manifest read would let the two disagree. Also recognises `PRINCIPAL_KIND_NOT_PERMITTED`. The same refusal is raised at two layers under two codes, and only the workspace-key one was matched, so the audit-log commands reported "Principal kind workspace_api_key cannot perform operation audit_logs.list" — accurate, written for a server log, and missing the one sentence that tells the reader a personal key resolves it. |
||
|
Waleed
|
60097c89b4 |
fix(cli): default to the host that serves the API (#6791)
`sim.ai` answers /api/** with a 301 to `www.sim.ai`, and the client refuses to follow redirects — a 301 rewrites a POST into a bodyless GET, so following one turns a write into a silent no-op and hands the API key to whatever host Location names. Defaulting to the apex therefore failed every command for anyone who never set an endpoint. Before the refusal shipped it was quieter and worse: reads succeeded while writes did nothing. Also trims the provider catalogue from eleven inferred columns to seven. `docsUrl`, `helpText`, `requiresClientGeneratedCredentialId` and the nested `fields` are what you read once you have chosen a provider, not what you scan to choose one, and they pushed the table well past a terminal. Both ids stay: `credentials connect` names an OAuth provider by `serviceId`, `credentials create` matches a service account on `providerId`. |
||
|
Waleed
|
ae2147645c |
fix(cli): resolve findings from a full command-surface audit (#6788)
* fix(cli): resolve findings from a full command-surface audit Exercised all 147 commands against a live deployment. Fixes the defects that surfaced, plus the docs and generator drift they exposed. Transport - Stop following redirects. A bare domain that 301s to www silently converted POST to GET and dropped the body, so reads worked while every write failed with a misleading validation error and login returned 405. Both the client and the device flow now explain the redirect and name the endpoint to configure, rather than carrying credentials off-origin. - Report a non-JSON response as one instead of printing the HTML page. - Name the personal-API-key remedy on a workspace-key refusal, reading the machine-readable code the API actually sends. - Drop union-branch noise from validation errors that contradicted itself. - Show paging progress on stderr for multi-page fetches. Output - Clamp record values for table only. text is the format built for pipes, and it was truncating signed URLs and tool source mid-value. - Infer timestamp, duration, bytes and boolean formatting for API-owned keys so undeclared commands stop printing raw ISO and float ms. Skips user-defined table cells and leaves json/yaml on the raw payload. - Render a declared-but-absent field as an em dash; billing credits were vanishing silently. Paths, naming and validation - Percent-encode folder paths per segment and decode them for display, so a folder reads and types as the name shown in the app. - Reject a malformed endpoint where it is set and where it resolves, instead of crashing with a URL parse trace. - Request the detail level logs list's own columns need; its workflow column could never populate. - Rename three commands that described themselves wrongly and align two flags with their siblings. Old spellings still work: hidden, warned on stderr, and kept out of help and docs. - Verify whoami against the API, separating a bad key from an unreachable endpoint, and report the workspace by name. - Correct the --yes help text, which advertised skipping a prompt that does not exist. Docs - Teach the docs generator that a flag required by the runtime is required, and that hidden commands are not documented. * fix(cli): clear the paging progress line when a page fails Progress is written without a trailing newline so it can be overwritten in place, and both paging loops cleaned it up only on success. A page that threw part-way through left `fetched 1200…` on the line the error was then printed onto, so the two ran together. * fix(cli): name a working API root when an endpoint redirects The suggested endpoint was the redirect target's origin, which drops a path prefix. A self-hosted deployment reached at https://host/sim was told to set https://www.host — not an API root, so following the advice replaced one broken endpoint with another. Derive it by stripping the request's own path from the target instead, so a prefix survives, and say nothing about --set-endpoint when the target resolves to the endpoint already configured: a trailing-slash or path normalization redirect keeps the origin, and naming the value the caller already has explains nothing. The login poll shared both faults and now shares the helper. |
||
|
Waleed
|
d152fad788 |
chore(deps): bump js-yaml to 4.3.1 in sim-cli and drop the aged-out release-age waivers (#6784)
js-yaml < 4.3.1 has quadratic CPU consumption in !!omap resolution (GHSA-5p4m-2wfm-xmqj / CVE-2026-59870). sim-cli builds with --packages=bundle, so its dev-scoped js-yaml is bundled into the published CLI. 4.3.1 is already what apps/sim pins, so this collapses sim-cli onto the hoisted copy. The minimumReleaseAge waivers for js-yaml and mermaid were temporary and have both aged past the 7-day window; leaving them behind would disable the supply-chain gate for those packages indefinitely. |
||
|
Waleed
|
746a4496ba |
chore(deps): upgrade next to 16.3.1, its optimizer no longer deletes live code (#6777)
* chore(deps): upgrade next to 16.3.1, its optimizer no longer deletes live code 16.3.0 was reverted in #6242 because its Turbopack optimizer modelled a bare `return <asyncCall>()` tail call inside an async function as returning the promise object, propagated that always-truthy fact through the caller's `await`, and deleted everything after the resulting `if`. That shipped two dead code paths to production: the whole `POST /api/credentials` create path, and the insert inside `upsertAsyncToolCall`. We reported it as vercel/next.js#96595. The fix — "[turbopack] Collapse nested promises in the analyzer" (vercel/next.js#96601) — folds `Promise<Promise<T>>` to `Promise<T>` in the analyzer, and was backported as #96675 and released in 16.3.1. Verified before taking the bump: - The minimal reproduction from the issue no longer reproduces on 16.3.1. All four routes keep their code; on 16.3.0 `/api/broken` lost everything after the `if`. - A production build of `apps/sim` on 16.3.1 still emits the markers whose disappearance was the original signal: `credential_connected` (43 files), `acquireOrganizationUserMutationLocks` (28), and the `upsertAsyncToolCall` insert-path warning (10). The `return await` hardening added to both sites in the revert stays as is, and so does the TypeScript toolchain configuration. 16.3.1 published 2026-08-13, so it is inside the 7-day `minimumReleaseAge` supply-chain window until 2026-08-20 and needs an exclusion to install. The alternative is sitting on 16.2.12, whose successor we already reverted once, so the entries go in dated and come out on the next touch of the file. The mermaid and js-yaml exclusions aged out on 2026-08-11 and 2026-08-07 and are dropped here per that same rule. * fix(deps): keep the musl and win32 SWC binaries in the lockfile The release-age exclusion only listed the four @next/swc platforms that package.json pins, but next declares all eight as its own optionalDependencies, so all eight are normally resolved into bun.lock. A gated optional dependency does not fail the install — bun drops it silently — so the first install stripped both musl variants and both win32 variants from the lockfile. That left the Alpine devcontainer and any Windows machine with no SWC binary to resolve. Adding the remaining four to the exclusion list restores all eight entries at 16.3.1. Worth knowing for the next time this happens: bun.lock is sticky here. Once an optional dependency has been dropped, re-running the install — even with --force, even with the age gate switched off entirely — does not bring it back, because the resolution is not reattempted. The lockfile has to be regenerated from a base that still contains the entries, which is why this restores bun.lock from staging before re-applying the bump. |
||
|
Waleed
|
025ea4d2bd |
fix(docs): serve JSON-LD in the HTML, fix sidebar spacing, and tighten the CLI guides (#6763)
* docs(cli): use -g for the install, and cut the prose that was not pulling weight
`--global` is valid but `-g` is what every comparable CLI documents, and the
long form only came from the package README. Also drops the yarn tab: it read
`yarn global add sim`, which works on Yarn 1 only — Yarn 2 removed global
installs, so that command fails for anyone on a modern Yarn. Adds `npx sim` for
running without installing.
The guides had accumulated design rationale that belongs in code comments rather
than user docs — why the filter grammar is JSON, why the config section naming
is asymmetric, why an unexpected error keeps its stack trace. Surveying how gh,
Vercel, Turborepo, Deno, Bun and Supabase write theirs, none carry that kind of
justification, and callouts are reserved for content whose absence produces a
wrong result rather than for general asides.
So: 1016 lines to 763, and 12 callouts to 3. The three that remain are the
pairing-code check, that `sim logout` does not revoke the key, and the
`--limit 100` default on `batch-delete`/`batch-update`, which silently truncates
a larger match. Troubleshooting drops the entries whose error message already
contained its own fix and keeps the seven whose cause is not obvious.
* fix(docs): render JSON-LD as native script tags so it reaches the HTML
All four structured-data blocks — WebSite, TechArticle, BreadcrumbList,
SoftwareApplication — were rendered with `next/script`, which never emitted a
script tag. Measured on a production build, `/api-reference/getting-started`
contained zero `<script type="application/ld+json">` elements; the payload
existed only in the `__next_s` client-injection queue and the RSC flight data,
so anything reading the served HTML saw no structured data at all. React was
also logging "Encountered a script tag while rendering React component" on every
page.
`next/script` is for loading and executing JavaScript. JSON-LD is data, and
Next's own guidance is a native `<script>` in the component. `serializeJsonLd`
already escapes the `<` character to its unicode form, which is the
sanitization that guidance calls for, so only the element changes.
Same build, after: three valid tags per page with `WebSite` in `<head>`, and the
injection queue gone entirely.
* fix(docs): scope the flush-separator rule to a container's first separator
`[data-separator]:not([data-separator] ~ [data-separator])` was meant to keep the
first sidebar group flush against the top padding, but `~` only reaches siblings,
so it also matched the first separator inside every expanded folder. Under
Self-Hosting, "Install" lost its top margin and crowded the "Architecture" link
above it — 25px of gap where "Configure" and "Operate" below it had 40px.
`:first-child` expresses the intent directly. Only the four sidebar roots open
with a separator; every nested folder starts with a page, so the intended case
still goes flush and nothing else changes.
* fix(docs): move the flush-separator rule onto the separator component
Keeps the styling with the component that owns it, per the repo standard, and
lets the global rule be deleted outright rather than corrected — `global.css`
now only loses a rule in this PR. Tailwind's `first:` variant compiles to the
same `:first-child` selector, so behavior is unchanged: the build emits
`.first\:mt-0:first-child{margin-top:0}` and the prerendered HTML carries the
class on the separator.
|
||
|
Waleed
|
fed891f69d |
docs(cli): add a CLI docs section generated from the command tree (#6762)
* docs(cli): add a CLI section, generated from the command tree
The `sim` CLI shipped with no coverage in the docs site. Adds a fourth
top-level tab for it, and moves Academy last.
The command reference is generated. `sim` exposes 147 leaf commands across
33 groups, most of them derived at runtime from the v2 route contracts, so a
hand-written reference would be wrong the week after it was written. The
generator walks the command tree `buildProgram()` hands to commander — the
same tree the terminal parses — rather than re-deriving it from the contract,
which would be a second implementation free to describe commands nobody can
invoke. `check:cli-docs` is a zero-arg `check:*` script, so the existing audit
runner picks it up and stale pages fail CI.
Generating against the real tree surfaced a collision it had been hiding:
`bulkUpdateKnowledgeDocuments` and `updateKnowledgeDocument` both derived to
`sim knowledge documents update`. Commander resolves a duplicate to the first
match, so the bulk form shadowed the single-document one and its flags were
unreachable while still appearing in `--help`. The bulk form is now
`batch-update`, matching how `tables rows batch-delete`/`batch-update` already
handle the same REST overload, and the generator fails on any duplicate path
so the next one cannot land silently.
Five hand-written guides cover install, auth, configuration, output formats,
and scripting. Also corrects two commands in the package README that do not
exist as documented (`tables columns <tableId>`, and `--sort score:desc`,
which is JSON).
* docs(cli): document every flag from the contracts, add troubleshooting and a single-page reference
The command reference was structurally complete but said almost nothing: 223 of
377 flags rendered as "Set sort by" because the CLI only ever read flag help
from its own contract overrides, and fell back to restating the flag name.
The prose already existed. The v2 route contracts carry 931 `.describe()` calls
and the OpenAPI specs publish all of them — 327 parameters and 282 body
properties, 100% coverage — but the generated operation table dropped every one,
carrying only a per-operation summary. It now carries the field descriptions,
the path-parameter descriptions, and positional help, so `--help` and the docs
explain a flag the same way the API reference does. Placeholder descriptions are
now zero, and 147/147 commands, 377/377 flags and 130/130 arguments are
documented.
`check:cli-docs` fails on a request field with no `.describe()` rather than
letting it render as documentation that says nothing.
Also in this pass:
- Commands are root-level sidebar entries under a Commands heading rather than
a folder, and headings are the command's description, so the table of
contents distinguishes entries at the first word instead of repeating
"sim knowledge documents …" fourteen times. A guard fails the build if two
descriptions on a page collide, since they would share an anchor.
- A single-page `Complete reference` carrying all 147 commands, for in-page
search and for agents fetching `/cli/reference.mdx`. It keys on exact command
paths because descriptions are only unique within a group.
- A troubleshooting page, with every message copied from the source.
- Table columns are sized by a local component; the flag column was starved
while descriptions kept most of the row empty.
- The prerelease install channels are dropped from the docs and the package
README, which is what npm renders.
* fix(docs): match the CLI tab by path segment, and escape backslashes before pipes
`pathname.includes('/cli')` also matches `/integrations/clickup` and
`/integrations/clickhouse`, so both existing integration pages lit the CLI tab
and unlit Documentation. Matching is now per path segment. Anchoring to the
start would not work either — a non-default locale prefixes the path, as in
`/ja/cli` — so the segment is matched wherever it sits.
Table cells now double a backslash before escaping pipes. A value ending in one
turned `a\` + `|` into `a\\|`, which the table parser reads as an escaped
backslash followed by an unescaped pipe, splitting the cell early. Nothing in
the command surface contains a backslash today, so this was latent rather than
visible.
The reference page's global options table is two-column and was being wrapped in
`CommandTable`, which sizes the second column for the `Required` cell of the
three-column tables and crushed the description into 5.5rem. It now matches the
overview page, which leaves that table unsized.
|
||
|
Waleed
|
4f722c6439 |
fix(deploy): resolve the error-output flag from edges on both sides of change detection (#6754)
* fix(deploy): resolve the error-output flag from edges on both sides of change detection A block can hold `errorEnabled: false` while an error edge still leaves it. `setBlockErrorEnabled` does not remove existing error edges, both block renderers draw the port on `errorEnabled || hasErrorConnection`, and the executor never reads the flag at all — the edge alone decides routing. So the two spellings are one state, and nothing about the block has functionally changed. Only the deployed side reconciled them. `materializeDeploymentState` backfills `errorEnabled: true` for any block with an error edge, while the live normalized tables are read verbatim. Change detection compared the raw flag, saw `true` against `false` for a block that had not changed, and no redeploy could clear it: deploying snapshots the live `false`, which the next read backfills straight back to `true`. The deploy button sat on "Update" permanently, and the server path (`checkNeedsRedeployment`, which reads the raw jsonb and skips the backfill) disagreed with it. Lift the rule into `@sim/workflow-types` as `collectErrorSourceBlockIds` / `resolveEffectiveErrorEnabled` so the backfill and the comparison share one definition, and apply it to both sides of the diff. Compared outside the structural gate, since the flag can match while the edges disagree. * chore(deploy): use TSDoc for the error-output comparison comments |
||
|
Theodore Li
|
623c30f05c |
fix(credentials): restore reliable OAuth connections (#6753)
* fix(credentials): restore reliable OAuth connections * fix(credentials): cover OAuth handoff edge cases * fix(credentials): preserve reauthorization outcomes |
||
|
Waleed
|
4fc0fb4bad |
refactor(resources): converge Files onto the shared drag hook and batch bulk authorization (#6748)
* refactor(resources): converge Files onto the shared drag hook and batch bulk authorization Files kept a 280-line copy of the foldered-list drag logic because it also accepts OS file drops. The copies had already drifted, so the external drop becomes an option on the shared hook and the copy goes away. - Add `externalDrop` to `useFolderRowDragDrop`: folder rows highlight and spring open for an OS file drag exactly as for a move, while the body and breadcrumb decline so the page-level upload overlay owns those regions - Collapse the three drop-active booleans into one `ActiveDropTarget` union, so exactly one affordance is armed by construction rather than by hand-clearing - Keep drop-target writes identity-stable so `dragover` does not re-render the list on every event - Give each list its own drag MIME again, restoring the cross-surface isolation `drag-payload.ts` documents - Let a folder spring open more than once per drag, so a drag can walk back out through the breadcrumb and descend again; the guard against re-entering the folder already on screen moves to `useSpringNavigation`, the only layer that can state it - Resolve each bulk item against the workspace context the batch already holds, and memoize the effective-permission lookup for the batch, replacing two invariant queries per item - Fill the drop target at `--surface-active`: `--surface-4` is the button-base token and is lighter than hover in light mode, so the strongest row state read the faintest * fix(resources): dismiss the upload overlay on a folder drop and re-check permission per item The drag hook stops propagation on a drop it handles, so the page-level handler that cleared the upload overlay never ran and the chrome stayed up over the finished upload. Both consuming paths now share one dismissal. Drop the batch permission memo: each item in a bulk move or delete commits independently, so reusing one allow verdict let a revocation part-way through a batch go unseen by the remaining items. The workspace context is still resolved once per batch, which was the larger saving. |
||
|
Waleed
|
0cd87bad2f |
feat(resources): multiselect on tables and knowledge, spring-loaded folders (#6721)
* feat(resources): multiselect on tables and knowledge, spring-loaded folders Tables and Knowledge lists get the checkbox multiselect Files already had — selection, shift-click ranges, select-all, and a shared bulk action bar for move and delete. Dragging a resource onto a folder row and resting there now opens that folder, so nested filing is one gesture (macOS Finder spring-loading). Works on Files, Tables, and Knowledge. Selection, the action bar, the drag payload, the drag ghost, and drag teardown are extracted to shared modules; Files migrates onto them rather than keeping its own copies. Bulk move and delete land as single authorized operations that take folders and resources together, so a mixed selection commits once instead of fanning out. Fixes two latent UI bugs: the drop-target outline referenced --accent, an HSL-channel token only valid via hsl(), so it silently rendered as currentColor; and rows painted hover and selected with the same surface token, making the two states indistinguishable. * chore(audits): re-record route ratchet after merging staging * fix(bulk): reject a move target inside the moving subtree and report contained folders deterministically * improvement(resources): neutral drop affordance, longer spring delay, and a body drop target * fix(resources): drop into the open folder on Files, return on an unused spring-open, and guard bulk caps * fix(drag): end a drag on pointer resume instead of an idle timer * feat(resources): drag onto breadcrumbs to move back up, and round the drop ring * fix(resources): tint the list region on drop instead of ringing it * refactor(folders): share the spring-navigation lifecycle so Files returns too * fix(breadcrumbs): accept a drag on the open-folder crumb too * fix(files): keep the view in a spring-opened folder when an OS upload lands there |
||
|
|
337a53f12c |
feat(cli): Sim CLI with AWS-style profiles and a platform key exchange (#6147)
* improvement(api): pull in the v2 external endpoint surface Cherry-picks improvement/v2-endpoints ( |
||
|
Theodore Li
|
b198080356 |
feat(credential-groups): complete managed account enrollment (#6729)
* fix(credential-groups): show reconnect after authorization * feat(slack): include managed user auth in custom bots * feat(credential-groups): add enrollment completion page * fix(credential-groups): align settings order and block color * fix(credential-groups): delete credentials on access revoke * fix(credential-groups): use person icon for enrollments * fix(credential-groups): keep enrollment submit visible * fix(credential-groups): make enrollment connections optional * fix(credential-groups): simplify people actions * fix(credential-groups): preserve pagination after deletion * fix(credential-groups): delete removed enrollments * fix(credential-groups): hydrate canvas labels |
||
|
Theodore Li
|
8fb571ac54 |
feat(credentials): add managed credential groups (#6697)
* feat(credentials): add managed credential groups * fix(audit): sync credential group mock * fix(credentials): serialize enrollment revocation * fix(credentials): isolate managed delegation * fix(credentials): serialize invitation lifecycle * fix(credentials): preserve enrollment lifecycle * refactor(credentials): migrate groups to application boundary * fix(credentials): serialize enrollment readiness * fix(credentials): preserve completed reconnect state * fix(credentials): revalidate policy before grant persistence * fix(credentials): prioritize expired invitations * fix(credentials): redirect unavailable oauth starts * fix(credentials): clarify managed oauth boundaries * fix(settings): complete feature flag test mocks * fix(credentials): clarify enrollment actions and entitlement errors * fix(credentials): preserve entitlement failure reasons * fix(credentials): refine managed oauth flow * fix(lint): use optional chain for pagination |
||
|
Waleed
|
2223b63851 |
chore(utils): consolidate record guards and remove dead code (#6715)
* refactor: consolidate local isRecord guards onto shared isRecordLike Nineteen files had re-declared a local `isRecord` guard rather than using the shared one from `@sim/utils/object`, drift that reappeared after #5061 first consolidated them. Two more imported the shared guard under an `isRecordLike as isRecord` alias. The copies were not interchangeable. Nine matched `isRecordLike` exactly. The rest omitted the array exclusion (`typeof x === 'object' && x !== null`, or `Boolean(x) && typeof x === 'object'`), so arrays passed the guard. Each of those call sites was reviewed individually: in every case the guard is followed by string/number field checks that an array fails anyway, so the outcome is unchanged. The one exception is `isOptionsTagData`, where `Object.values` on an array of option items really did make an array-form `<options>` tag render. It now accepts arrays explicitly rather than by accident. `executor/handlers/pi/search/extension-source.ts` keeps its own copy: it is source text written into an E2B/Daytona sandbox at runtime and cannot import. * refactor: replace inline record guards with isRecordLike 103 inline `typeof x === 'object' && x !== null && !Array.isArray(x)` guards (and the `x &&` / `Boolean(x)` spellings of the same conjunction) now call the shared guard. Inside a conjunction that already asserts `typeof x === 'object'`, `x &&` and `x !== null` are interchangeable, so all three orderings are the same predicate at runtime. Only exactly-equivalent conjunctions were converted. Matching required the three clauses to be one adjacent conjunction over the same operand, so a nearby but unrelated clause cannot be absorbed — generic-handler.ts, where the array case is handled inside the block rather than excluded by the guard, is correctly left alone. Two sites were reverted after type-check rejected them: instagram/server-utils.ts and workflows/[id]/log/route.ts both cast straight to a specific interface, which is legal from `object` but not from `Record<string, unknown>`. Their narrowing is genuinely not identical, so they keep the inline form rather than acquiring a double cast. Left as-is: `packages/ts-sdk` (published with no runtime dependencies) and the two sandbox sources written into E2B/Daytona as text, which cannot import. * refactor: consolidate duplicate record coercion helpers onto @sim/utils A second sweep found 28 more local record helpers hiding under names the previous `isRecord` grep never matched — `asRecord`, `toRecord`, `toRecordOrNull`, `asObject`, `isJsonObject`. rabbitmq defined the same `asRecord` twice within one service; dynatrace had three variants in one file. Fourteen of them were re-deriving the same two shapes, so those shapes now live in `@sim/utils/object` beside the guards they wrap: toRecord(value) // isRecordLike(value) ? value : {} toRecordOrNull(value) // isRecordLike(value) ? value : null Both preserve identity on a hit, so no call site starts copying. Eighteen local definitions are gone. `tools/instantly/utils.ts` keeps its exported `asRecord` because its return type is the local `JsonRecord` alias, but its body now delegates. `app/api/mcp/serve/[serverId]/route.ts` had an `isJsonObject` with zero call sites — deleted outright. Six helpers were deliberately left alone because they are NOT equivalent: `pagerduty`/`zendesk`/`gitlab` do `(value as Record) || {}`, which type-checks nothing at all, and `copilot/resources/extraction.ts`, `edit-workflow/validation.ts`, `pi/core/events.ts` omit the array exclusion. Those sit on webhook ingress and copilot paths where tightening is a behavior change, not a cleanup; they are audited separately. Two guards were removed rather than substituted, each proven dominated by an earlier check: the bedrock streaming `toolUse.input` guard was unreachable (`parseToolInput` already throws on non-objects before the loop builds `assembledToolUses`), and four `driver.ts` re-narrows follow an `if (!isRecordLike(x) || ...) throw` that dominates the later use. * fix(webhooks): guard non-string GitLab ref, and consolidate the last record helpers Two audits covered the six helpers held back from the previous commit for not being equivalent to isRecordLike. Five are now migrated; one is deliberately not. `gitlab.ts` carried a real crash path, independent of the guard work: const ref = (b.ref as string) || '' const branch = ref.replace('refs/heads/', '') The cast is unchecked and `|| ''` only catches falsy values, so a truthy non-string body field — `{"ref": 12345}` — reaches `.replace` and throws `TypeError: ref.replace is not a function` inside `formatInput`. That runs in the background worker after the webhook is already 200-ACKed, and GitLab does not auto-retry, so the delivery is lost silently. Now checks `typeof`. `pagerduty`/`zendesk`/`gitlab` each defined `asRecord` as `(value as Record<string, unknown>) || {}`, which type-checks nothing — a string or array passed through and was then spread into the workflow trigger payload (`gitlab.ts:114`) as character- or index-keyed garbage. All three now use the shared `toRecord`. These sit behind `verifyProviderAuth`, so reaching them requires the shared secret; this is robustness, not authorization. `copilot/resources/extraction.ts` and `pi/core/events.ts` were array-permissive but provably inert — extraction.ts has no key enumeration or spread anywhere, and events.ts only diverges by returning `null` instead of `{type:'other'}` for an array, which every consumer already no-ops on. Pinned with a test. `edit-workflow/validation.ts` is left permissive ON PURPOSE. Its `Object.entries` walk mirrors the unguarded walk in `operations.ts:86,191`, so an array-shaped `nestedNodes` from the model is currently visited by both. Tightening only the validation side would stop `collectHostedApiKeyInput` from stripping platform-managed API keys while the apply side still creates those child blocks. Both paths have to change together, with tests, in their own PR. * refactor: delete 31 unreachable module-local functions Removes 815 lines of provably dead code: module-local (non-exported) declarations whose identifier appears exactly once in their own file — the declaration itself. A non-exported symbol cannot be reached by an import, a barrel, a dynamic import, or a framework convention, so "unreferenced in its own file" is a complete proof of deadness rather than a heuristic. Notable removals include whole abandoned code paths: `findWebhookAndWorkflow` (78 lines), `calculateBillingProjection` and `initializeUserUsageLimit` (80 lines), `removeCredits` + `deductFromCredits` (54), `sendBatchSMS`, `executeToolBatch`, and four unused `async-runs/repository.ts` queries. Spans come from the TypeScript AST, not a regex. A regex cannot find a declaration's extent — a first-brace scan cuts inside a return-type annotation such as `Promise<{ canCreate: boolean }>` and silently corrupts the file. The AST pass also re-derives deadness from identifier nodes, which caught two wealthbox helpers that a regex export-check had wrongly reported as local. Note the runtime TypeScript API here is `@typescript/typescript6`; the bare `typescript` specifier resolves to the native compiler, which exposes no `createSourceFile`. * refactor: delete 434 stranded exported symbols Removes ~5,930 lines of unreachable code across 166 files: exported symbols that no other file in the repo mentions and that are unused inside their own file. Whole abandoned surfaces go with them — unused React Query hooks (useOrganizations, useOrganizationMembers, useUpgradeSubscription, ...), unused admin route contracts, dead executor constants and reference builders, and the landing-page StageWorkflow/LandingPreviewMount components. Three files left with no remaining code were removed outright. Candidates came from an AST pass; each was then verified individually against an UNFILTERED repo-wide search plus the reachability paths a name search misses: string-keyed tool/block registries, dynamic imports, `export *` barrels, and the docs generator's source-text parsing of tool files. 29 candidates were verified LIVE and kept. Those exposed a flaw in the candidate generator: it indexed only .ts/.tsx, while apps/docs/content/**/*.mdx imports React components directly — ActionImage appears in ~190 MDX pages and ActionVideo in ~115, and both scanned as dead. `app/global-error.tsx` was likewise kept, since Next.js reaches it by filename and its default export can never have a name reference. All 434 deleted names were afterwards cross-checked against every .mdx/.md/.json/.yaml in the repo: no hits. Verified with turbo type-check (23 workspaces), the full apps/sim suite (25,219 tests), all 26 audits, and a production `next build` — the last of these being what actually exercises route- and component-level reachability. |