184 Commits

Include renames

Author	SHA1	Message	Date
Theodore LiandClaude Opus 5	c5cc6ce26c	feat(chat): hide the Chat module when NEXT_PUBLIC_CHAT_DISABLED is set (#6137) ... * feat(chat): hide the Chat module when CHAT_ENABLED is unset A self-hosted deployment that skipped the chat key still rendered the full mothership Chat UI, landing on the composer and 401ing on every message. Gate it behind a CHAT_ENABLED / NEXT_PUBLIC_CHAT_ENABLED twin, written by the setup wizard alongside COPILOT_API_KEY and validated by the existing FLAG_TWINS doctor check. The flag resolves at module scope on both render passes, so no chat surface renders then disappears. With Chat off the workspace lands on its first workflow (resolved server-side, behind the cached host-context check so no workflow id leaks to non-members), and the chats list, scheduled tasks, editor Chat panel, and chat CTAs are absent. Routes are gated rather than deleted: /home redirects because it is baked into delivered invitation emails and the accept contract. Also fixes two bugs the gate exposed: a persisted activeTab of 'copilot' left the workflow panel blank from first paint, and the panel's handoff listener claimed MOTHERSHIP_SEND_MESSAGE events outside its own gate, silently swallowing "Fix in Chat" messages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ErcRgvi7VQBeKDQ3MBMha * refactor(chat): gate the UI on NEXT_PUBLIC_CHAT_DISABLED, not an opt-in flag CHAT_ENABLED made Chat opt-in, so every existing deployment that already had COPILOT_API_KEY would have lost the module until it set a new variable. Invert to an opt-out so nothing changes for them. That also collapses the twin. The only reason the flag needed a server/client pair was that it projected a secret; NEXT_PUBLIC_CHAT_DISABLED is not one, so getEnv resolves the same value from process.env on the server and window.__ENV in the browser. Gone with it: the FLAG_TWINS entry and its doctor sync check, the two-variable wizard write, and the boot-time throw, whose contradiction (flag on, key absent) can no longer be expressed. Presentation and capability are now separate concerns. NEXT_PUBLIC_CHAT_DISABLED decides whether the surfaces render; COPILOT_API_KEY decides whether the work can run, and gates the paths that need it — the Sim Chat block, prompt-job claims, and inbox access — each failing on its own terms. The wizard writes the opt-out when you skip the chat key, which is the case this started from: a fresh self-host that never configured Chat. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ErcRgvi7VQBeKDQ3MBMha * feat(setup): prompt for the chat key in k8s mode The dev and compose flows minted a chat key and wrote the Chat opt-out alongside it; k8s did neither, so a cluster install with no COPILOT_API_KEY in its Helm values rendered a Chat module that rejects every message. Prompt with the same flow and feed both values into `app.env`, which the chart already renders as arbitrary container env. Reading the previous release's key matters here in a way it does not for the file-based modes: `helm upgrade` without `--reuse-values` keeps only what this document carries, so a key the user elects to keep has to be re-supplied or it is silently dropped. Splits the release-values read from the secret-reuse check so both the key and the secrets come from one `helm get values` call, and carries the mothership override across for the same mint-here-validate-there reason the other modes document. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ErcRgvi7VQBeKDQ3MBMha * fix(setup): write app-behavior flags to every env file the app can start from The wizard wrote the Chat opt-out only to the env file its own mode owns, so choosing compose put it in the root `.env` while `bun run dev` reads `apps/sim/.env` and never saw it. Skipping the chat key appeared to do nothing. Mirror values that change how the app behaves — as opposed to where it connects — across both targets. Connection settings deliberately do not go through this: DATABASE_URL and friends differ between the compose stack and a local dev run, which is why this takes an explicit set of values rather than the whole batch. The mirrored file is written even when absent, since missing is exactly the case that stranded the flag, but with seeding suppressed so a compose run leaves a one-line apps/sim/.env instead of a full .env.example for a stack the user is not running. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ErcRgvi7VQBeKDQ3MBMha * fix(compose): forward NEXT_PUBLIC_CHAT_DISABLED to the app container The wizard wrote the flag into the root .env, but compose only passes through variables the service's `environment` block names — and that block listed COPILOT_API_KEY without its companion. Skipping the chat key on a Docker install therefore did nothing: the value sat in .env and never reached the container. Add the passthrough to all four compose files. Reverts the previous commit's mirroring into apps/sim/.env, which treated the symptom — each mode writes only the env file it owns, and that file is now wired correctly. k8s needs no equivalent: its values flow into `app.env`, which the chart renders key by key. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ErcRgvi7VQBeKDQ3MBMha * fix(chat): resolve the landing route without blocking on the database Server-resolving the first workflow meant a session lookup, an access check and a query had to finish before anything rendered. A slow or unreachable database left the user on a blank page under a populated sidebar — worse than the instant redirect it replaced, and with no signal that anything was wrong. Redirect straight to `/w` instead and let it pick from the workflow list the layout already prefetches, so the choice costs no round trip and cannot hang. Repoints the sidebar's primary action rather than hiding it: the slot that offered "New chat" now offers "New workflow" and creates one, since with Chat off there is no composer to open but the intent is the same. Sends the CLI key handoff to signup rather than login. It is reached from a terminal — usually the setup wizard standing up a fresh self-host — where the visitor has no account yet. Both auth pages cross-link carrying the callback, so a returning user is one click from login with their destination intact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ErcRgvi7VQBeKDQ3MBMha * improvement(chat): address cleanup-pass findings on the Chat gate Effects: the panel's auto-select effect read the copilot chat list while the list query was deliberately skipped, took "empty" for "deleted in another tab", and cleared the user's selection — latching a ref that stopped it ever being restored. Guarded on the same condition as the handoff listener. Memo: `/w` filtered workflows through a useMemo whose array dependency was a fresh `[]` on every render while the query had no data — the exact window the page exists for — so it memoized nothing and re-fired the redirect effect. Keyed on the workflow id instead. Same unstable-default problem on the sidebar's chat list, where it invalidated five downstream memos; given a stable empty constant. Callback: `handleCreateWorkflow` listed the whole mutation object in its deps, which TanStack recreates every render. Harmless until this branch wired it into the top nav, where it defeated `memo(SidebarNavItem)`. React Query: Recently Deleted still fetched archived chats unconditionally and offered restores into routes that now 404. Also surfaces an error state on `/w` — it is the landing route now, so a failed list fetch would otherwise spin forever behind a log line — fixes a spinner using a token undefined in dark mode, and trims comments that restated code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ErcRgvi7VQBeKDQ3MBMha * fix(chat): gate workflow creation on write access, pin the key in schedule tests The zero-workflow landing offered "Create workflow" to every member. Creation navigates optimistically, so a read-only member was sent to a workflow the server had already refused to create, with the failure never surfaced. Gate both entry points — the empty state and the sidebar's "New workflow" row — on the same `canEdit` check the rest of the sidebar uses, and tell read-only members who can make one instead of offering an action that cannot succeed. The schedule-execution tests only passed locally because vitest loads the developer's own `.env`, which supplied COPILOT_API_KEY; CI has none, so the prompt-job claim guard skipped the claims those cases assert on. Pin the key through the env mock so the suite states its own preconditions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ErcRgvi7VQBeKDQ3MBMha * fix(setup): name both variables in the chat-key failure hint The caller writes the Chat opt-out whenever the prompt returns no key, so the hint's "or set COPILOT_API_KEY yourself" restored capability while leaving the module hidden — the one path where following setup's own advice does not work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ErcRgvi7VQBeKDQ3MBMha --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-31 19:23:24 -04:00
Waleed	f0b79c5cc6	chore(deps): upgrade next 16.2.11 -> 16.2.12 (#6077) ... 16.2.12 is the current stable (published 2026-07-25) and its entire changelog is two PRs: a docs backport and vercel/next.js#95831, "Fixes to support TypeScript 7". That second one matters here. `apps/sim` declares `typescript: ^7.0.2` and the lockfile resolves 7.0.2, while 16.2.11 predates any TS7 handling — not even the actionable-error guard (#95837), which was never merged. The upstream symptom is `next build` dying with a silent SIGSEGV during its type-check step, because the legacy TypeScript JS API that Next called is gone in TS7. Builds pass today only because Next detects @typescript/native-preview as the compiler and takes a different path, so we are accidentally-working rather than supported. 16.2.12 adds the `experimental.useTypeScriptCli` backend that makes this configuration official. Zero build-performance content in the patch, so this is not a speed change. Bumps all eight pins in lockstep — next, @next/env and the four @next/swc-* binaries at the root, plus the three app/package copies. The swc binaries must move with next: they are platform-gated optionalDependencies, so a version skew or a gate exclusion leaves them out of bun.lock entirely and `bun install --frozen-lockfile` installs no compiler at all (the #5945 failure). Also re-dates the bunfig.toml gate note, which said to drop the next entries on 2026-07-28 — yesterday. 16.2.12 is inside the 7-day window until 2026-08-01, so following that instruction would have blocked this bump and re-triggered the missing-compiler failure. Re-date on future bumps rather than deleting the entries early.	2026-07-29 18:10:50 -07:00
5 people	1d64b92b41	feat(desktop): desktop app (#5998) ... * top on a desk * fix auth stuff * intermediate state * update * local filesystem fixes * Huge * fix banner * ci: disable desktop release + e2e in CI for now The desktop-release reusable-workflow call requested contents: write, which ci.yml's permission grant (contents: read) rejects — invalidating the whole CI workflow. Desktop is tested locally for now; signed builds remain available manually via desktop-release.yml workflow_dispatch, and desktop e2e via its own workflow_dispatch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci: exempt electron from the release-age gate (time-boxed) electron@43.1.1 (published 2026-07-14) is exact-pinned for the desktop shell and blocked by minimumReleaseAge until 2026-07-21. Excluded with a drop-after date, following the vetted-typescript precedent. Verified the rest of the desktop dependency set clears the 7-day gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * desktop: brand app icon (packaged + dev Dock) - build/icon.icns regenerated from public/logo/primary/large.png on the Apple icon grid (824px body, r=185.4, centered on a transparent 1024 canvas), compiled with iconutil - dev runs set the same mark via app.dock.setIcon (static/dock-icon.png) — unpackaged Electron otherwise shows its default atom icon - un-ignore apps/desktop/build: it holds electron-builder INPUTS (icon, entitlements), which the /apps/**/build output rule was swallowing — the icns and entitlements were never actually tracked - revert resetAdHocDarwinSignature fuse: it corrupts the packaged binary signature (app killed at launch on arm64); the local ad-hoc deep-sign flow doesn't need it Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * desktop: switch app icon to the b&w brand mark White rounded tile with the black sim wordmark (from public/logo/b&w/large.png), replacing the purple variant. Same Apple icon grid geometry (824px body, r=185.4, 1024 canvas). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix banner * Fix * clean up launcher * fix oauth * update desktop app * Improve browser use and consolidate desktop app * Desktop app ui cleanup * Updates * Updates * remove dev tool option * Browser updates * Fix electron bug * Browser shortcuts * lifecycle * feat(desktop): SSRF hardening + shared @sim/security/ssrf (re-home of #5763) (#5784) * feat: re-home @sim/security/ssrf + sim SSRF dedup onto dev (clean core) * feat(desktop): re-integrate SSRF guard + hardening onto rewritten dev Re-applies the browser-agent SSRF guard and hardening onto dev's evolved desktop files (dev rewrote session/driver/handoff/index and split out errors.ts/keyboard.ts): - session.ts: agent-partition onBeforeRequest is the SSRF choke point — DNS-resolving check (fail-closed) for document navigations, synchronous literal-IP backstop for subresources. - driver.ts: browser_navigate/browser_open_tab validate via checkAgentUrl for a clean model error; also adopt shared sleep/getErrorMessage and drop the local reimplementations + banner separators. - index.ts: local-only crashReporter (native minidumps, no upload) + CSP fallback wired into the app session. - window.ts: record the crash-dump dir on renderer_gone. - config.ts: drop the local LOCAL_HOSTNAMES set for the shared isLoopbackHostname (also removes the dead bare '::1'). - cdp.ts: per-WebContents callbacks so a background tab's events reach its own driver. - updater.ts: the manual check now surfaces network/manifest failures instead of silently swallowing them. - README: correct the App Sandbox / security-scoped-bookmark note. - electron-mock: webRequest.onBeforeRequest + crashReporter stubs. - api-validation: annotate dev's validated-envelope double-cast; bump the route-count baseline 964→965 for dev's already-merged route (ratchets stay tight; non-Zod and double-cast at baseline). Skipped as moot (dev already did them independently): launcher isVisible removal, decideStartRoute param drop, local-filesystem clear() removal. * chore(desktop): biome format install-local.ts (pre-existing dev lint failure) * refactor: apply audit cleanup (reuse + simplify) - domain-check: drop the redundant isIpLiteral guard (isLoopbackIp already validates and returns false for non-literals). - session.ts: use shared getErrorMessage instead of the local error ternary (the file already imports it). - tray.ts: use shared sleep() instead of a hand-rolled setTimeout promise. - updater.ts: distinguish the synchronous-throw log from the async-rejection log on the manual update check. * refactor: /simplify pass + review fixes - url-guard: bound the SSRF dns.lookup with a 5s deadline (fails closed on timeout) so a slow/hung resolver can't suspend the check and the onBeforeRequest callback indefinitely (Greptile P2); + test. - Finish the reuse consolidation the earlier pass missed: session.ts second error ternary → getErrorMessage; the bracket-strip idiom → unwrapIpv6Brackets in input-validation.ts, input-validation.server.ts (×2), onepassword/utils.ts (fixes the check:utils banned-pattern CI failure). - driver: document why the tool-level checkAgentUrl coexists with the onBeforeRequest enforcement seam (clean model error; loadURL rejection is swallowed). * fix(desktop): swallow late DNS rejection after the SSRF lookup timeout (Cursor) * refactor: split pure host helpers into @sim/security/hostnames (ipaddr-free) (#5787) unwrapIpv6Brackets + isLoopbackHostname move to a new ipaddr-free sub-export so client code can share them without pulling ipaddr.js into the browser bundle. ssrf.ts re-exports both, so its server/desktop consumers are unchanged. This eliminates the duplicate isLoopbackHostname in apps/sim/lib/core/utils/urls.ts: urls.ts and its three client importers (mcp queries, oauth probe, oauth url-validation) now use the single shared definition. * Desktop app fullscreen mode * fix(copilot): report closed browser session as a distinct terminal tool error A dead agent browser session used to answer every browser tool with an indistinguishable generic ~30s IPC timeout, which the model retried indefinitely (one turn: 59 minutes of failing browser_snapshot calls). - When the desktop app has reported the session closed, page-dependent browser tools fail immediately with an explicit session-closed message (and sessionClosed: true in the result data) instead of burning the full timeout per call. browser_navigate / browser_open_tab / browser_list_tabs still run, since they can start a new session. - A failure whose session died mid-call (e.g. during a takeover) gets the same tag appended, so the model learns the terminal cause rather than seeing a plain timeout. Companion to mothership's tool_failure_loop circuit breaker. * fix(desktop): route Cmd+W to focused browser tabs * fix(desktop): reserve macOS title bar safe area * fix(desktop): limit title bar safe area to login * fix install script * feat(desktop): improve local folder settings * feat(desktop): harden local capabilities and window chrome * fix(invitations): live refetches * fix(desktop): make manual update checks use updater state * fix(desktop): review fixes — OAuth error handling, query freshness, invitations Findings from an end-to-end review of the desktop work, fixed and verified. OAuth connect/login handoff: - Add a friendly /oauth-error landing page + onAPIError.errorURL so provider Cancel/Deny (which Better Auth redirects before the flow state is parsed) no longer dead-ends on a 404; re-initiating supersedes the idle loopback. - Stop a post-consent failure from reporting success (drop the baked-in errorCallbackURL param that collided with Better Auth's appended code; coerce an array error defensively on the complete page). - Guard the desktop connect listener with the same context-age check the web routers use, so an abandoned flow can't mislabel a later completion. - Clear an orphaned pending handoff when a loopback re-bind fails. Query freshness (desktop refetchOnWindowFocus): - Pin refetchOnWindowFocus off on queries that seed editable forms (environment/secrets, credential detail, schedules) so a background focus refetch can't drop an unsaved draft, and on the useWorkflowStates fan-out so returning to a large table doesn't fire N heavy envelope fetches. All no-ops on web (default already false). Invitations (in-app pending invitations): - Map accept/decline failures to friendly copy instead of raw machine codes. - Invalidate subscription + refresh session on accept (parity with the email path); reconcile the list on failure (onSettled) so dead rows drop. - Gate the modal's query on open so it no longer fetches on every app load. CI: - Wrap the latest-mac.yml update-feed route in withRouteHandler and allowlist it as a non-boundary route (input-less, YAML) so the contract audit passes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * updates * fix(desktop): use workflow colors for environment icons * fix(desktop): use orange for dev icon border * fix(login): change one time token generation to GET * improvement(desktop): reveal local folders from settings Local-folder rows rendered their glyph at 20px inside the bordered credential tile — chrome meant for brand and logo icons — above a static subtitle that repeated what the section already said. The row now shows a plain 14px folder icon and the folder name alone. Clicking a row reveals the folder in the OS file manager through a new reveal_mount bridge op, which resolves the opaque localfs URI to a live grant and requires an active user gesture, matching the other grant mutations. The absolute host path still never crosses the bridge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C54QHj4WPV777Fq2yRwkcb * improvement(desktop): row actions menu for folder grants, larger version text Revoke moves from an always-visible chip into the canonical RowActionsMenu, matching the MCP server rows. The version value moves off text-caption onto text-sm — it was rendering at the subtitle size, which also shrank the "x -> y on restart" line that matters most. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C54QHj4WPV777Fq2yRwkcb * feat(desktop): improve browser tab usability * fix(desktop): thicken environment icon borders * fix(desktop): strengthen environment icon borders * feat(desktop): support multiple windows and harden the agent browser Sim can now open many full windows in one process. The embedded browser is still a single native surface, so exactly one window owns it at a time. Ownership transfers only to the focused window: without that rule, two windows both showing the browser reclaim it on every bounds heartbeat and re-parent the native view back and forth roughly once a second while Sim sits in the background, where no window is focused. A destroyed owner is now forgotten rather than left rejecting updates from the window actually on screen, and a closing window's release is honoured even though Electron destroys it before emitting `closed` — previously that release was dropped and the next layout could re-parent the browser onto a window that never asked for it. The agent's password boundary is now enforced rather than assumed. It was treated as settled but had four ways through: `browser_press_key` sent trusted CDP keystrokes to whatever held focus, `clickElement` focused credential fields, `readActiveElementState` returned a preview of any focused value, and snapshots printed the contents of revealed password fields. Detection also used `instanceof HTMLInputElement`, which is realm-bound and returned false for inputs inside same-origin iframes — the nested login forms that need it most. Detection now matches on tagName/type/autocomplete, the keystroke guard runs in the driver where trusted CDP input is visible, and typing re-checks the real target before inserting, since login forms advance focus between the username and password steps. Signing out clears the embedded browser's profile. Its cookies, cache, pinned tabs, browsing trail, and reopen list all survived sign-out, so the next account on the machine inherited the previous user's live sessions. Partition hardening is keyed per session instead of a process-wide flag, which would have left a second partition with no permission handlers, no SSRF filtering, and no download blocking — silently, and still type-checking. Adds the first tests for page-functions.ts, including a serialization contract check: those functions ship to the page as String(fn), so a reference to module scope passes every other test and fails only against a real page. * fix(desktop): close clipboard, glob DoS, and authorization holes Found by a full audit of the desktop app against origin/staging. Each of these was measured or asserted rather than reasoned about. The agent could read the user's system clipboard. `browser_press_key('Cmd+V')` pasted it into a focused field and the next `browser_snapshot` returned it as an ordinary `value` — snapshots redact password fields, not pasted content, and clipboards routinely hold a password copied out of a manager. The credential guard added earlier did not catch it: `insertedTextFor` returns undefined whenever `meta` is set, so `Cmd+V` was classified as not text-inserting. `Control+V` reached the same place because the macOS normalizer rewrites it. Clipboard combos are now refused before dispatch rather than by withholding the CDP `commands` array, since off macOS these are Blink-native and a key event alone still performs them. Copy and cut go too — they clobber the user's clipboard as a side effect. A glob pattern could freeze the whole app. Micromatch compiles to a backtracking regex whose cost is exponential in wildcard count: measured against a single 46-character path with the options this code passes, ten wildcards took 2.7s and twelve took 43s, once per scanned entry, in one synchronous call that the surrounding abort checks never get to interrupt. That is the main process, so every window, the menu bar and the tray freeze with Force Quit as the only recourse, and the pattern is model-supplied. `safeRegex` reports the generated source as safe, so it was no defense. Patterns are now bounded at six wildcards, which keeps the worst case near 2ms while leaving headroom over real patterns (which top out around four). A timing probe backs it up, with a budget loose enough that JIT warmth and machine load cannot make it fire on a legitimate pattern — a tight budget proved flaky in both directions. The grep authorization guard compared `request.pattern !== args.pattern`, so a tool call carrying no pattern made that `undefined !== undefined` and the guard passed — grep then fell back to searching the renderer's own `query` across the whole grant. `include` and `query` were never bound at all, letting a renderer widen a search or silently narrow results the agent believes are complete. The sibling glob case already had the `typeof` check, which is what made the asymmetry clearly unintentional. The IPC sender gate used `startsWith`, the exact pattern `isAppOrigin` warns against 200 lines away ("that prefix-matches lookalike hosts"). It was safe only because of a trailing slash. It now uses that helper, which also fixes a false negative on an explicitly stated default port. * fix(desktop): stop double sign-out, stranded retries, and redundant writes Three correctness bugs from the same audit. Menu Sign Out tore down the session directly instead of going through the lifecycle coordinator, so it skipped the in-progress guard — and its own cookie removal then tripped the coordinator's cookie watcher into a second concurrent teardown, duplicating the sign_out event, the storage clear, and the /login load. Teardown also existed as two divergent copies. The coordinator now exposes `signOut()` and owns the single path; the menu just calls it. That `tearDownSession` is no longer imported in index.ts is the check that it landed. Offline recovery could strand permanently. The auto-retry loop stops itself before calling `retry()`, and `retry()` never re-armed the load watchdog, which is started once per window. So if a retried load hung — precisely what the watchdog is for — no load event fired and no timer remained anywhere; the user sat on the offline page until the window was closed. `retry()` now re-arms before loading. Pinned tabs were persisted on `did-navigate` and `did-navigate-in-page` for every tab, pinned or not, with no change check, and the settings store compares with `===` so a freshly built array never matched. Any single-page app therefore triggered a synchronous mkdir + write + rename of the whole settings file on the main thread on every route change — writing `[]` over `[]` when nothing was pinned. The list is now fingerprinted, seeded at restore from what is already on disk so the first navigation after launch is not a write either. * fix(desktop): leaked timers, silent grep failures, and crashed tabs Second pass on the audit backlog, all verified against tests that fail without the change. Every browser tool call leaked a timer. The watchdog raced the tool against `sleep()`, which cannot be cancelled, so when the tool won — the normal case — the timer stayed pending for the full window, up to two minutes, dozens deep during an agent run. Replaced with a cancellable timeout cleared in a `finally`; a test asserts the fake-timer count is unchanged across a call. An invalid grep regex reported "no matches". A SyntaxError from `new RegExp` returned an empty result set, which tells the model the string appears nowhere in the user's files — a factual claim it acts on, when the search never ran. It now fails as INVALID_REQUEST. The `safeRegex` guard moved out of the try while there, since it was only inside it to be re-thrown. A crashed tab wedged the session. Tabs left `tabs` only via close, so a dead renderer stayed forever: `activeTab()` filtered it out while `activeTabId` still named it, making `requireTab()` report "no page is open" with other tabs open, and the panel went blank with no recovery. `render-process-gone` now drops the tab, advances the active id, and reports session closure when it was the last. `probeSession` cleared its abort timer inline after the await, so a thrown fetch — the case the function exists for — skipped it. Moved to `finally`, which also brings the body read inside the deadline. One vanished file failed a whole directory listing: `Promise.all` over per-entry `lstat` turned a single ENOENT into NOT_FOUND for the directory. Churning directories like build output would intermittently fail to list. Removed the `session-lifecycle -> browser-agent/driver` import edge, which dragged the entire browser subsystem and its module-load `nativeTheme` listener into the auth path to reach one four-line function. `clearBrowserProfile` is now a required dependency wired from index.ts, which already owns both sides. Also deleted `attachSessionLifecycle`, a compatibility wrapper with zero callers. Added a channel-parity test between the preload bridge and the IPC table. They share ~20 channel names as bare string literals with nothing tying them together, so a typo on either side is a silently dead feature that type-checks and ships. Verified it fails on a one-character change. * fix(desktop): reach framed elements and harden the loopback sign-in Two behaviour fixes from the audit backlog. Interaction with same-origin iframes was broken. The snapshot deliberately walks into those frames and hands the model ids for what it finds, but every interaction then tested `instanceof HTMLInputElement` against the top frame's constructors — false for nodes owned by a frame, because element wrappers are realm-bound. So the driver reported a real `<input>` as "not a text input", which took out framed login forms and editors that put a contenteditable body in an iframe, such as TinyMCE. Framed selects reported "not a select" and framed clicks skipped focus entirely. Checks now compare `tagName` or duck-type the method being called, matching the realm-safe approach the credential guard already used. The native value setter is taken from the element's own realm: calling the top frame's setter on a frame's node throws "Illegal invocation". Snapshot value reporting follows the same rule, which is safe because the credential redaction above it is realm-safe and runs first. The loopback sign-in server could be cancelled by anything on the machine. It validated only the shape of the returned state, then tore the one-shot server down and dispatched, leaving the real constant-time comparison to the callback. So a request carrying any well-formed state killed an in-flight sign-in — and the port is reachable by any local process and by any page the user has open via a no-CORS GET, which cannot read the response but does not need to, since the side effect is the kill. The state is now checked before anything is torn down, and a Host that does not name the loopback is refused, which closes the DNS-rebinding shape. * refactor(desktop): drop duplicated helpers and stop logging query strings Net -3 lines, and one of them was a real leak. `navigation.ts` and `windows.ts` truncated URLs for their log lines with a bare `.slice(0, 200)`, which keeps the query string — the five other log sites in the app go through `scrubUrl` for exactly that reason. Tokens and signed parameters live in query strings, so a blocked-URL warning could write one to disk. Both now scrub. `local-filesystem.ts` carried a private `isRecord` byte-identical to `isRecordLike` in `@sim/utils/object`, and four more sites inlined the same check. All now use the shared helper, which also tightens three of them: the inline versions omitted the array exclusion, so an array satisfied a check that then cast it to a record. `tray.ts` hand-rolled slice-plus-ellipsis, the case `@sim/utils/string`'s `truncate` exists for. Titles between 58 and 60 characters now get an ellipsis where they previously did not — cosmetic, in a tray menu label. Removed the `getTabsState` passthrough in the driver, a one-line re-export of the session's own function, and renamed the session-level clear to `clearProfileStorage`. `clearBrowserProfile` existed twice under one name, the driver's being the composite that also clears the browsing-trail registry; index.ts was already aliasing at the import to tell them apart. Two things deliberately not done. The hand-rolled semver in updater.ts stays: replacing it needs `semver` plus `@types/semver` as new declared dependencies in the Electron main process, and the 90 lines it would delete are already covered by eight assertions that I verified match the library's behaviour case for case. Note the same prerelease comparison is duplicated in apps/sim/lib/desktop/min-version.ts, so a future consolidation should do both. No barrel for browser-agent either: routing `security-guards.ts` through one to reach a single leaf function would pull the whole browser subsystem into its module graph, which is the edge just removed from session-lifecycle. * refactor(desktop): move browser compositing out of the session module session.ts held five responsibilities in one flat namespace: 1,061 lines, 29 exports, 26 mutable module-level bindings. For contrast local-filesystem.ts is a comparable 1,125 lines with two exports and no ambient state — size was never the problem, the shared mutable namespace was. Compositing is the part worth isolating. Where the native view sits, when it is visible, which window owns it, the renderer bounds lease, and the occlusion snapshot are the most intricate logic in the browser and are almost entirely separable from tab bookkeeping. They now live in panel.ts (342 lines) and session.ts is 792, with 15 bindings instead of 26. The two modules were mutually dependent, which is what makes this kind of split go wrong. Rather than events or a shared store, panel.ts takes the four things it needs from the session through one PanelHost passed to initPanel — the same shape as the existing initSession — so the import graph is one-way and there is no new indirection to trace. Tab changes reach the panel by the session calling layout(), exactly as before. Two behaviours became explicit rather than implicit in the move: detachIfAttached replaces callers reading `attachedView` to decide whether a closing tab owns the surface, and isPanelVisible replaces `panelBounds !== null`. Nothing about the split is verified by the split itself, so the bounds lease got characterization tests first. It had none — there was not a single fake timer in the suite — despite being the mechanism that hides the view when the renderer crashes or wedges. Both tests were confirmed to fail against a broken lease before the refactor began. The other 47 tests were not rewritten: only the module their calls address changed, which is the useful signal that behaviour was preserved. Deliberately not split further. Focus tracking stays with tabs because it keys off tab ids, and profile teardown stays put; separating either would be taxonomy rather than decoupling. * refactor: drop the legacy local_* filesystem tool shim Granted folders are addressed through the ordinary VFS: the model calls read/grep/glob against paths under user-local/, exactly as it does for workspace files. A parallel local_read / local_grep / local_glob / local_list / local_stat / local_mount_directory / local_list_mounts / local_forget_mount / local_stage_file toolset existed alongside it, recognized but never advertised, so an in-flight checkpoint written by an older desktop build could still finish. There are no older desktop builds. apps/desktop is at version 0.0.0, the only artifacts are a local 0.0.0 build, MIN_DESKTOP_VERSION is '0.0.0' meaning no floor, and the app does not exist on staging at all — the v0.7.x tags are the web app's. Nothing can have persisted a checkpoint naming these tools, and nothing advertises them: they are absent from the generated tool catalog and from mothership's catalog. The shim was defending against a past that never happened. Removes the name table, the legacy request builder, the server-side LEGACY_READ_ONLY_TOOLS allowlist, the five local_* branches in the desktop authorization switch, and nine display labels. isDesktopFilesystemToolCall collapsed into isUserLocalVfsToolCall, which it had become a synonym for. Two tests went with it. One asserted that local_list_mounts routes to the desktop; the test immediately after it already covers the real path, an ordinary read against a user-local path. The other asserted that legacy names cannot open a folder picker, revoke a grant, or upload bytes — that property now holds because no such tool name exists, which is a stronger guarantee than refusing one. * refactor(copilot): remove the plan/changelog VFS artifacts and workflow aliases These beta surfaces are not a direction we are taking, so they come out rather than staying behind a flag. Gone: the workflow alias modules (path resolution, DB-backed resolver, .plans/.changelogs backing provisioning), the alias materialization in the copilot VFS, the alias write paths in resource-writer and workspace_file, the sandbox alias mounts in function_execute, the reserved backing-path guards across mkdir/mv/create, and the alias resolution in the chat home file picker. xlsx survives but changes owner. It was gated twice across the repo boundary: mothership's xlsx-writing flag gates the skill and prompt, while Sim gated the compile path on mothership-beta. Those live in separate AppConfig applications, so an operator had to flip two flags in two consoles, and off-hosted Sim fell back to the MOTHERSHIP_BETA_FEATURES secret while the mothership half stayed in Sim Cloud's AppConfig — split-brain across an ownership boundary. Mothership controls whether the model ever learns xlsx exists, so if it is never offered it is never requested and the second chokepoint only created a way for the two halves to disagree. Sim's gate is removed; xlsx-writing is now the single owner. With its last consumer gone, the mothership-beta flag and the MOTHERSHIP_BETA_FEATURES secret are deleted. The two entries in the infra repo are harmless until removed separately: they only inject an env var nothing reads, and createEnv runs with skipValidation. The reserved-system-file/folder concept goes with the aliases, since it existed only to hide the backing rows. includeReservedSystemFiles and includeReservedSystemFolders are removed rather than left as options every caller passes true to. backingVfsPath is removed for the same reason — nothing sets it once aliases are gone, so it was an always-undefined field on tool results. Test coverage is preserved rather than deleted with the feature. resource-writer.test.ts looked alias-only but three of its eleven cases cover the generic create path that survives; those are kept and the file retitled. Two open_resource tests and one output-path test used alias-shaped strings while asserting generic behavior; retargeted or dropped where a sibling already covers it. * refactor(copilot): remove the dead planArtifact column plumbing copilot_chats.plan_artifact has no writer and no reader that does anything with it. No client sends it, nothing renders it, and its whole history is fork-chat and duplicate-chat plumbing faithfully copying a column that is always null — the one change that might have populated it (mothership v0.8) was reverted. Removed from the schema, the copilot API contract, the chat lifecycle column sets, the fork route, superuser import, the data drain, the update-messages write path, and the legacy chat detail response. No migration here on purpose. The column stays in the database, orphaned and null; dropping it is a separate deliberate step rather than something that rides along with a code cleanup. Note that the next drizzle-kit generate will now want to emit the DROP COLUMN, and check-migrations-safety will ask for it to be annotated — that is the right moment to decide, not now. Mothership never saw this field; it is Sim-side only. * chore(copilot): sync the tool catalog for load_skill Picks up the new load_skill tool plus the grep description that dropped its stale reference to VFS "plans" entries. Generated from copilot/contracts/tool-catalog-v1.json. * refactor(copilot): follow the load_custom_tool rename to load_mcp_tool Mothership renamed the loader once it was clear MCP was the only catalog kind it could match, and dropped the single-valued `type` parameter. The two prompt strings that teach the model the call shape are updated to load_mcp_tool({ name }). load_custom_tool stays in the UI hide-list next to load_agent_skill so tool rows in historical transcripts keep rendering; nothing emits it any more. * chore(copilot): sync the tool catalog and hide load_skill in the UI load_integration_tool and list_integration_tools now publish route go/sync instead of sim/async. Nothing changes in Sim's behavior — they always ran in Go; the contract had been wrong. load_skill joins the hidden tools. It is the same shape as the other loaders already there: the agent pulling in a reference guide before doing the work is a step toward the action, not the action. Sim's display-coverage test caught that a newly added visible tool had no title or completed verb, which is the guard working. * fix(auth): handle session expiry in the app, not the desktop shell The workspace auth gate is a Server Component, so it only re-evaluates on a server render. A session that expired or was revoked mid-visit left the SPA mounted and silently 401ing every request, with nothing to redirect it. The desktop shell had grown its own detector for this: a 401 listener over /api/*, a session probe, and a native "your session has expired" prompt. It could only infer session state from cookie events and HTTP statuses, and it inferred wrong — it fired on ordinary sign-outs (in-flight requests 401 during teardown) and on launching already signed out (the window still shows the restored route while the web app redirects). Those were nearly all of its firings, since a 30-day sliding window means real expiry is rare. Generalizes the impersonation-expired screen instead, which already had the right shape: it keys off the session query settling to null after a session that was live. A signed-out visitor never arms it, and `error` is excluded so an offline blip cannot read as an expiry. The session query now refetches on focus for every session, not just impersonation ones, so returning to a window that slept through its session re-checks it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C54QHj4WPV777Fq2yRwkcb * fix(copilot): port the scheduled-task and VFS fixes onto staging-v4 Replays the sim-side prompt-audit work on top of staging-v4. complete_scheduled_task was filtered out of the execute route's response payload, so an until_complete job could report completion and still be rescheduled; the post-run bookkeeping now also refuses to revive a job that already completed. Also clamps browser_wait_for's timeout the way the desktop agent does, and replaces the oversized-read error's offset/limit advice, which sent the model into a guaranteed retry loop. * feat(desktop): let the model actually see browser screenshots browser_screenshot captured an image and then threw it away. The renderer stripped the data URL and substituted a note, and the tool's own description told the model not to bother: "Dead end for perception." So the agent was blind to anything not expressible as DOM text — canvas, charts, maps, images, rendering and layout bugs. The copilot has carried the machinery for this all along. A tool result shaped as { content, attachment: { type: "image", source: { type: "base64", ... } } } is serialized into a real image content block, with the media type sniffed from the bytes rather than trusted from the declaration, and degraded to a text stub when the routed model has no vision so the provider never 400s. The screenshot result is now reshaped into that contract instead of discarded. A malformed data URL still falls back to a note rather than shipping an attachment the provider would reject. Captures are bounded to a 1024px longest edge at quality 70. CDP clip.scale is relative to CSS pixels, so this also sidesteps the device pixel ratio — an unclipped capture on a retina display returns a 2x image, which was several hundred kilobytes for no legibility the model could use. The description is rewritten to bias toward visual questions only: appearance, layout, rendering, charts, canvas. Reading content or finding something to click stays with browser_snapshot, which is cheaper and returns the element ids a screenshot cannot. That distinction is structural, not just advisory — having seen the page does not let the agent act on it. Companion change in mothership generalizes the tool-result inline-budget exemption from "the read tool" to "any result carrying a model attachment". Keyed on the tool name, an oversized screenshot fell through to the artifact branch: the image was replaced by a reference the model cannot open, and the result still reported success. Silent, and it would have hit almost every call. * fix(desktop): polish browser panel and environment tray icon * fix(desktop): enlarge environment tray markers * fix(desktop): smooth environment tray markers * refactor(copilot): consolidate resource mutation tools * chore(copilot): clean up VFS follow-ups * fix(desktop): round the dev tray marker * feat(desktop): add integrated terminal resources * Fix electron app resize causing glitchy browser frames * feat(copilot): add persistent tool permissions * fix(copilot): retire stale tool permission prompts * fix(desktop): keep terminal rendering responsive * fix(desktop): preserve resource rendering continuity * feat(desktop): add browser tab duplication actions * feat(desktop): add terminal tab context actions * fix(desktop): allow browser agent localhost navigation * feat(desktop): add tmux-backed terminal sessions * fix(desktop): restore terminal scrollback per view * chore(copilot): sync updated wait tool contract * poll terminal session state for non regular shells * add terminal right click menu * feat(desktop): add terminal handoff and key batching * fix(desktop): reserve the traffic-light lane from the platform macOS draws the window controls itself, at a fixed physical size, above all web content. The page renders full-bleed beneath them, so it has to reserve that lane — and it did so with five hardcoded CSS pixel values. CSS pixels scale with page zoom and the OS-drawn lights do not, so zooming out shrank the reservation until the lights were drawn over the sidebar toggle, and the header row below sat inside their band. Electron's `titleBarOverlay` publishes the controls' real geometry to the page as the `titlebar-area-*` env vars, which Chromium rescales per zoom so a reservation derived from them holds its physical size. Measured across zoom 0.58-1.2, the reserved area stays within ~0.6 DIP, the residual coming from env values being quantized to whole CSS pixels. Every lane length now derives from those vars, so the login route and the mothership content offset were fixed without being touched — they already read `--desktop-title-bar-height`. Two of the replaced constants were also simply wrong: the platform reports the lane at 38px and the safe area at 81px, against the hand-measured 36 and 83. The toggle keeps a constant physical size beside the lights, expressed as a proportion of the lane rather than in pixels: a px literal would scale with zoom, and calc cannot divide a length by a length to recover a scale factor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C54QHj4WPV777Fq2yRwkcb * fix(desktop): avoid transient terminal tab labels * feat(copilot): attach browser and terminal tab context * feat(desktop): close tmux panes from terminal tools * fix(desktop): keep terminal tab icons stable * add right click to browser and cleanup terminal right click options * fix(desktop): reduce hidden panel background work * perf(desktop): shrink browser panel snapshots * perf(desktop): reduce terminal main process overhead * perf(terminal): pause work for hidden sessions * fix session arch for desktop * fix(desktop): replace exited terminal sessions * feat(copilot): persist desktop resources across chats * fix(emcn): keep resource tab widths consistent * fix(copilot): restore active client panels * feat(desktop): import Chrome browser data * fix(copilot): close resources before chat creation * feat(desktop): suggest imported browser sites * fix(desktop): autofill identifier-first sign-ins * fix resizing issues + cookies source * fix visits marking * chore(db): drop branch migrations ahead of staging merge 0264/0265 on this branch collide with staging's 0264-0270 on both the journal idx slots and the meta snapshot filenames. Reverting the migration artifacts to the merge-base lets staging's chain merge cleanly; schema.ts keeps the copilot changes and drizzle-kit regenerates a single migration on top of 0270 after the merge. Co-Authored-By: Claude <noreply@anthropic.com> * feat(db): regenerate copilot tool-permission migration on top of staging Replaces the branch's old 0264/0265 (dropped pre-merge so staging's 0264-0270 chain could apply cleanly) with a single 0271 generated against staging's schema: the permission-decision enum, the two copilot_async_tool_calls decision columns, and copilot_chats.auto_allowed_tools. Deliberately does NOT drop copilot_chats.plan_artifact. The branch removed every reader, but the currently-deployed code still SELECTs that column, so dropping it in the same deploy breaks the old app version during blue/green overlap — `check:migrations` flags it for exactly this reason, and the honest fix is to defer rather than annotate around it. The column is retained in schema.ts marked @deprecated; drop it in a follow-up once this has rolled out. Also in this commit, all fallout from the merge itself: - pinned-fetch/revoke tests: their private-IP stub moved to @sim/security/ssrf alongside the source change. Worth noting the stub exists because the suite's 203.0.113.10 is TEST-NET-3, which the real classifier correctly calls reserved — the old stub had been quietly disagreeing with production. - materialize-file test: dropped the reserved-system-folder case, which covered the workflow-alias backing folders this branch deleted. - api-validation route ratchet 977 -> 983 (this branch's new routes). Co-Authored-By: Claude <noreply@anthropic.com> * add cmd f * review pass * chore(db): drop branch migration ahead of staging merge Both sides independently claimed idx 0271, so the snapshot and journal would conflict add/add. Ours is plain additive DDL that drizzle regenerates from schema.ts; staging's is a hand-written CONCURRENTLY index build that cannot be regenerated. Dropping ours and re-generating on top of staging's is the only order that preserves both. schema.ts is deliberately untouched — it is the source of the regeneration. Co-Authored-By: Claude <noreply@anthropic.com> * chore(db): drop branch migration ahead of staging merge Both sides independently claimed idx 0272, so the snapshot and journal would conflict add/add. Ours is plain additive DDL (one enum, two columns, one jsonb default) that drizzle regenerates from schema.ts; staging's is a hand-written migration with DO blocks and CONCURRENTLY index builds that cannot be regenerated. Dropping ours and re-generating on top of staging's is the only order that preserves both. schema.ts is deliberately untouched — it is the source of the regeneration. Co-Authored-By: Claude <noreply@anthropic.com> * style(db): biome-format the regenerated migration metadata drizzle-kit emits _journal.json and the snapshot with expanded arrays, which biome check rejects. The merge commit used --no-verify, so lint-staged never formatted them and CI's lint step failed on exactly these two files. Whitespace only — both files are byte-identical under `jq -S -c`. Co-Authored-By: Claude <noreply@anthropic.com> * fix(desktop): pin the platform in the OS-auth tests promptForSecret gates Touch ID on process.platform === 'darwin'. The suite mocked electron's systemPreferences but inherited the runner's real platform, so the eight biometric expectations passed on a Mac and failed on Linux CI, where every call fell through to the confirmation dialog instead. Pins the platform per-test and restores it after, and adds a case for the gate itself — the branch whose absence from the suite is what let this through. Co-Authored-By: Claude <noreply@anthropic.com> * fix(desktop): refine environment dock icons * fix(desktop): align packaged environment icons * fix(desktop): keep packaged dock icon rendering consistent --------- Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Theodore Li <theo@sim.ai>	2026-07-28 19:25:59 -07:00
4 people	e8e3d6984c	feat(pi): optional multi-provider web search for the coding agent (#5951) ... * feat(pi): optional multi-provider web search for the coding agent Adds a search provider dropdown (Exa, Serper, Parallel, Firecrawl) to the Pi block, off by default. The selected provider's key comes from the block field or Workspace Settings → BYOK; a Sim-hosted key is never spent, so a missing key fails the run with a setup message instead of quietly billing Sim. Search is available in all three modes. Local Dev and Review Code register a host-side tool that goes through the existing provider tools, while Create PR has no host in the loop and gets a generated Pi extension in the sandbox. Both paths derive their requests from one normalizer and are held together by a parity test, since the sandbox copy cannot import Sim's code. Results are normalized to title, URL, snippet, and publication date, capped per field and per envelope, marked untrusted in the prompt, and limited to 20 searches per run so a tool loop cannot drain the workspace's quota. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(pi): drop the banned JSON round-trip from the search parity test `check:utils` bans `JSON.parse(JSON.stringify(...))`. The round-trip was normalizing the host body to its wire form, which buys nothing here: the bodies are plain JSON and `toEqual` already ignores undefined members. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(pi): upgrade the E2B SDK so long Pi output streams stop failing Create PR streams the whole Pi run through one Connect server-stream (`commands.run` -> envd `Process.Start`), held open for the full `PI_TIMEOUT_MS`. Mid-stream it could die with: [internal] protocol error: received unsupported compressed output That string is `@connectrpc/connect-web`, not Pi — Pi has no Connect dependency at all. connect's `compressedFlag` is `0b00000001` and gzip's magic first byte is `0x1f`; `0x1f & 0x01 === 1`, so a raw gzip body fed to the envelope reader trips this on byte one. It reads as "the server sent a compressed envelope" but really means "this was never a Connect envelope" — an HTTP-level gzip that was not transparently decompressed. e2b 2.30.0 pinned `@connectrpc/connect-web@2.0.0-rc.3` and drove envd through undici 7 with `allowH2: true`. e2b 2.36.1 moves to stable connect-web 2.1.2 and loads undici 8.8.0 when Node >= 22.19.0 — exactly our engine floor — so the failing path gets a different HTTP stack. The connect-web upgrade alone is not the fix: 2.0.0-rc.3 and 2.1.2 ship a byte-identical `connect-transport.js` (bar the copyright year), and connect-web still has no `acceptCompression` option by design. The undici 8 swap is the part that matters. `@e2b/code-interpreter@2.7.0` only asks for `e2b: ^2.28.0`, so the override pins the floor we actually need. Verified API-compatible: every method we call (`Sandbox.create`, `runCode`, `commands.run`, `files.read/write`, `kill`, `Template`, `defaultBuildLogger`, `waitForTimeout`) has an identical signature across the two versions, and we never touch `SandboxPaginator`, the one type that changed. * fix(pi): correct search normalization edge cases and the budget's stated scope Follow-ups from review of the web-search work. Each fix lands in both the host adapter (`normalize.ts`) and the Create PR sandbox copy (`extension-source.ts`), with the extension test asserting the two produce byte-identical envelopes. - `usableUrl` was the one provider-controlled field not whitespace-bounded: title/snippet/date all go through `collapseWhitespace`, `url` only trimmed. Up to 2048 chars of newlines and control characters could ride into the envelope. Dropped rather than collapsed — `url` must stay byte-exact to stay resolvable, so collapsing would emit a different, still-dead link, and a URL carrying raw whitespace is already malformed under RFC 3986. - `numResults: null` (or `''`, or `[]`) returned 1 result, not the documented default of 5: `Number(null)` is a finite 0, so the clamp floor won rather than the default. Only a real number or a non-blank numeric string now counts as the model having asked for a count. - Envelope truncation was silent. When results were dropped to fit the 50 KB ceiling the model read the short list as the complete answer. It now carries a message saying so, and the message is inside what gets measured so the note cannot push a truncated envelope back over the ceiling. - The budget is per *block execution*, not per workflow run: the counter lives in the tool spec and both adapters build a fresh one per execution, so a Pi block inside a Loop gets the full allowance every iteration. The constant, the agent-facing message, and the docs all claimed "per run". Renamed to `PI_SEARCH_MAX_CALLS_PER_EXECUTION` and corrected the wording rather than tightening the cap, since a shared ceiling would fail late iterations of a legitimate fan-out. - The Search API Key tooltip promised "switching providers clears this field". That clear is driven through the collaborative editor setter, so a workflow imported, forked, or updated via the API keeps the previous provider's key — exactly the case where sending it to a new vendor matters. Docs also gain a warning that Create PR hands both the model key and the search key to the agent as environment variables, which Pi copies into every bash child. That matters most for Settings > BYOK keys: those are workspace-scoped, only admins can manage them, and the API only ever returns them masked — yet anyone who can run a Pi block in Create PR mode can read the raw value. * fix(pi): make the search provider drift guards actually fire The "you cannot add a provider without mirroring it" story rested on two mechanisms that did not hold. Verified by adding a fifth provider to `PI_SEARCH_PROVIDERS` and running the build: it produced only two errors, and every test still passed. - `normalizePiSearchRecords` assigns to `let built` inside its switch rather than returning, so unlike its two siblings a missing case was not a type error — it silently normalized the new provider to zero results. Added an explicit `never` check. - The sandbox copy's `normalizeRecords` used a trailing `else` for Firecrawl, so an unmirrored provider was silently normalized with Firecrawl's field names; `extractRecords` did the same with its `payload.data` tail. Both now test for `firecrawl` explicitly and throw otherwise. - `Record<PiSearchProvider, ...>` on the `TOOLS` and `payloads` fixtures looked like exhaustiveness guards but are inert: `apps/sim/tsconfig.json` excludes `**/*.test.ts`, and vitest transpiles without typechecking. Both suites drive their providers off `Object.keys(fixture)`, so a missing provider was skipped rather than failed. Each suite now asserts its fixture covers the registry. Re-running the same experiment now yields three compile errors plus two test failures naming the missing fixtures. * fix(pi): drop the workspace BYOK fallback for the search key A fallback exists so a key has somewhere to go when the field is unavailable. The Search API Key field is unconditionally available: unlike the model key, whose visibility runs through `shouldRequireApiKeyForModel` and its `isHosted` branch, `getSearchApiKeyCondition` gates only on whether a provider is selected. So the fallback never had a configuration to cover. Removing it also closes an escalation. Workspace BYOK keys are admin-managed and the API only ever returns them masked, yet `resolvePiSearchKey` would resolve one for any member who could run the block — and in Create PR that key is handed to the sandbox as an environment variable, which Pi copies into every bash child. A member could read a credential the product deliberately never shows them. Requiring the key on the block keeps the sandbox exposure to a key its author already holds. Nothing depends on the fallback: it has never shipped. - `resolvePiSearchKey` is now synchronous and returns the key, since there is no lookup left to await. `byokProviderId` leaves the search registry and `PiSearchKeySource` / `PiSearchKeyResolution` are gone — with one source, `keySource` carried no information, and the logging rationale for it (a block field silently shadowing a stored key) no longer exists. - The field is now `required`. Safe alongside its condition: the serializer's required check returns early for fields that are not visible, so a Pi block with search off still validates. Pinned by a test. Docs and the block's tooltip, placeholder, and best practices updated. The Create PR key-exposure callout now explains the missing fallback rather than recommending the block field as a way around it. * docs(pi): import Callout explicitly, as the sibling block docs do `fumadocs-ui/mdx`'s `defaultMdxComponents` already provides `Callout`, so the callout added earlier rendered fine without this — but logs.mdx, credential.mdx, and response.mdx all import it explicitly and pi.mdx was the outlier. Not a build fix: the docs Vercel deployment is failing on staging HEAD as well. * chore(deps): exclude the e2b packages from the release-age gate CI's `bun install --frozen-lockfile` failed on the E2B upgrade: error: No version matching "@e2b/code-interpreter" found for specifier "^2.7.0" (blocked by minimum-release-age: 604800 seconds) This did not reproduce locally because the checkout's bun was 1.2.15, which predates `minimumReleaseAge` support and ignored the gate outright; CI runs the pinned 1.3.13 and enforces it. Excludes only the two packages that are actually too young — @e2b/code-interpreter 2.7.0 (2026-07-23) and e2b 2.36.1 (2026-07-27). The rest of the chain already clears the gate: @connectrpc/connect{,-web} 2.1.2 and @bufbuild/protobuf 2.13.0 and undici 8.8.0 are all older than a week, and `tar` resolves from the lockfile at 7.5.22 without needing an exception (the original CI error named only @e2b/code-interpreter, and `bun install --frozen-lockfile --ignore-scripts` under 1.3.13 now passes locally). The lockfile is regenerated with bun 1.3.13 rather than 1.2.15, which also corrects hoisting the older bun had gotten wrong on the merge commit: the root `lucide-react` hoist moves from 1.23.0 back to 0.511.0 and `@radix-ui/react-slot` from 1.3.0 to 1.2.2, each with the proper scoped entries. Package resolution still differs from staging by exactly the e2b chain and nothing else. Both entries age out on 2026-07-30 and 2026-08-03; drop them then. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>	2026-07-28 16:16:51 -07:00
WaleedandWaleed Latif	a042f0bd0f	chore(deps): fix OTel version split, drop dead deps, declare emcn peers (#5994) ... * chore(deps): fix OTel version split, drop dead deps, declare emcn peers - Pin @opentelemetry/{resources,sdk-metrics,sdk-trace-base,sdk-trace-node} to exact 2.7.1 so they match sdk-node's pins instead of floating to 2.8.0. The carets meant app code built spans with 2.8.0 and passed them into NodeSDK from 2.7.1, which only worked by duck-typing. - Declare the 13 packages @sim/emcn imports but never declared, as peers mirrored into devDeps. @radix-ui/react-dismissable-layer had no declaration anywhere in the repo and resolved only transitively. - Remove ffmpeg-static: its binary downloads via postinstall, but it is not in trustedDependencies and Docker installs with --ignore-scripts, so the accessSync branch never succeeded and both call sites always fell through to system ffmpeg. - Remove critters + experimental.optimizeCss: Next only loads critters from the Pages Router renderer, and apps/sim is App Router only. - Make simstudio-ts-sdk zero-dependency by dropping node-fetch for native fetch; engines >=18. - Remove unused @vercel/og and postgres from docs, dotenv/inquirer/listr2 from the CLI, and yaml from the root. - Move @aws-sdk/client-appconfig from the root to apps/sim, its only consumer. - Delete the apps/sim overrides block; Bun only honors top-level overrides. - Bump free-email-domains 1.2.25 -> 1.9.70 (4,779 -> 13,059 domains). - Validate object/array variables with JSON.parse instead of JSON5, matching what the executor actually parses. - Swap the changelog GitHub icon off lucide to GithubOutlineIcon, matching the navbar chip on the same page. - Unify @types/node on 24.2.1 and lucide-react on ^0.511.0; bump chalk to 5 and image-size to 2. * fix(deps): complete the OTel pin, restore SDK error detail, revert email list Follow-ups from an independent audit of the previous commit. - Pin @opentelemetry/sdk-node and the three otlp-http exporters to exact 0.217.0. Pinning only their four dependents was self-reversing: sdk-node 0.219.0 requires core 2.8.0 exactly, so the next update would have silently rebuilt the split this PR removes. - Declare @opentelemetry/core (2.7.1). It is imported by lib/copilot/request/go/propagation.ts but resolved only by hoisting, and it is the OTel package with the most version churn in the tree. - Pin @radix-ui/react-dismissable-layer to exact 1.1.13 in @sim/emcn. All five transitive parents pin it exactly; a caret would fork a second copy on 1.1.14, which is the duplicate-context bug the declaration prevents. - Surface error.cause in simstudio-ts-sdk. Native fetch reports network failures as a bare "fetch failed" and puts the reason on cause, so every DNS/TLS/refused error was reaching callers with no diagnostic content. - Revert free-email-domains to 1.2.25. Upstream now merges the free-domain list with two disposable-email blocklists, so 1.9.70 classifies real organization domains as free — UK charities, some companies and universities, and the JP/KR ISP domains APAC SMBs use for business mail. The demo form blocks submission on that check, so a false positive costs the booking entirely. Worth doing deliberately, not inside a deps change. - Lower packages/cli engines to >=18. chalk 5 and commander 11 both accept >=16 and the source uses no Node 20 API, so >=20 only produced EBADENGINE for Node 18 users. --------- Co-authored-by: Waleed Latif <waleed@simstudio.ai>	2026-07-27 19:16:32 -07:00
Theodore Li	75b8b6f3e5	feat(settings): self-host settings plane, Sim wordmark in sidebar (#5990) ... * feat(settings): self-host settings plane, Sim wordmark in sidebar Chat keys were only reachable at a standalone /account/settings/chat-keys page that nothing linked to. They now live on a dedicated self-host plane alongside the two other settings a self-hoster needs from the managed service. - new /selfhost/settings/{general,billing,chat-keys} plane, open to any signed-in user - chat keys move off the account plane entirely; no isHosted gate - registry `unified` projection is now optional (mirroring `planes`), so a section can opt out of the editor sidebar - plane items resolve their own description and throw when one is missing, since a plane-only section has no unified projection to inherit from - settings sidebar shows the Sim wordmark linking to /?home instead of a Back chip; drops the now-dead backHref prop - `bun run setup` runs `bun install` first so a fresh clone is one command * docs(readme): point Chat keys at the self-host settings plane The account-plane URL stopped resolving when chat keys moved to /selfhost/settings. * improvement(settings): make the sidebar wordmark a plane attribute Replacing the Back chip everywhere was too broad — account and organization are reached from inside the app, so Back is right there. Self-host is reached from outside it (the CLI wizard, the README), so it leads with the brand mark instead. SETTINGS_PLANE_CHROME declares that per plane, keyed on StandaloneSettingsPlane so adding a plane forces the decision rather than defaulting silently. It also absorbs the shell's parallel plane-label map. * fix(settings): hide self-host Chat keys on non-hosted deployments Chat keys are issued by the managed service and useCopilotKeys is `enabled: isHosted` for that reason. Moving the section onto the self-host plane dropped its hosted gate, so a self-hosted deployment rendered a Chat keys nav item whose list could never populate. Restores the gate on the plane that now owns the URL. sim.ai is unaffected — the section stays visible there to every signed-in user, which is the surface self-hosters are pointed at.	2026-07-27 20:01:26 -04:00
Waleed	66ac015c4c	fix(library): generate every post cover from one template (#5980) ... * fix(library): generate every post cover from one template Three posts shipped an `ogImage` pointing at a file that was never committed, so the library index rendered broken images and their `og:image`, JSON-LD, and sitemap entries all 404'd. Several others were authored without the brand font loaded or with the title clipping off the bottom edge. Covers were hand-made per post with no generator, which is why they drifted. Adds `bun run library:covers`, rendering each cover from the post's frontmatter title using the reference template already encoded in the docs OG route, and regenerates all 20 so the grid is uniform. Line widths come from the font's real advance metrics rather than an average-glyph-width estimate: the template joins words with non-breaking spaces to dodge a Satori space-measurement bug, which leaves hyphens as the only fallback break points, so an under-measured line breaks mid-compound. Also drops six orphaned `cover.png` sources left over from the JPEG compression pass in #5528. * fix(library): re-render covers every run and add a sync check Covers are derived artifacts, so skipping outputs that already exist left an image showing the old title after a post's frontmatter `title` changed. Every run now re-renders from scratch; rendering is deterministic, so an unchanged title re-encodes to identical bytes and a full run stays a no-op in git. Replaces `--force` (now the default) with `--check`, which renders in memory and compares against the committed bytes without writing, so CI can catch both a stale cover and the missing-cover case that caused the original breakage. * fix(library): compare decoded pixels in the cover sync check Byte-equality on the mozjpeg output assumed portable encoder bytes. libvips/mozjpeg does not guarantee that across OS and CPU, so identical input can encode differently on a contributor's machine or a Linux CI runner and fail the check for no real reason — exactly where the check was meant to run. Decodes both images to greyscale and compares mean absolute difference instead, which discards encoder variance while still testing what the check is about. Measured on this cover set: re-encoding an identical render with a deliberately different encoder moves it ~0.26, a one-word title change moves it ~12; the threshold of 2 sits between them with ~8x margin. * fix(library): count redrawn pixels in the cover sync check Averaging the difference diluted a local edit across all 810,000 pixels. Changing a title's "2026" to "2027" moved the mean by 0.42 — under the tolerance that absorbed encoder noise — so the check passed a cover still showing the old year. Counts pixels that moved more than 48 greyscale levels instead. Measured on this cover set, that one-character edit redraws 2,559 pixels while three deliberately different encodes of an identical render (quality 60/70 without mozjpeg, quality 95 with) redraw none, so the count separates real drift from encoder variance in both directions. * fix(library): parse frontmatter with gray-matter and split oversized tokens Two issues in the cover generator, neither reachable from a current title. The hand-rolled frontmatter regex could disagree with `gray-matter`, which is what renders the page and its `og:title`. On a double-quoted escape or a block scalar the cover would have rendered a title the page never shows, with `--check` calling it in sync. Uses `gray-matter` directly so there is one parser. `wrapTitleLines` only breaks between space-separated words, so a token wider than the title box on its own stayed on an overflowing line, and the non-breaking spaces left Satori no recourse but to break it at a hyphen — the mid-compound break this layout exists to prevent. Oversized tokens now split here, at hyphens first and per-character only for something like a URL, and a font size is accepted only if every line measures within the box. All 20 covers re-render byte-identically, so neither change alters current output.	2026-07-27 14:19:01 -07:00
Theodore Li	19c3b6f47d	feat(setup): setup wizard with browser-based Chat key handoff (#5911) ... * feat(setup): setup wizard with browser-based Chat key handoff Adds `bun run setup` and `bun run doctor` for local installs, and replaces the wizard's paste-your-Chat-key step with a browser handoff that never puts the key in a URL. * improvement(setup): drop the paste-a-key fallback, simplify consent copy The browser handoff is now the only path — the wizard waits on a spinner instead of racing a paste prompt. Consent card leads with "Connect your terminal" and moves the match-the-code disclaimer into the description. * fix(setup): pin kube context, keep secrets out of argv, validate reused keys Review findings from #5911: - helm/kubectl now run against the validated context instead of the ambient one - helm values are piped on stdin rather than passed as --set arguments - ENCRYPTION_KEY/API_ENCRYPTION_KEY are checked for the 64-hex format the app requires, not just length, so an unusable key is replaced rather than kept - the managed Redis container's published port is read back instead of assumed * refactor(copilot): one module for Chat API key operations list/generate/delete each repeated the same /api/validate-key envelope in their route. They now share callValidateKey in lib/copilot/server/api-keys.ts, which also keeps the display masking server-side so the full key can only ever leave at creation. * improvement(setup): reuse shared helpers, parallelize probes, drop dead code - PKCE verifier/state/pairing code now use generateSecureToken, generateRandomHex and generateShortId instead of hand-rolled randomBytes; the pairing loop's modulo was unbiased only because 256 % 32 == 0 - new sha256Base64Url in @sim/security/hash so both sides of the PKCE exchange derive the challenge from one implementation - isUsableSecret moved beside SECRET_KEYS so setup and doctor apply the same rule; doctor previously passed a key setup would replace - isTruthy narrowed to true/1, matching the app it claims to mirror — it accepted yes/on, so a flag could read on in doctor and off in the app - checkLive runs its five probes concurrently (~17s serial worst case) - detection overlaps the banner animation instead of queueing behind it - glyph.fail/glyph.warn at 13 sites that bypassed the constant; removed unused prompter exports, a dead ENV_PATHS re-export, and an unused export keyword * fix(setup): make doctor understand the compose env layout Compose writes a single root .env (what docker-compose reads via env_file) but the checks required the three per-app files, so a successful compose install was followed by doctor printing three failures and exiting 1 — and the whole coherence catalog was skipped because it keyed off apps/sim/.env existing. Layout is now derived from what's on disk and every check consults it: file and schema checks iterate the layout's targets, consistency reports skip when there's only one file to mirror, and coherence/live read the layout's primary file. The wizard's existing-config detection counts root for the same reason — a compose install used to read as unconfigured and re-run from scratch. * feat(cli-auth): device-authorization poll flow, drop the loopback listener The CLI no longer binds a local port. It generates a request id + poll secret, opens /cli/auth, and polls /api/cli/auth/poll over TLS while the user approves in the browser — so the flow works over SSH and inside containers, where the browser and terminal don't share a machine. - approve stores the approval keyed by request id (session-authed, userId from the session only); poll verifies the secret before an atomic claim, so an observer of the semi-public request id can neither mint nor cancel it - pairing code stays as the anti-phishing compare; no key ever crosses the browser; done page just confirms - removes the loopback listener, /token exchange, buildCliHandoffUrl, and validateCliCallbackUrl (+ its tests) — nothing hands a key to a URL anymore * fix(setup): reuse an existing managed Postgres container instead of colliding A running sim-postgres fell through to `docker run --name sim-postgres` and died on the name conflict; a stopped one failed with "no DATABASE_URL to reach it" because the generated password only lived in the env files a fresh clone lacks. Both facts are recoverable from Docker: the ladder now reads the published port and password back via `docker inspect` and reuses the container (starting it if stopped). A container that won't answer prompts before recreating, and never drops the data volume silently. * improvement(setup): audience-first run-mode hints Each run mode now names who it's for — compose for self-hosting/evaluating, dev for contributing to Sim, k8s for rehearsing a production deploy — with the live detection state (Docker/kube/VM) appended. * fix(cli-auth): retry a failed mint, port container/port fixes to Redis + k8s Review findings from #5911: - poll now reserves the mint with an atomic NX lock instead of deleting the approval up front, so a failed mint (e.g. mothership blip) is retried by the next poll instead of forcing a fresh browser approval; the lock still prevents a double-mint and its TTL frees the slot if the caller dies - setup reuses/recreates an unhealthy managed sim-redis instead of colliding on the name (Redis has no data volume, so it removes and recreates without a prompt) - k8s failure-path hints carry --context, matching the success-path hints, so a changed ambient context can't send diagnostics to the wrong cluster - compose port-free waits for a killed port to actually release before re-checking; SIGKILL is async, so the immediate re-check re-saw the port * fix(setup): harden mint cleanup, Windows browser, container detection, helm cwd Review findings from #5911: - a post-mint completeApproval failure no longer routes into releaseMint — the mint lock now outlives the approval (shared TTL), so a cleanup blip can't leave a re-mintable window and orphan a key; cleanup is best-effort after the key ships - compose doctor --fix writes the feature-flag twin to the layout's primary env (root .env on a compose install), not always apps/sim/.env - Windows opens the browser via `cmd /c start "" <url>` — `start` is a shell builtin, so spawning it directly ENOENT'd and the handoff never opened - managed-container detection filters loosely and pins the exact name in code; Docker's `name=^x$` anchor matches the internal `/x` form and often missed, skipping the reuse branch - the shared helm/kind run helper pins cwd to the repo root, matching helm test, so `helm upgrade --install ./helm/sim` works from any working directory * feat(chat-keys): standalone manage page, drop from settings nav, refresh README - Add /account/settings/chat-keys — a linkable page to view, create, and revoke Chat API keys - Remove Chat keys from the settings sidebar (account + unified nav) and its render branches - README: replace Docker Compose + Manual Setup with the bun run setup wizard; drop the manual COPILOT_API_KEY step, point to the manage page * fix(setup): per-key reason in the secret-replacement warning Cursor: the warn hardcoded '64-character hex key', but only ENCRYPTION_KEY/API_ENCRYPTION_KEY require that — BETTER_AUTH_SECRET/INTERNAL_API_SECRET only need length >= 32. Use the existing secretRequirement(key) helper so each replaced key reports its actual requirement. * fix(setup): compose doctor schema, cross-platform binary detection, quoted context hints - Doctor: for the compose (root) env layout, require only the secrets compose has no interpolation default for (BETTER_AUTH_SECRET/ENCRYPTION_KEY/INTERNAL_API_SECRET). DATABASE_URL/BETTER_AUTH_URL/NEXT_PUBLIC_APP_URL come from docker-compose ${VAR:-default}, so a healthy compose install no longer fails doctor. - Binary detection: use Bun.which instead of which (which is absent on Windows), so kubectl/helm/kind/docker resolve cross-platform. - k8s diagnostic hints: POSIX-quote the kube-context so a context with whitespace/metacharacters can't break or inject into a copied command. * fix(setup): quote kube-context in the helm uninstall tear-down hint too The tear-down hint used --kube-context ${context} raw while the sibling kubectl hints already used shq(); a context with whitespace/metacharacters could break or inject into the copied command. All copyable k8s hints now go through shq(context). * feat(setup): sim lifecycle CLI — start/stop/status/logs/down/reset Turn the setup entry into a 'sim' command umbrella so there's one place to run everything, not scattered docker/bun commands. Adds a global bin (bun link) + a bun run sim fallback. - Detects how you're running (compose file / managed dev containers / helm release) from disk + docker/helm state — no persisted mode. Ambiguous installs prompt. - start/stop/restart/logs work per mode; down removes containers (volumes kept); reset archives .env + wipes managed data; both destructive verbs confirm first. - status shows detected mode, container states, and app/realtime health. - Wizard outro + README now point at the sim commands and the one-time bun link. * feat(setup): 'bun run sim' is the primary entry; bare invocation prints help - Lead usage/wizard-outro/README with 'bun run sim <cmd>' (works with zero PATH setup); global bare 'sim' via bun link is an optional upgrade, with the ~/.bun/bin PATH caveat spelled out (Homebrew's bun omits it). - Bare 'sim' now prints help instead of launching the wizard; the wizard is 'sim setup'. The 'setup' npm script passes the keyword so 'bun run setup' is unchanged. * fix(setup): quote the auth URL for cmd /c start on Windows Cursor (High): cmd re-parses the command line and treats & in the query string as a command separator, so cmd /c start opened a URL truncated at the first &, breaking the key flow on win32 (the handoff URL always has request/challenge/pairing). Quote the URL and pass args verbatim so & stays literal. * fix(setup): verify kube-context is really local; lengthen CLI handoff wait - k8s: a context named like a local cluster (kind-*, docker-desktop) can actually point at a remote API server. Verify the server host is loopback/docker-internal before defaulting the 'use this context?' confirm to yes; otherwise warn and default to no, so generated secrets can't ship to a remote cluster on a blind Enter. - cli-auth: bump the device-flow wait from 3 to 15 minutes so first-time users have time to sign up, wait for the email OTP, and approve before the terminal stops polling. The server-side approval record keeps its own short TTL, so a longer client wait only costs cheap rate-limited polls. * fix(setup): only manage k8s lifecycle on a verified-local context Greptile: sim down/reset used the ambient kube-context, so switching context after setup could uninstall a same-named sim-dev release from the wrong cluster. Gate k8sInstall on the same locality check the wizard uses (API server is loopback/docker-internal) via a shared isLocalKubeContext helper — the wizard only ever deploys locally, so a remote current-context is never treated as a Sim install. * fix(setup): doctor skips placeholder secrets when seeding; reset names its target - checks: the missing-file autofix copied shared keys from apps/sim/.env whenever truthy, including .env.example placeholders — doctor --fix could seed unusable secrets into realtime/db env files. Skip placeholders, matching autofixForMissing. - lifecycle: reset now names the exact install (k8s context / compose file / dev containers) in its confirm, so a destructive reset can't silently hit the wrong same-named install after a context switch (down already names the context). * fix(cli-auth): size the poll rate limit to the poll cadence; honor Retry-After The poll route used the default public-IP bucket (10 burst, 5/min) but the CLI polls every 2s (30/min), so it 429'd within ~20s — worse behind a slow dev cold-compile. Give the endpoint a bucket matched to its cadence (60 burst, 60/min); it's not a brute-force surface (unknown request id returns pending, minting needs the 256-bit verifier). Also make the CLI honor Retry-After and back off on 429 so a shared-NAT per-IP limit degrades gracefully instead of hammering. * fix(setup): check ports before starting the dev server, not just compose Local dev auto-start spawned bun run dev:full with no port check, so it silently started a server that couldn't bind when 3000/3002 were already taken (e.g. another worktree's dev server). Extract compose's port-conflict resolver into a shared ensurePortsFree(ports) and run it before the dev start too — kill/recheck/leave, same as compose. Leaving the ports skips the auto-start with guidance instead of failing; compose still treats it as fatal. * fix(setup): verify the kube cluster is reachable, not just local A kubeconfig context can outlive its cluster — a kind cluster gets deleted or its Docker container stops (Docker/machine restart), but the context entry remains, pointing at a dead API-server port. The wizard checked the context looked local and handed it to helm, which failed with 'cluster unreachable'. Add a clusterReachable() liveness probe: only offer the current context when it actually answers; if a local context is dead, fall through to the kind path. There, if kind still knows 'sim' but it's stopped, start its node containers and wait for the API; if it's gone, create fresh. Either way the user gets a working cluster instead of a cryptic helm failure. * fix(helm): point appVersion at published image tags (v-prefixed, current) The chart's appVersion was "0.6.73", but CI publishes GHCR tags with a v prefix (its release-commit regex captures v0.7.45). Since sim.image defaults every image tag to Chart.AppVersion, a default helm install requested ghcr.io/simstudioai/{simstudio,realtime,migrations}:0.6.73 — a tag that has never existed — so app and realtime sat in ImagePullBackOff and helm --wait failed with 'progress deadline exceeded'. Any self-hoster installing with default values hit this, not just the setup wizard. Set appVersion to v0.7.45 (latest release on main; all three images verified present on ghcr) and bump the chart version to 1.1.1. Verified with helm lint, helm template (all images render as v0.7.45), and a live helm upgrade on a kind cluster where the new pods pull successfully while the old 0.6.73 pods remain in ImagePullBackOff. * Revert "fix(helm): point appVersion at published image tags (v-prefixed, current)" This reverts commit 28b6047d1d. * chore(api-validation): rebaseline route count to 977 after staging merge Staging moved the baseline to 975; this branch's two CLI-auth routes (approve, poll) make 977. The clean merge absorbed the earlier +2 adjustment. * fix(settings): don't highlight a sibling nav item on nested settings pages /account/settings/chat-keys is a real page but deliberately not a nav item, so the sidebar's parseSettingsPathSection fell through to defaultSection ('general') and highlighted General — the page read as though it lived inside General. Resolve the sidebar's active item with a null default so an unmatched nested route highlights nothing, and widen SettingsSidebar's activeSection to string | null. The section feeding the title/description provider keeps its default (pages override title/description anyway), and /account/settings/billing/credit-usage still correctly highlights Billing. * fix(setup,auth): manage explicitly-confirmed k8s contexts, fail loudly on reset, clear stale post-auth redirect - lifecycle: detection is now factual — a sim-dev release either exists on the current context or it doesn't. Gating on locality stranded a release the user explicitly confirmed during setup (status/start/stop/down/reset all claimed no k8s install). Locality is recorded instead and surfaced through describeInstall, which every destructive confirm renders, so acting on a non-local cluster is named and defaulted to no rather than silently blocked or silently allowed. - lifecycle: reset no longer discards helm uninstall's exit status. Env files are archived by that point, so claiming 'Reset complete' while the release still runs is the worst outcome — it now throws with retry/inspect commands. - auth: signup clears POST_AUTH_REDIRECT_STORAGE_KEY when it has no callbackUrl, and the verification-disabled path consumes it, so a stale CLI/invite destination can't leak into a later flow in the same tab.	2026-07-25 04:24:36 -04:00
Waleed	d64739cf4f	fix(ci): unblock @next/swc, lockfile-keyed node_modules, per-image runner sizing (#5945) ... * fix(ci): key node_modules sticky disk on the lockfile hash * improvement(ci): per-image Blacksmith runner sizing + cold-build memory preflight * fix(deps): exclude @next/swc binaries from the release-age gate * fix(deps): pin @next/swc binaries so frozen installs get a compiler * docs(ci): explain ARM runner sizing rationale	2026-07-24 16:12:22 -07:00
Waleed	91a9c20acc	fix(deps): build isolated-vm on install via root trustedDependencies (#5935) ... .npmrc set ignore-scripts=true globally, which overrode Bun's trustedDependencies allowlist and blocked every lifecycle script — including the repo's own root scripts. isolated-vm therefore shipped unbuilt and each contributor had to npm rebuild it by hand. The .npmrc line landed in the May 2025 Bun migration, seven months before isolated-vm was introduced. Two later commits added isolated-vm to trustedDependencies (root, then apps/sim) and both were silently inert. - delete .npmrc; lifecycle policy now lives solely in root trustedDependencies - add isolated-vm to root trustedDependencies - drop the workspace-level trustedDependencies from apps/sim (Bun only reads the root list, so it never had any effect) - drop the .npmrc entry from CODEOWNERS Naming any package in trustedDependencies replaces Bun's curated default-trust list, so only isolated-vm and sharp may run install scripts. Docker is unaffected — all three Dockerfiles pass --ignore-scripts explicitly and rebuild isolated-vm against the Node ABI by hand.	2026-07-24 12:03:55 -07:00
3 people	6dcc65be89	feat(skills): add skill editors (#5705) ... * feat(skills): permissions layer * chore(db): drop skill_member migration 0261 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0262 on latest staging Same DDL as the dropped 0261 (skill_member table, enums, indexes, skill.workspace_shared) plus the hand-written write-user backfill, renumbered after staging's 0261. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0263 after staging merge Staging claimed 0262 (strong_storm); same DDL plus the hand-written write-user backfill, renumbered on the merged snapshot chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(db): drop skill_member migration 0263 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0264 after staging merge Staging claimed 0263 (workflow_fork_sync_excluded); same DDL plus the hand-written write-user backfill, renumbered on the merged snapshot chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * make editing skills full page * fix disclaimer * edit access msg * fix lint * chore(db): drop skill_member migration 0264 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0265 after staging merge Staging claimed 0264 (fat_ikaris); same DDL plus the hand-written write-user backfill, renumbered on the merged snapshot chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix tests * simplify system * fix * fix lint * add mship skills docs * chore(db): drop skill_member migration 0265 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0266 after staging merge Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deps): override zod to 4.3.6 to dedupe nested copies breaking type-check better-auth 1.6.23 and fumadocs-mdx resolve ^4.3.6 to a nested zod 4.4.3, which makes @sim/auth's inferred betterAuth types non-portable (TS2883) and split docs onto a second zod instance. Both ranges accept the repo-wide pinned 4.3.6, so a single hoisted copy satisfies everything. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix lint * feat(skills,tools): fullscreen skill create + shared custom tool editor Moves the rich-markdown and custom-tool editing surfaces out of modals and onto full-page surfaces, and collapses the duplicated chrome behind shared components. Skills - Add /skills/new, a full-page create surface mirroring the skill detail page (CredentialDetailLayout + DetailSection + unsaved-changes guard). "Add to Sim" navigates there instead of opening a modal. - Import moves to a header action (SkillImportButton) backed by a shared readSkillFile helper; the GitHub-URL import and its /api/skills/import route are removed. - Skill name validation is now one shared validateSkillName, replacing three copies of the kebab-case rule and its messages. - The skill editor roster renders through the shared MemberRow instead of re-deriving its identity block, with a locked role control and a lock-reason tooltip explaining inherited workspace-admin access. Custom tools - Extract the canvas modal's schema/code editors into a shared custom-tool-editor module (fields, wand generation, schema helpers), cutting custom-tool-modal.tsx by ~900 lines. - Settings > Custom tools gains a full-page detail sub-view (SettingsPanel + SettingsSection + saveDiscardActions), deep-linkable via ?custom-tool-id. Rows are clickable; delete now lives only in the detail view. - Replace legacy Button/Input/Badge/Label with the chip family, move chip-field chrome into CodeEditor behind an error prop, and delete its dead wand button. Rich markdown field - maxHeight is now opt-in: omit it on a page and the editor grows with its content so the page owns the only scrollbar. Modals pass explicit caps. - The field variant drops to font-weight 400 to match adjacent chip fields. * fix(skills): address review round on create navigation, 409 copy, and editor audit - Skill create navigated using the first element of the upsert response, but that endpoint returns the caller's whole skill list (built-ins prepended) — match the new skill by its workspace-unique name instead. - The suggested-skill 409 toast claimed the skill existed but was not shared and told the user to ask a skill admin. Every workspace member can already see and use every skill, so a 409 only means the name is taken. - Adding an editor emitted the skill_shared event and SKILL_MEMBER_ADDED audit even when onConflictDoNothing skipped the insert on a concurrent add. Gate both on the insert actually returning a row. * chore: format skills-resolver test import * fix(skills,tools): audit fixes — autocomplete boundary, resize clipping, error routing Two real regressions introduced while simplifying the extracted editor: - The schema-param autocomplete's trigger was rewritten to match a trailing identifier, but the completion still split on separators. The two disagreed, so typing `data.ci` opened the menu and selecting replaced `data.ci` whole — eating the member-access prefix. Both now share one SCHEMA_PARAM_WORD regex. - The uncapped markdown field measured its height only on value change while always setting overflow-hidden, so any width change that re-wrapped lines clipped the tail with no scrollbar to reach it. Now re-measures via ResizeObserver. Also from the audit: - Generation writes bypass the code field's change handler, so an open autocomplete stayed over a disabled streaming editor; close it on busy. - Delete failures rendered in the Schema section's error slot on both custom tool surfaces; route them to a toast instead. - Skill create navigated away while still dirty, stranding the unsaved-changes guard's history sentinel so Back landed on an empty create form. - The Description field on skill create never received its error border. - Drop a double-applied opacity-50 (the editor already dims when disabled), a dead try/catch around a non-throwing call that also shadowed the error prop, and a stale reference to a /tools page that does not exist. - Docs still described the removed GitHub-URL import and the old Add Skill dialog; rewrite for the create page and file/paste import. * feat(tools): read-only tool detail, create lands on the new tool, drop dead wand prompt API - Viewers without edit rights could not open a custom tool at all, while the equivalent skill and custom-block surfaces both offer a read-only view. The detail page now takes `readOnly`: editors inert, no Save/Discard/Delete, no Generate. Creating still requires edit rights. - Creating a tool bounced back to the list while creating a skill lands on the new skill. Tools now do the same. The upsert returns the workspace's whole tool list (newest first) rather than just the new row, so the id is matched by title instead of by index — the same trap that produced the skill-create navigation bug. - Remove `openPrompt`/`closePrompt` from useWand. `closePrompt`'s last callers went away with the custom-tool-modal extraction and `openPrompt` had none before it; nothing reads `isPromptVisible` any more either. * fix(tools): read-only editors, design-system wrench, skills-matching tool identity - readOnly never reached the editors: the prop gated actions and Generate but the schema and code fields were still typable for viewers without edit rights. Wire disabled through both fields into CodeEditor. - The row icon used lucide's Wrench (strokeWidth 2) where @sim/emcn/icons ships one drawn for this system (1.55, tuned viewBox), and it inherited body text colour instead of --text-icon. Swap it. - Give the tool detail page the same identity heading as skill detail: tile, name, and description at the top left, instead of only a header title. - Extract ResourceTile so the skills and tools tiles share one definition (SkillTile now composes it), and add an opt-in `iconFilled` to SettingsResourceRow so the tools list tile matches the skills gallery. Both default to today's behaviour for every existing consumer. * fix(mentions): use the product's own glyph for every @ mention kind The `@` menu and the inserted chip mapped kinds to arbitrary lucide icons — `Sparkles` for a skill, a generic `File` for every file — while the rest of the product has a settled glyph per resource. Mirror CHAT_CONTEXT_KIND_REGISTRY, which Chat's `@` menu already renders from: - skill now uses AgentSkillsIcon, the same glyph SkillTile shows everywhere - workflow / folder / table / knowledge use the @sim/emcn/icons set the sidebar and the chat registry use - file derives its icon from the filename extension, so a .pdf and a .csv are distinguishable, matching the file list and Chat's context chips - integration keeps the block's brand icon from the registry Also drop the generic placeholder. `kind` is untrusted — the node schema defaults it to `''` and a hand-written `sim:` link can carry anything — but an unrecognized kind now yields no icon instead of a meaningless box, which is what the chat registry does. The menu already guarded a missing icon; the chip now does too, so this cannot crash on a malformed link. * chore(db): drop skill_member migration 0266 for regeneration on latest staging * feat(db): regenerate skill_member migration as 0267 after staging merge --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-07-23 19:52:00 -07:00
4 people	d24bc7eccb	feat(agent-stream): thinking and tool streaming (#5671) ... * feat(agent-stream): add agent-events thinking/tool streaming for chat and canvas Ship the agent-events-v1 protocol with provider tool loops, dual-gated chat thinking, DeepSeek/Groq/OpenAI reasoning wiring, and ChatGPT-like thinking chrome. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): clear stuck streaming UI and format db snapshot Biome was failing CI on migrations/meta/0261_snapshot.json. Also settle assistant streaming/tool flags when SSE ends without a terminal frame, without clobbering Stop's finalized content. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): satisfy biome format and import order Auto-format the sim package for CI lint:check, and repair the Anthropic streaming tool-loop payload after an unsafe delete-to-undefined rewrite. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): keep drained answer on abort and update migration journal test Treat AbortError from reader.cancel as a cancelled pump result so soft-complete retains answerText. Point the workspace storage migration journal assertion at 0261_chat_include_thinking. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(chat): keep Stop notice when server emits cancel error Ignore terminal SSE error frames after the user aborts so "Client cancelled request" cannot overwrite "Response stopped by user". Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(chat): ChatGPT-style thinking shimmer and stick-to-bottom scroll Add left-to-right shimmer on live thinking label/body, keep scroll working by shimmering an inner node, and follow the answer only while near the bottom. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): stop pump on client disconnect; soft-complete agents only Abort the agent stream pump when the projected HTTP body is cancelled so provider work does not continue after disconnect. Limit AbortError soft-success to Agent blocks so Function/HTTP cancels still fail in logs. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): persist includeThinking across pause snapshots Paused chat runs with Include thinking enabled were dropping the flag when serializing the pause snapshot, so resume always rebuilt streams without thinking/tool SSE frames. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): keep drained answer text when stream times out Persist pump answerText onto the streaming execution before throwing on timeout, and carry that partial content into the failed block output so logs match what the client already saw. Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(chat): auto-collapse tools chrome when tool streaming ends Match thinking UX: open while tools run, collapse when finished, and keep the panel open only if the user manually reopens it. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): settle canvas stream chrome on failure paths Clear agentStreamActive and settle running tool chips when blocks error, timeouts cancel runs, or execution ends without stream:done so the output panel does not stay on live Thinking/Using tools chrome. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lint): organize imports in terminal console store Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): mark open tools cancelled on HITL pause Pause can interrupt a tool loop without tool end events; settling those chips as success incorrectly showed unfinished tools as complete. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(db): drop branch-local 0261 migration ahead of staging merge * chore(db): regenerate include_thinking migration as 0266 post staging merge * fix(providers): resolve type errors in streaming tool loop call sites * fix(agent-stream): gate agent events opt-in and correct provider loop behavior - streamToolCalls and provider thinking requests now require run-level agentEvents opt-in (canvas on, chat dual-gated, API off) so existing runs keep pre-agent-events behavior exactly - OpenAI reasoning summaries opt-in + strip-and-retry on unverified-org 400 - streaming loops run tool postProcess again (firecrawl/exa async results) - bedrock live loop falls back to silent path for responseFormat - deepseek: reasoning_content pass-back unconditional, 'none' sends disabled - groq: x_groq.usage fallback, reasoning params gated, qwen none disables - gemini: functionCall parts echoed verbatim, local ids only for events - truncated turns (max_tokens/length) no longer execute partial tool calls - MAX_TOOL_ITERATIONS exit flushes last turn text as final answer - iterations reports actual model calls; shared loop plumbing extracted * refactor(agent-stream): consolidate protocol, dedupe client/server plumbing, hygiene - canonical ChatStreamFrame union + type guards consumed by server emitters and the chat client; stream_error restored to legacy log-only handling - strip thinking/tool args from providerTiming on public final envelopes - shared tool-chip lifecycle module for chat, canvas, and console store - shared sink-to-execution-events forwarder replaces the copy-pasted adapter in the execute route and HITL manager; LIVE_ONLY event set shared - stream:thinking payload field renamed data->text; canvas thinking batched - abort reasons carried as AbortError DOMExceptions so raw fetch consumers classify correctly; thinking cap renamed to chars and scope-documented - kimi wired for agent events like the other compat providers - deleted dead exports/step-N comments; fixtures match real wire shapes; loop tests use explicit mocks instead of importOriginal * test(agent-stream): cover the dual-gated execution path and typed abort reasons - chat route tests assert agentEvents reaches executeWorkflow only when policy and protocol header agree - execution-limits tests assert AbortError-typed reasons - executor metadata type carries agentEvents * fix(deploy-modal): align include-thinking spacing with the modal's 6.5px rhythm * docs(agent-stream): autogenerate per-model thinking/tool stream support on the Agent block page - capabilities.thinking.streamed ('full' | 'summary' | 'none') on models.ts, explicit for the Anthropic family where visibility varies per generation; getThinkingStreamVisibility exposes the derivation for docs and UI alike - scripts/sync-agent-stream-docs.ts regenerates the support tables between markers in workflows/blocks/agent.mdx from the model registry and STREAMING_TOOL_CALL_PROVIDERS; --check fails on drift or missing metadata - wired agent-stream-docs:check into CI next to the other sync gates * feat(anthropic): request summarized thinking display for omitted-default Claude models The newest Claude generations (Fable 5, Sonnet 5, Opus 4.8/4.7) default thinking.display to omitted — empty thinking blocks, no deltas. On agent-events runs Sim now opts back in with display: 'summarized', driven by the registry's streamed metadata; legacy runs keep the exact pre-agent-events request shape. Registry, generated docs, and the family capability table updated accordingly. * docs(skills): cover thinking.streamed and agent-stream docs sync in model skills * chore(deps): upgrade @anthropic-ai/sdk to 0.114.0 and adopt official types - adaptive thinking, display, and output_config are now SDK-typed; the only remaining custom payload field is output_format (beta-header structured outputs, which the SDK models as output_config.format instead) - anthropic stream events narrow on the SDK's discriminated unions instead of anonymous casts; compat deltas type content/tool_calls from the OpenAI SDK with vendor reasoning fields as an explicit optional extension - @sim/auth exposes an explicit VerifyAuth contract so its declarations no longer reference better-auth's nested zod instance (TS2883 under fresh install layouts); realtime consumer aligned - docs app zod pinned to the repo's exact 4.3.6 so ai SDK types bind the same zod instance (docs type-check was latently broken) - knowledge embedding tests made hermetic against local .env keys and hosted rotation fallback * refactor(providers): replace legacy as-any stream casts with annotated typed casts * refactor(providers): finish provider audit — remove dead byte-stream helper, annotate remaining legacy casts Audit of all 26 providers for the agent-events feature confirmed every streaming execution declares agent-events-v1 and every adapter emits AgentStreamEvent objects. Cleanup from the audit: the unconsumed legacy createOpenAICompatibleStream byte helper is deleted, and the remaining streamResponse-as-any casts (xai, nvidia, kimi, meta, zai, sakana) are annotated typed casts matching the groq/deepseek fix. * feat(streaming): stream answer text live during tool loops via turn_end protocol The live tool loops buffered all answer text per model turn (classification of intermediate vs final is only known at turn end), so gated surfaces saw thinking stream, then dead air with the thinking chrome stuck open, then the whole answer at once. Loops now emit text deltas live as `turn: 'pending'` plus a `turn_end` event per turn. The pump buffers pending text and projects it to the byte path (answerText/logs/memory/legacy clients) only on a final turn_end, so all settled semantics are unchanged. Gated surfaces render the pending text as it streams and reconcile with a reset when a turn resolves to tools: - public chat: live `chunk` frames from the sink + dual-gated `chunk_reset`; byte-path frame emission is suppressed to avoid duplicates (kept for response-format transformed streams via clientStreamTransformed) - canvas: forwarder emits live `stream:chunk` + `stream:chunk_reset`; the execute route and HITL resume readers stop re-emitting byte chunks; panel chat tracks per-block segments and replaces content on flush - chat client: per-block text segments, chunk_reset handling, and thinking chrome now settles on tool start as well as first answer chunk * fix(streaming): address validated review findings across provider gating and reset reconciliation Three-reviewer pass over the branch, findings validated against staging: - agent-handler forwards agentEvents to executeProviderRequest — the flag was computed but dropped in the field-by-field copy, so provider-side thinking requests (OpenAI summaries, Gemini includeThoughts, Anthropic summarized display) never activated on opted-in runs - openai: restore summary:'auto' alongside explicit reasoning effort — staging always paired them; gating summary purely on agentEvents changed legacy payloads - gemini: Gemini 2 + tools + responseFormat falls back to the silent path; the live loop never applied the deferred responseSchema for AUTO tools - openai-compat loop: malformed tool-argument JSON fails the call instead of executing with defaulted {} args (staging parsed inside the execution try) - openai-compat parser: a vendor id arriving after a synthesized start no longer renames the call (start/end ids stayed consistent) - stream-pump: abort closes the byte projection so a drain blocked on backpressure cannot deadlock teardown - chunk_reset removes the block from the client text order (deployed chat + panel chat) so a reset block re-registers at arrival position — fixes separator/order corruption when parallel blocks stream around a reset - resume route echoes the negotiated X-Sim-Stream-Protocol response header (parity with the chat route); docs: [DONE] wire shape + final-vs-error terminal semantics corrected * chore(deps): exempt pinned @anthropic-ai/sdk 0.114.0 from the release-age gate CI's bun install --frozen-lockfile blocks 0.114.0 (published 2026-07-23, younger than the 7-day supply-chain gate). The pin is exact and was vetted for the agent-events streaming work; following the existing bunfig pattern, the exclusion ages out on 2026-07-30 and should be dropped then. * chore(providers): fix double-cast-allowed annotation placement for the strict boundary audit The audit only recognizes the annotation on the line directly above the cast; two annotations had drifted behind intervening code lines (groq stream params, deepseek loop messages) and the OpenAI reasoning-summary widening cast was never annotated. No behavior change. * fix(chat): settle straggler tool chips as error when final reports failure A failed run can still terminate with a `final` frame carrying success: false; running chips previously settled green regardless of the outcome. * fix(canvas): wire agent stream chrome into run-from-block Run-from-block executions emit the same live stream:thinking/stream:tool events as full runs but registered none of the handlers, so the terminal never showed thinking or tool chips on that path. The per-run chrome (batched thinking writes + tool chip lifecycle + settlement on stream done, block error, and every terminal execution state) is extracted into a shared createAgentStreamChrome factory consumed by both paths. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>	2026-07-23 19:39:03 -07:00
Waleed	78fb2c0679	chore(deps): bump next to 16.2.11 to clear security advisories (#5890) ... * chore(deps): bump next to 16.2.11 to clear security advisories Patches SSRF, cache confusion, DoS, and middleware-bypass advisories (GHSA-89xv-2m56-2m9x et al.) affecting next < 16.2.11 across apps/sim, apps/docs, and packages/emcn. Excludes next/@next/env from the minimum-release-age gate until the 7-day window elapses on 2026-07-28. * chore(deps): drop aged-out typescript entries from release-age excludes typescript and @typescript/typescript6 passed the 7-day minimum-release-age gate (aged out 2026-07-15 and 2026-07-13), so their exclusions are no longer needed. Keeps @typescript/native-preview (permanent nightly builds) and the Pi packages (age out 2026-07-24).	2026-07-23 10:47:07 -07:00
Theodore LiandClaude Opus 4.8	66d1e61beb	feat(skills): canonicalize skills to a single source with generated .claude/.cursor projections (#5609) ... * improvement(cleanup-skill): parallelize analysis, apply fixes sequentially * improvement(cleanup-skill): add comment-reduction pass; mirror 6 missing skills into .claude/commands * fix(cleanup-skill): substitute parsed scope into analysis passes instead of literal <scope> * fix(cleanup-skill): parse fix token anywhere; preserve pass labels through convergence for ordered apply * fix(cleanup-skill): apply Step 1 proposals content-anchored, re-derive when a prior pass invalidated the snippet * fix(babysit-skill): correct garbled --reverse explanation across all three copies * fix(skills): propagate parallel cleanup to cursor/agents copies; disambiguate babysit /ship refs in claude copy * fix(skills): port url-state + comment passes to cursor/agents; clarify converge pass-label ordering * feat(skills): canonicalize skills under .agents/skills with generated .claude/.cursor projections Establish .agents/skills/<name>/SKILL.md as the single source of truth (latest content reconciled per skill from the three drifted copies), and generate the .claude/commands and .cursor/commands projections from it via scripts/sync-skills.ts. Adds skills:sync/skills:check, a CI gate, a pre-commit regen hook, and CONTRIBUTING docs. Structurally fixes prior drift (e.g. abbreviated .claude ship -> full ship). * fix(skills): strip leaked XML tags from skill tails; clarify lint:check has no per-file target Removes stray </content>/</invoke> markup that leaked into add-block, add-connector, add-hosted-key canonical skills, and reword the cleanup skill's lint step to note bun run lint:check runs repo-wide via turbo (no per-path API). Projections regenerated via skills:sync. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QjefwescJoHZ6zcc3C17FR * fix(add-block-skill): restore unknown-output stop in Final Validation Re-add the "if any tool outputs are still unknown, tell the user instead of guessing block outputs" step that was dropped when Final Validation step 5 became the BlockMeta template check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QjefwescJoHZ6zcc3C17FR --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 19:12:11 -04:00
Vikhyath Mondreti	ef7c8e24b2	feat(platform): settings permissions, admin, billing attribution (#5545) ... * fix(invites): preserve active organization for external access Keep organization activation server-owned so failed membership checks cannot clear a valid session context. * feat(admin, billing, settings): cleanup settings visibility, billing actor resolution, new admin routes * address comments * chore(db): reset pending migrations before staging merge Remove locally generated migrations so they can be regenerated against the latest staging schema without preserving stale snapshots or numbering. * regen migrations * address comments * chore(db): reset generated migrations before staging merge Remove this branch's generated migrations so they can be regenerated against the latest staging schema with fresh numbering. * upgrade global work * fix lint * address comments * legacy callbacks correctness * address comments * update * guardrail attribution	2026-07-13 00:53:49 -07:00
Waleed	507cee1187	fix(integrations): repair corrupt icons, backfill missing block metas, restore scroll on back-nav (#5342) ... * fix(integrations): repair corrupt icons, backfill missing block metas, restore scroll on back-nav - Restore 7 brand icons (Google, Outlook, MongoDB, Postgres, OpenRouter, Groq, Cerebras) whose SVG path data was corrupted by a past bulk reformat, flooding the integrations page console with <path> parse errors; add a check:icon-paths CI gate that validates every icon d attribute (operand counts + arc flags). - Backfill BlockMeta (tags/url/templates/skills) for postgresql, mysql, ssh, sftp, smtp — previously catalog integrations with empty detail pages; add an integration meta-coverage CI check so every catalog block must have a meta. - Add scroll-position restoration for the integrations index/detail inner scroll containers so browser Back returns to where you were. - Remove the error digest pill from the shared workspace ErrorShell (kept in logs, dropped from UI). * fix(integrations): make scroll restoration robust — value-based echo detection + Back/Forward-only gate Addresses review: replace the racy programmatic-scroll flag with value comparison (a restore's echo equals lastApplied and is ignored, so a stuck flag can never drop the first user scroll or overwrite the saved target), and gate restoration on popstate history traversals so fresh push navigations open at the top instead of jumping mid-list. TSDoc-only comments. * fix(ci): attribute icon-path errors for export-const icons too Greptile review: iconNameAt only matched 'export function', so a malformed path inside an 'export const XxxIcon = (...)' arrow-function icon would be misattributed to the preceding function-declared icon. Match both forms (mirrors check-bare-icons indexIconBodies).	2026-07-01 16:58:45 -07:00
5 people	ad19f7fc40	improvement(landing): refine hero and mothership visuals (#5181) ... * stash * feat(landing): mothership feature stages + pre-footer CTA Tell-then-show landing: the Mothership section defines the five capabilities (Mothership · Pod · Formation · Dispatch · Return); the Features section now shows each as a real Sim UI callout floating over a static, edge-faded platform backdrop (Linear's "callout over a faded platform" pattern). - FeatureStage template: copy + masked static LandingPreview + elevated callout - LandingPreview: static autoplay=false snapshots with per-stage view/workflowId - Callouts: Mothership chat, model picker, parallel-agents Formation graph, deploy targets, logs table - Pre-footer CTA set over the Mothership render; removed the old capabilities grid Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(landing): reusable platform-page + solutions-page layouts and routes Add config-driven, padding-safe layouts consumed by route pages: - platform-page: hero (shared CTA) + centered logos + N card rows (3|4), JSON-LD, single <h1>, server-only; Workflows route as first consumer. - solutions-page: structural mirror (kept separate to diverge later); IT, Engineering, Finance, Compliance, HR routes under /solutions. - Hoist shared LandingShell/HeroCta/Logos to components/ (top-level = shared); refactor hero to consume them. - Restructure all of (landing) to the workspace folder-per-component convention (each component in its own folder + index.ts barrel). * refactor(landing): convert hero-visual CSS-module keyframes to Tailwind Move the hero-visual + stage-home keyframe animations out of CSS modules into tailwind.config (matching the existing dash-animation pattern) and delete both module.css files. Components now use animate-hero-* utilities + arbitrary properties for the per-element delays, SVG stroke draw, and gradient shimmer; reduced-motion preserved via motion-reduce: variants. Upgrade the shimmer's hardcoded #b4b4b4 to the --text-subtle token. brand-tokens.module.css is intentionally kept: it reassigns --surface-*/ --text-* token VALUES via a doubled-class selector for specificity over .light, which Tailwind utilities cannot express. * refactor(landing): move brand palette from CSS module into LandingShell Replace brand-tokens.module.css with a BRAND_TOKENS constant of Tailwind arbitrary-property utilities applied on the LandingShell wrapper, so the brand hex lives in the component, not a stylesheet. They emit in the utilities layer and override .light (@layer base) by cascade order — verified the brand --text-primary (#121212) wins over .light (#1a1a1a). No more .module.css files remain in the landing. * chore(landing): remove Testimonials from the home page for now Drop <Testimonials /> from the landing composition (component kept for re-adding later). * feat(landing): hero send→loader→workflow animation + landing WIP Hero visual: clicking send zooms into the button, morphs the disc into the gooey thinking loader (held, then cycling), slides it straight across to a phrase indicator with the camera following (no zoom-out), then zooms back out as the reply types and the chat morphs into the GitHub→Agent→Jira workflow. The chat card holds a fixed size through the zoomed scene and the greeting reserves its space, so nothing drifts; the user bubble reveals only on zoom-out. Loader ink tweens dark→gradient via the thinking-loader stop-color/flood-color transition. Also folds in in-progress landing work: knowledge + integrations feature callouts, CTA chat, mothership + line-glyph, wordmark tweak; removes the ethos and testimonials sections. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(landing): responsive pass for iPad + mobile Make the landing page fully responsive while keeping the desktop layout byte-identical (desktop classes stay the unprefixed baseline; smaller screens layer max-* overrides on top). - Navbar: hide desktop clusters below lg, add MobileNav hamburger sheet (scroll-lock, Escape/tap close, reduced-motion aware) - Hero: collapse the absolute split (visual + logos) to a stacked column below xl so iPad-landscape avoids the headline/visual collision - Mothership: 4-col grid steps to 2 (tablet) then 1 (phone) - Features: drop the floating callout below md, show the un-masked backdrop preview full-width - CTA + Footer: scale type/padding; footer 7-col steps to 3 then 2 - Document the breakpoint strategy in the landing CLAUDE.md Also includes the in-progress mothership goo/iso brand marks and the marks-lab preview route the section depends on. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(landing): align hero visual panel to text + logos extent Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(landing): delay hero user bubble until card finishes expanding The grey user bubble's fade-in raced the card's upward grow on send. Hold the bubble's reveal until after the parent-driven grow settles so the card expands fully before the bubble appears. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(landing): isolate new landing — remove dead old-folder code + --landing-* coupling - Delete dead (landing) auth-modal (a duplicate of (home)'s, still on the old --landing-* / dark tokens) — its removal drops the new landing's last styling tie to the old landing. - Delete 5 merge-orphaned, zero-consumer callouts (deploy-callout, mothership-chat-callout, mothership-chat-preview, workflow-graph-preview, model-picker-preview). - Relocate the one live preview (logs-table-preview) into its consumer features/components/ + add a barrel; dissolve the owner-less feature-callouts/ shell. - Fix stale --landing-bg-surface reference in landing-preview-mount. (landing) now has zero (home) imports and zero --landing-* token usage. * refactor(landing): token-map hex, fix a11y/SEO, align structure Styling (within (landing)): - Replace ~90 hardcoded hex colors with the in-scope brand tokens they already equal (--surface-*/--text-*/--border*); divider edges -> --border, field/card edges -> --border-1. Delete the redundant C color-palette mirrors in the landing-preview home/sidebar and route them through tokens. - Convert static inline SVG styles (display:block/outline:none) to Tailwind. - 6 un-tokenizable hexes remain (dark send-button fills, status-green dot) — no brand token exists; left as-is. a11y / SEO: - Decorative mothership goo/iso marks: role='img'+aria-label -> aria-hidden. - Preview chrome titles <h1> -> <span> (kills duplicate client-only H1s). - sitemap.ts: add /workflows and the five /solutions/* routes. Structure: - Folder the bare logo-mark/mobile-nav leaves + barrels; complete the navbar components barrel and consolidate navbar.tsx to a single barrel import. * style(landing): restore taller hero panel with border-shadow chip chrome - Revert the right visual panel to the previous full-height framing (top-8 bottom-8) — hero text (pt-[112px]) and the logos panel are untouched, so their positions and spacing are unchanged. - Apply the canonical border-shadow chip surface: --surface-2 fill + the shared chipBorderShadowRing (1px hairline ring + soft drop shadow) from emcn, the documented chrome for a landing media panel. * feat(landing): swap Volvo for thinkproject and reposition hero logos - Replace Volvo with the thinkproject wordmark (official SVG, tagline/descriptor cropped out, all paths unified to --text-primary #1a1a1a; aspect 6.01). - Reorder the shared 6-logo set so the 3x2 hero grid reads: Rivian|VW (top-left), eXp Realty (top-center), Russell (top-right); Artie (bottom-left), thinkproject (bottom-center), Mobile Health (bottom-right). - Enlarge Rivian|VW a touch (height 15 -> 17, same aspect). - eXp Realty, Artie, Russell, Mobile Health, Rivian|VW all retained. * style(landing): size hero description with the type scale (text-lg) Replace the arbitrary text-[20px]/text-[16px] on the hero description with named scale tokens — text-lg (18px) desktop, text-md (16px) on phones — a touch smaller and the canonical lead size (1.2x the platform's 15px base). * style(landing): hero headline "for AI automations" with break after "agent" Replace "solving automations" with the higher-intent "AI automations" and move the line break after "agent" so "for AI automations." sits on the second line. * style(landing): unify CTA radius and box hero logos in cards - HeroCta email bar: rounded-[13px] -> rounded-lg, so the bar, the inset Book-a-demo chip, the Sign-up chip, and the navbar chips all share one radius. - Hero logos: box each wordmark in a bordered --surface-1 card (platform card chrome: rounded-lg + --border-1, 100px tall) on a responsive 3-up grid (2-up on phones) at a consistent gap-5 rhythm. Wide marks scale to fit (max-w-full h-auto). The platform/solutions 'row' layout stays bare wordmarks. * style(landing): concentric CTA bar radius + tighter logo cards - HeroCta email bar back to rounded-[13px] (= inner chip 8px + ~5px inset) so the Book-a-demo chip's right corners nest concentrically inside the bar. - Logo cards: smaller and tighter — h-20 (80px), px-4, gap-3 (12px, the product UI card-grid rhythm). * style(landing): restore 100px logo cards, scale icons down 15% The 80px cards read too wide-for-their-height. Restore h-[100px] (keeping the tighter gap-3/px-4) and instead shrink the wordmarks to 0.85x their optical size in the grid via GRID_ICON_SCALE — row layout unchanged. * style(landing): match sign-up radius to email bar + shrink logo icons - Sign-up chip overridden to the email bar's rounded-[13px], so the two hero CTAs share one corner radius. - Logo icons: GRID_ICON_SCALE 0.85 -> 0.65 and card padding px-4 -> px-2; card dimensions (h-[100px], gap-3) unchanged. * style(landing): shrink hero logo cards Cards read massive — too tall (100px) and stretched to fill the panel. Drop to h-16 (64px), cap width at w-[150px], and make the grid w-fit so it hugs the cards instead of stretching. gap-3 and the 0.65 icon scale unchanged. * style(landing): upscale hero logo cards ~25% Cards read too small. Bump all dimensions: h-16->h-20 (80px), w-[150px]->w-[180px], px-2->px-3, and icon scale 0.65->0.8. Grid stays content-hugging at gap-3. * style(landing): taller logo cards, larger icons, reorder top row - Card height h-20 -> h-[88px] (width w-[180px] unchanged), icon scale 0.8 -> 0.85. - Top row reordered: eXp (left), Russell (center), Rivian|VW (right). * style(landing): more card height, swap top-row Rivian/eXp back - Card height h-[88px] -> h-24 (96px); width unchanged. - Top row: Rivian|VW (left), Russell (center), eXp (right). * feat(landing): add "Trusted by technical teams at" label above hero logos Top-left, gap-3 above the logo grid (matching the grid rhythm); text-sm (navbar text size) in --text-muted (the label token). * style(landing): recolor logos to --text-body, match label gap to hero rhythm - Recolor all six customer logo SVGs to #3b3b3b (--text-body light value), so they match the Sim navbar wordmark's color. Landing is light-only, so the hardcoded value always equals var(--text-body). - Trusted-by label gap gap-3 -> gap-[22px] (the hero's description->CTA spacing). * style(landing): scale hero CTA down a hair, drop radius to the nav chip's Sign-up read too round. Take the bar + Sign-up to h-[40px] / rounded-lg (8px, the navbar chip radius), and keep the inset Book-a-demo concentric: h-[2em] + rounded (4px) with a 4px inset (8 = 4 + 4). * style(landing): round Book-a-demo to rounded-md to match the bar curve rounded (4px) read too square next to the bar's rounded-lg (8px). Bump to rounded-md (6px) — echoes the bar's curvature, still inside the 4px inset. * style(landing): match Book-a-demo proportions to the navbar chip Restore h-[2.143em] (the chip's 30/14 height ratio); with px-[0.571em] (its 8/14 padding ratio) and the 16px label, Book-a-demo now shares the navbar chip's exact height/padding/text proportions. * style(landing): equal inset around Book-a-demo (h-[30px]) Button was h-[2.143em] (34.3px) -> only ~1.9px top/bottom vs 4px right inside the bar's 38px inner box (40px minus the 1px border). Drop to h-[30px] (the nav chip height) so it centers to an equal 4px inset on top, bottom, and right. * style(landing): enlarge Book-a-demo to h-[32px], tighten inset to 3px h-[30px] read too small/airy in the bar. Bump to h-[32px] and pr-[4px] -> pr-[3px] so the inset is an equal, snugger 3px on top, bottom, and right. * style(landing): lift hero logos off the bottom again (pb-20) Restore the 80px bottom padding so the logos rest 112px above the section bottom (mirroring the hero text's 112px top) instead of sitting flush with the visual panel's bottom. max-xl:pb-0 keeps the stacked layout tight. * improvement(landing): refine hero and mothership visuals * fix(landing): cap hero fold height so it doesn't stretch on huge monitors The section was min-h-[calc(100vh-62px)], so on very tall displays both absolute panels (top-8 bottom-8) stretched — the visual panel grew gigantic and the bottom-anchored logos sank to the very bottom. Cap the fold at 960px via h-[min(calc(100vh-62px),960px)] (min-height can't be capped by max-height): the whole hero stops growing, panels/logos stay proportioned like a large laptop, and the next section just starts below. Laptops (<=16in) are unaffected; max-xl:h-auto keeps the stacked layout below xl. * refactor(landing): session cleanup — DRY CTA label, drop dead grayscale Final tidy after this session's hero/CTA/logo iteration: - hero-cta: extract the duplicated 16px label knob (px-[0.571em] + text-[16px] + font-size:inherit) into a single CTA_LABEL constant, matching the 'single knob' the TSDoc already describes — used by both Book-a-demo and Sign-up. - logos: remove the grayscale filter (now a no-op — all wordmarks were recolored to a single #3b3b3b), inline the single-use LOGO_GAP_X, and flatten the nested cn() into plain layout ternaries (dropping the now-unused cn import). * improvement(landing): animate mothership illustrations * style(landing): solid-ink branding + hero cursor/loader polish Branding: drop the bespoke BRAND_TOKENS palette and bottom-reveal from LandingShell (use the platform's own light tokens); re-ink the wordmark, logo-mark, and hero loader from the gradient+glow to a solid --text-body so the marks read as one ink with the nav text. Add a `shimmer` prop to ThinkingLoader for a static --text-body label, and stroke the squeeze arcs with the shared gradient. Hero visual: the cursor now enters from below the field and chases the send button live through the zoom (retimed beats, no arrive-then-wait); the greeting fades in gently instead of shimmer-revealing; the click ring becomes a press-dip (hero-cursor-press replaces hero-click-ring and hero-greeting-reveal). Extract BlockHandles so the morphed GitHub card carries a real edge handle in scene space; seed the compose card at its true height; pop the sent bubble in immediately. * improvement(landing): update feature iso-marks to perfected geometry Re-author the four Mothership iso-mark illustrations (Integrate, Ingest, Build, Monitor) on the refined isometric geometry, keeping the existing animation vocabulary intact: hover line-draw plus per-mark auto-motion (integrate float, ingest pulse, monitor panel-separate, build grid-flow). Map the raw exports onto the shared token palette/line weight for consistency and tune per-mark sizes for one optical weight. Build is now pure CSS (grid-flow replaces the RAF wave), so it drops 'use client' and renders as a server component. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(landing): add pricing, privacy, terms, and changelog pages - New public /pricing page: Free/Pro/Max/Enterprise cards with the full comparison breakdown transposed from shared upgrade data + JSON-LD; prices, CTAs, and features derive from shared billing constants so they can't drift from the in-app upgrade page. - Migrate /privacy, /terms, and /changelog into the (landing) route group via a shared prose-page system (single source of truth for legal/prose chrome). - Landing polish: solid-ink iso-mark illustrations + footer/cta/features/ mothership spacing and token cleanups; sitemap adds /pricing. - Audit pass: crawlable ChipLink CTAs, correct heading hierarchy, structured-data featureList derived from the visible comparison data, legal plan name Team->Max. * large edits across landing finalization * feat(auth): port OAuth-only signup + Microsoft provider from staging Align auth-page logic with origin/staging (PR #5073) while keeping the new chip-styled UI: - Add Microsoft as a better-auth social sign-in provider (auth.ts) and surface it through the OAuth provider checker, providers API + contract, login/signup forms, SocialLoginButtons, and the landing auth modal. - Gate email/password signup behind the emailSignupEnabled server flag (DISABLE_EMAIL_SIGNUP) so signup becomes OAuth-only when configured. - Add DISABLE_MICROSOFT_AUTH / DISABLE_EMAIL_SIGNUP env + feature flags. * fix(icons): render brand icons legibly when bare and on light tiles (#5292) Monochrome brand icons hardcoded a single white or black fill matched to their colored tile, so they vanished when rendered bare on the home Suggested actions list (white-on-white in light mode, black-on-black in dark mode). Convert those marks to currentColor so they adapt to context, and make tile foregrounds contrast-aware via getTileIconColorClass instead of a hardcoded text-white. Also centralize all color math in apps/sim/lib/colors (perceived brightness, hex/rgb/hsl conversion, contrast-text) and route every consumer through it: the bare-icon audit, block tiles, logs trace view, whitelabeling theming, workspace presence, and the PPTX renderer no longer carry duplicate copies. Adds a bare-icon CI audit (scripts/check-bare-icons.ts) and authoring guidance. --------- Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu> Co-authored-by: andresdjasso <andresdjasso@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Waleed <walif6@gmail.com>	2026-06-30 13:50:04 -07:00
Theodore LiandClaude Opus 4.8	2686f043bc	perf(dev): SIM_DEV_MINIMAL_REGISTRY mode to slash local dev-server RAM (#5223) ... * perf(dev): SIM_DEV_MINIMAL_REGISTRY mode to slash local dev-server RAM Adds a dev-only escape hatch (`bun run dev:minimal`, or `dev:full:minimal` with the realtime server): when SIM_DEV_MINIMAL_REGISTRY=1, a Turbopack/webpack resolve-alias swaps the two heavy registries for tiny curated variants — `@/tools/registry` → 2 tools, `@/blocks/registry-maps` → ~20 core blocks. The shared workspace layout drags the full ~247-tool registry (~2,074 modules) into every route via providers/utils → tools/params, and the editor/executor pull all ~268 block configs; aliasing both stops Turbopack from compiling those graphs at all. To make the blocks alias clean, the heavy block import maps move out of registry.ts into registry-maps.ts (registry.ts keeps only its accessors, importing the maps); its public API is unchanged and full builds/tests use the full maps. The alias is gated on isDev + the flag and is never applied in production. Measured (Turbopack dev, authenticated, /logs): peak next-server RSS ~16 GB → ~4.7 GB, compile 4.9 min → ~18 s; the workflow editor route similarly drops to ~5 GB / ~17 s. Only http_request + function_execute and the curated core blocks work in minimal mode; unset the flag for the full set. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(dev): add table block to minimal registry; doc block registration in registry-maps Adds the Table block to the SIM_DEV_MINIMAL_REGISTRY curated set so the tables surface works under dev:minimal. Updates the integration skills/rules and CLAUDE.md to point block registration at blocks/registry-maps.ts (the BLOCK_REGISTRY / BLOCK_META_REGISTRY maps), reflecting that registry.ts now holds only the accessor functions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(dev): rename dev:full:minimal → dev:full:minimal-registry Matches the dev:full:* formatting and makes the suffix self-explanatory — it is the registry that's minimal, not the dev stack. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-27 16:41:32 -04:00
Waleed	6260eda226	fix(ssr): harden credential query-key factory + fetchers against the 'use client' stub bug (#5206) ... * fix(ssr): move credential query-key factory + fetchers to non-client modules Preventively closes the same 'use client' SSR client-reference-stub class that crashed the tables page. Server-evaluated modules (the credential block def, the workflow-comparison helpers) imported workspaceCredentialKeys / fetchWorkspaceCredentialList / fetchCredentialSetById from 'use client' hook modules, where they resolve to client-reference stubs on the server (a future server call path would throw 'X is not a function'). Extract them into non-client hooks/queries/utils/{credential-keys, fetch-workspace-credentials,fetch-credential-set}.ts (mirroring folder-keys.ts / fetch-workflow-envelope.ts) and import from there. No behavior change — these values were only ever called from browser paths. * docs+ci: codify the 'use client' server-import rule + add check:client-boundary Document the Next.js rule that server code can only render a 'use client' export as a component, never call it (server imports resolve to client-reference stubs that throw — the tables-page crash). Add the rule to .claude/rules/sim-queries.md + a cross-ref in sim-architecture.md. Add scripts/check-client-boundary-imports.ts (wired into CI as check:client-boundary) that flags any value import from a 'use client' module in a server-evaluated, non-JSX surface (prefetch / route handler / trigger / block definition), so this class can't silently recur. Escape hatch: // client-boundary-allow: <reason>.	2026-06-24 18:46:18 -07:00
Siddharth Ganesan	5925651cbc	feat(vfs): add lazy vfs + remove dynamic fields for prompt caching hits (#5138) ... * feat(vfs): add lazy vfs + remove dynamic fields for prompt caching hits * feat(vfs): send typed workspace snapshot for append-only deltas Build the workspace inventory from the primary db (fixes replica-lag staleness) and emit it as a typed VfsSnapshotV1 `vfs` payload alongside the markdown, so the mothership can diff it into append-only baseline/delta messages. Generate the TS contract mirror from the Go-owned JSON schema (sync-vfs-snapshot-contract) and sort connector types so diffs stay byte-stable. * fix(lint): fix lint * fix(vfs): forward the typed snapshot through the branch payload builder The branch buildPayload implementations hand-list the params they pass to buildCopilotRequestPayload and forwarded workspaceContext but dropped vfs, so the typed snapshot never reached the Go request (req.Vfs was always nil and the append-only delta path never engaged). Forward vfs in both the workflow and workspace branches, and add a regression guard asserting the branch threads it through (the bug slipped past tests because post.test mocked the payload builder and payload.test called it directly, bypassing the branch). * improvement(contracts): update vfs contracts	2026-06-19 15:12:42 -07:00
Theodore LiandClaude Opus 4.8	f277f5fba5	feat(db): zero-downtime migration safety lint + db-migrate skill (#5041) ... * feat(db): zero-downtime migration safety lint + db-migrate skill Add scripts/check-migrations-safety.ts (check:migrations), a CI gate that classifies statements in newly-added migrations into hard errors (rewrite), annotate-to-acknowledge contract ops (`-- migration-safe: <reason>`), and backfill warnings. Wire it into test-build.yml. Add the /db-migrate skill as the judgment half (expand/contract phasing, app-code cross-ref, annotation authoring). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(skills): run cleanup and db-migrate safety checks in /ship * fix(db): address review — DROP INDEX lock symmetry, RENAME CONSTRAINT false-positive, alter-type literal match - Non-concurrent DROP INDEX is now a hard error (ACCESS EXCLUSIVE lock), symmetric with CREATE INDEX; DROP INDEX CONCURRENTLY after a COMMIT passes clean. Removes the false-confidence annotate path. - RENAME rule narrowed to RENAME COLUMN / table RENAME TO; RENAME CONSTRAINT and ALTER INDEX ... RENAME (metadata-only) no longer flagged. - alter-type regex now requires TYPE to follow the column identifier, so it no longer matches TYPE inside a string default. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(db): enforce IF EXISTS on DROP INDEX CONCURRENTLY for replay idempotency Symmetric with the CREATE INDEX CONCURRENTLY rule: a post-COMMIT DROP INDEX CONCURRENTLY replays from the top on failure, so without IF EXISTS it aborts re-dropping an already-gone index. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * improvement(skills): gate /ship cleanup on UI changes; default migration base to staging - /ship runs /cleanup only when the diff touches UI code (.tsx or apps/sim/components|hooks|stores); the six passes are React-only. - /ship runs check:migrations against origin/staging (the PR base). - check:migrations default baseRef is now origin/staging instead of origin/main. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(db-migrate): add contract-pending TODO convention for deferred drops Establishes a durable, greppable marker (`contract-pending(<precondition>): ...`) left on the legacy column in schema.ts when an expand defers a drop, so the contract phase doesn't rot. The outstanding-work list is `grep -rn contract-pending`; the contract PR's `-- migration-safe:` annotation references the expand and deletes the marker. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 20:00:50 -04:00
Siddharth Ganesan	d538b76eda	feat(copilot): server-side mothership tool/vfs/file metrics (#5071)	2026-06-15 16:32:35 -07:00
Siddharth Ganesan	cefb2dc239	improvement(mship): add enrichment tool, clean up dead tools (#5058) ... * feat(mothership): add enrichment_run server tool for one-off lookups Implement the Sim-side handler for the copilot enrichment_run tool: runs the enrichment provider cascade for a single entity and returns the result inline, surfacing the hosted-key cost as _serviceCost for per-round billing (matching the media tools). Registered in the server-tool router. Regenerate the copilot tool catalog/schemas to include enrichment_run. * fix(mothership): remove leftover touch_plan tool references touch_plan was removed from the copilot tool catalog earlier, but the Sim side still referenced it. Regenerating the generated tool catalog (for enrichment_run) synced it to the current contract and dropped the stale TouchPlan export, which broke the build where router.ts still imported it. Remove the dead touch_plan server tool and its test, drop its router registration and WRITE_ACTIONS entry, simplify getServerToolRegistry (no more beta-gated server tools), and clean up stale "use touch_plan" guidance strings. * chore(mothership): trigger dev redeploy * improvement(mothership): log billed cost on enrichment_run lookups * fix(contracts): regenerate mship contracts * fix(contracts): fix mship contracts	2026-06-15 10:57:29 -07:00
Emir Karabegandwaleed	bcedadf1b1	feat(scheduled-tasks): calendar views + persisted, runnable tasks (#4979) ... * improvement(resource): simplify table shell, toasts, and loading breadcrumbs - Resource.Table: remove internal sorting (defaultSort/sortValues) and the emptyMessage state — rows render in the order given, chrome always paints - Resource: root is now the positioning context for overlays; consumers (files, tables, knowledge, document) wrap detail views in <Resource> instead of hand-rolled divs - ResourceHeader: root titles no longer truncate during initial layout; LocationFocusVeil gates the portal on mount to fix a hydration mismatch - Toasts: drop the StackDismiss ring and stack countdown — each toast runs its own timer; remove the Mod+E clear-notifications command; align toast typography and icons with chip chrome - Breadcrumbs: use the canonical '…' placeholder while names load - incident.io: fix display name and catalog slug (with redirect) - Add dev:capped / dev:full:capped scripts with a 4GB heap cap * feat(scheduled-tasks): calendar views; rename Mothership to Sim/Chat Add month/time calendar views for scheduled tasks with toolbar, event chips, and a create-task modal, backed by calendar-grid and schedule-events utils (with tests) and a use-calendar hook. Replace the old schedule-modal/context-menu flow. Rename the "Mothership" agent to "Sim" and the chat surface to "Chat" across landing copy, constitution, block metadata, API error messages, and copilot/data-drain internals. Drop unused workspace route layouts. * fix(emcn): force dropdown menus modal inside dialogs so they scroll A non-modal DropdownMenu portals outside an open dialog's react-remove-scroll subtree, so its content cannot be wheel-scrolled (e.g. the time picker in the scheduled-task create modal). ModalContent now marks its subtree via an InsideModal context, and the emcn DropdownMenu root upgrades itself to modal inside dialogs so it mounts its own scroll lock and focus scope; page-level menus keep their consumer-chosen modality. Also stretch the create-task modal's date/time chip controls to full width and drop the dead EDGE_GUTTER constant left behind by the equal-tracks calendar layout. * fix(scheduled-tasks): address review — midnight rollover, stub feedback, smooth Today scroll - useCalendar: today was frozen at mount, so after midnight the isToday column highlight and the current-time indicator stayed on the previous day. today is now state refreshed by a sleep-resilient minute poll that only re-renders when the calendar day actually changes - CreateTaskModal: the stub submit closed silently, reading as false success; it now shows an info toast that the task was not created - ScheduleCalendar: Today presses scroll smoothly as an orientation cue; mount and scope switches keep instant positioning * feat(emcn): view-only field primitives + scheduled-task modals - ChipCopyInput (canonical view-only copy field), ChipTimePicker, ChipModalField type='copy', ChipTextarea viewOnly; new border chip variant and shared chipPrimaryFillTokens - migrate ~40 consumers off disabled inputs and the deleted CopyableValueField; ChipConfirmModal description->text and secondaryActions[] API sweep - scheduled-tasks: rename create-task-modal to task-modal, add task-details-modal + task-context-menu, useScheduledTasks hook - home: extract prompt-editor (usePromptEditor) out of user-input * feat(scheduled-tasks): persist + run tasks via the job-schedule backend Wire the calendar UI to the existing sourceType='job' workflow_schedule backend instead of local component state, so tasks persist and actually run as Sim agent invocations. - schema: add contexts (@-mentions resolved into the run), excludedDates (per-occurrence deletes), and endsAt (recurrence end) to workflow_schedule (migration 0235) - contracts/schedules: expose one-time `time`, contexts, endsAt on create; add the exclude_occurrence action; nullable cron in the create response - orchestration: persist the new fields, honor exclusions + end boundary via a shared computeNextRunAt, add performExcludeOccurrence - execution: forward contexts to /api/mothership/execute and recompute the next run through computeNextRunAt - mothership/execute: accept + resolve contexts like the interactive chat path - frontend: replace the local hook with React Query (create/update/delete + exclude-occurrence), expand recurrences into calendar occurrences, add the recurrence control (frequency + end) and the recurring this/all delete dialog * chore(scheduled-tasks): satisfy biome line-width on user-input imports * fix(scheduled-tasks): log agent-context resolution failures in execute route * fix(scheduled-tasks): keep context double-cast adjacent to its boundary annotation * fix(scheduled-tasks): preserve @-mention contexts on edit; sync editor valueRef on input * fix(scheduled-tasks): footer-wrap modal controls; audit fixes; cleanup - chip-modal: footer secondary cluster now wraps (min-w-0 flex-wrap) with a non-shrinking action cluster, so scheduling controls can never clip Cancel/ primary; recurrence labels compacted so the common case stays one row - schedule-execution: failure path now completes a recurring job when maxRuns/ endsAt/exclusions are exhausted (and a one-time/maxRuns job), mirroring the success path instead of leaving it active with a stale nextRunAt - prompt-editor: commitValue keeps valueRef in lockstep with state on the mention-hook setter paths, completing the stale-ref fix - task-modal: preserve @-mention contexts on edit (seed editor.setContexts); single emptiness source of truth - recurrence-control: preserve prior count when toggling end type; drop a needless useMemo - contracts: reuse scheduleContextSchema for the execute contexts shape * feat(scheduled-tasks): valid future default launch time; test scheduleToTasks mapping * chore(scheduled-tasks): convert added inline comments to TSDoc * simplify(scheduled-tasks): reuse date-fns for launch/end math; drop dead 'running' status + single-use helper --------- Co-authored-by: waleed <walif6@gmail.com>	2026-06-13 16:16:40 -07:00
Waleed	eb1009da1c	improvement(react-query): codebase-wide audit — server-state hooks, webhook coherence, resume migration (#5024) ... * improvement(react-query): codebase-wide audit — server-state hooks, webhook coherence, resume migration * chore(react-query): add static pattern linter + address review feedback - add scripts/check-react-query-patterns.ts (staleTime/signal/key-factory/inline-key enforcement) wired into CI as check:react-query; strict zone hooks/queries/**, ratchet elsewhere - fix(resume): use same-origin relative path for resume POST (getBaseUrl could cross origin on whitelabel/preview hosts and drop session cookies) — Cursor Bugbot - remove explanatory inline comments in favor of TSDoc per repo convention	2026-06-13 10:14:07 -07:00
WaleedandTheodore Li	540835a7a5	feat(integrations): add AWS AppConfig integration with tools, block, and docs (#4928) ... * feat(integrations): add AWS AppConfig integration with tools, block, and docs * fix(integrations): preserve latestVersionNumber 0 and tighten locationUri type in AppConfig * fix(integrations): use valid IntegrationTag values in AppConfig BlockMeta * feat(appconfig): add list_hosted_configuration_versions operation * fix(appconfig): stringify configurationVersion in block params * feat(appconfig): add full CRUD for applications, environments, and configuration profiles Adds get/update/delete for applications, environments, and configuration profiles, plus delete_hosted_configuration_version — 10 tools rounding out the integration to management-grade CRUD completeness. --------- Co-authored-by: Theodore Li <theo@sim.ai>	2026-06-09 13:17:40 -07:00
Waleed	5d1aa228f5	fix(files): attach wheel listener before paint and guard SVG src (#4744) ... * fix(files): attach wheel listener before paint and guard SVG src * chore(deps): upgrade turbo to 2.9.14	2026-05-26 14:14:49 -07:00
Waleed	8d7bbbc670	chore(utils): migrate to shared random/ID utilities and add enforcement linting (#4623) ... * chore(utils): migrate to shared random/ID utilities and add enforcement linting - Replace all Math.random(), crypto.randomUUID(), crypto.randomBytes(), nanoid, and uuid usages with shared @sim/utils/random and @sim/utils/id helpers across 72 files - Add new @sim/utils exports: deepClone, omit, filterUndefined (object), truncate (string), backoffWithJitter, parseRetryAfter (retry), getErrorMessage (errors) - Sweep all getErrorMessage, sleep, deepClone callsites across 500+ files to use shared utilities - Add Biome noRestrictedImports rule to catch nanoid, uuid, and crypto named imports at lint time - Add scripts/check-utils-enforcement.ts to catch Math.random and crypto.* global property access - Add check:utils script to package.json * chore(utils): replace deepClone wrapper with structuredClone built-in deepClone() was a one-line wrapper around structuredClone(), which is universally available in Node 17+ and all modern browsers. Removing the abstraction reduces indirection and means contributors don't need to learn a project-specific name for a well-known built-in. - Remove deepClone from packages/utils/src/object.ts and index.ts - Replace all 17 call sites with structuredClone() directly - Update check:utils script suggestion text - Update CLAUDE.md and global.md docs * fix(utils): add missing biome noRestrictedImports rule and correct truncate docs - Add noRestrictedImports to biome.json under style — bans nanoid and uuid package imports at lint time (crypto.randomUUID/randomBytes are caught by the check:utils grep script which handles global property access) - Correct truncate() TSDoc and parameter name: sliceLength makes it clear that total output length is sliceLength + suffix.length, matching the behavior all callers were already written to expect * fix(utils): add missing getErrorMessage imports at 4 call sites The sweep agents added getErrorMessage calls without the corresponding import in 4 files, causing test failures. Added the missing imports. * fix(utils): fix build errors from getErrorMessage sweep and retry.ts Turbopack issue - Fix retry.ts cross-file import: Turbopack cannot resolve './random.js' for internal package imports; inline the jitter crypto call directly - Add missing getErrorMessage imports to 32 files where the sweep added calls without the corresponding import (caught by type-check and test runs) - Remove accidental getErrorMessage import from crowdstrike/query/route.ts which has its own domain-specific getErrorMessage for parsing CrowdStrike's JSON error format - Fix use-sub-block-value.ts type error from structuredClone narrowing: add 'as T' cast at emitValue callsite (safe — valueCopy is always a structural copy of newValue) * fix(tools): use toError in crowdstrike catch block instead of local getErrorMessage The catch block was calling the local getErrorMessage function which parses CrowdStrike API JSON responses, not JavaScript Error objects. Use toError(error).message to correctly extract the message from a caught value in this context.	2026-05-15 17:31:27 -07:00
WaleedandCursor	a64b338236	fix(files): fixed resource spacing on files directories pages (#4618) ... * fix(files): fixed resource spacing on files directories pages * fix(build): align turbo schema version Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-15 12:48:15 -07:00
Waleed	9bbbe0adcb	chore(deps): bump mermaid to 11.15.0 for GHSA-ghcm-xqfw-q4vr (#4615) ... * chore(deps): bump mermaid to 11.15.0 for GHSA-ghcm-xqfw-q4vr * chore(deps): override transitive mermaid to 11.15.0	2026-05-15 10:42:42 -07:00
Waleed	11fa96cac0	chore(deps): bump next to 16.2.5 for CVE-2026-44578 SSRF fix (#4606) ... * chore(deps): bump next to 16.2.5 for CVE-2026-44578 SSRF fix * chore(deps): bump next to 16.2.6 for full May 2026 security release coverage	2026-05-14 14:34:05 -07:00
Vikhyath Mondreti	13666b162d	fix(zustand): v5 selector stability issues (#4539) ... * fix(zustand): v5 selector stability issues * address comments	2026-05-09 16:07:12 -07:00
WaleedandClaude Opus 4.7	28e60bfd4e	fix(docker): drop scripts/ from workspaces array (#4484) ... `turbo prune sim --docker` strips `scripts/` from the pruned output (sim doesn't depend on it), but the pruned root package.json still listed it as a workspace, causing `bun install` to fail with "Workspace not found 'scripts'" in the Docker build. scripts/ is dev-only tooling that runs from the repo root via `bun run scripts/*.ts`. Its imports (glob, yaml) resolve against the root node_modules — they're already in root devDependencies. - Remove "scripts" from root workspaces array - Delete scripts/package.json (no longer a workspace, manifest unused) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-06 20:25:07 -07:00
WaleedandClaude Opus 4.7	7953c56aca	fix(security): xlsx CVE bump and bundled security hardening (#4481) ... * fix(security): xlsx CVE bump and bundled security hardening * fix(stripe): use configured secret key for SDK init Avoids leaving a recognisable placeholder string in heap dumps and error serialisations. Webhook verification remains a purely local HMAC operation; the SDK's constructor key is unused by it. Addresses Greptile feedback on #4481. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(stripe): use static Stripe.webhooks for verification Avoids instantiating a Stripe client just to access constructEvent. The webhook signing secret is per-trigger (user-provided whsec_…) and unrelated to our billing STRIPE_SECRET_KEY, so coupling them was wrong. Stripe.webhooks is exposed as a static — no client, no API key needed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(ci): revert client-bundled tools to avoid .server import in client * fix(security): collapse 403 to 404 on v1 detail-by-ID routes * chore(security): remove unused validateAgiloftInstanceUrl helper * fix(security): bump minimatch + clean up scripts/ workspace Resolves CVE-2026-27903 (GHSA-7r86-cg39-jmmj) by adding a root-level minimatch ^10.2.5 override. Also resolves CVE-2026-0969 in next-mdx-remote (bumped to ^6.0.0). Cleanup: - Make scripts/ a proper bun workspace (root workspaces array) - Remove duplicate scripts/package-lock.json (this repo uses bun) - Remove redundant scripts/bun.lock (now hoisted to root) - Remove vestigial scripts/setup-doc-generator.sh - Slim scripts/package.json to its real deps (glob, yaml) - Gitignore stray package-lock.json files - Update scripts/README.md Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-06 16:36:51 -07:00
Waleed	48331451b0	chore(deps): upgrade next.js to 16.2.4 (#4460) ... * chore(deps): upgrade next.js to 16.2.4 - Bump next and @next/env to 16.2.4 across root, apps/sim, apps/docs - Replace next-runtime-env's env() helper (calls unstable_noStore(), rejected by Next 16.2 outside request scope) with a direct window.__ENV / process.env getter - Add export const dynamic = 'force-dynamic' on landing /privacy and /terms pages so NEXT_PUBLIC_* runtime env reads aren't baked at build * fix(whitelabel): force dynamic rendering for manifest.ts Without this, NEXT_PUBLIC_BRAND_* values are baked into the manifest at build time. Pairs with the next-runtime-env removal in the prior commit, restoring Docker runtime injection for whitelabel deployments. * fix(oauth): wrap consent page useSearchParams in Suspense Next 16.2's stricter prerender check fails the build when useSearchParams() is used without a Suspense boundary. Splits the client component into an outer wrapper and inner body. * fix(whitelabel): force dynamic rendering for landing segment Client components in (landing) (e.g. Navbar) read NEXT_PUBLIC_BRAND_* via getEnv. Without this, SSR prerender would bake the build-time process.env values into HTML, mismatching window.__ENV after hydration in Docker runtime-env deployments. Cascades to all landing routes via the layout. * revert(whitelabel): drop force-dynamic from landing layout Cascading force-dynamic neutered dynamicParams = false + generateStaticParams on /blog/[slug], /integrations/[slug], /models/[provider], /models/[provider]/[model] — killing static prerender for SEO-critical pages. The hydration concern only materializes for whitelabel Docker deployments where build-time and runtime NEXT_PUBLIC_BRAND_* differ; those deployments can set the vars at build instead. Keeping force-dynamic on /privacy, /terms, and /manifest where it actually matters. * fix(prerender): wrap useSearchParams callsites for Next 16.2 Next 16.2 fails the build when a client component using useSearchParams() is statically prerendered without a Suspense boundary. - Wrap landing Navbar in Suspense (imported by /oauth/consent and other pages) - Add force-dynamic to reset-password, invite/[id], and unsubscribe pages whose client bodies call useSearchParams * fix(navbar): preserve SSR HTML, drop Suspense bailout Reading useSearchParams() forced a Suspense fallback that emitted no navbar HTML during SSR — leaving crawlers and no-JS users without nav. The 'home' query param only affects client-side link targets, so read it from window.location in an effect after hydration. Restores full SSR navbar markup. * chore: trim verbose comments in next.js upgrade The force-dynamic export name is self-documenting; the remaining env.ts comment is tightened to the essential WHY (why we don't use next-runtime-env's helper).	2026-05-06 11:08:01 -07:00
Vikhyath Mondreti	b8959eb20d	improvement(repo): zod based client-server boundary (#4355) ... * improvement(repo): centralized zod contracts (#4336) * improvement(repo): zod schema contracts * type checks * fix(notion): correctly register tool (#4337) * fix func blokc * more improvements * fix tests * type check * remove v3 refs * minor type improvements * address comments * update jira contract * remove validateJsonBody * improvement(repo): consolidation of boundary helpers + better unknown usage (#4352) * improvement(repo): consolidation of boundary helpers + better unknown usage * address comments * improve file transfer error messaging * fix docs listing schema drift * fix inocrrect type casting * address council comments * remove prefix	2026-04-30 12:16:28 -07:00
Waleed	5fba724818	chore(bun): bump bun to 1.3.13 (#4291)	2026-04-24 15:59:10 -07:00
3 people	5f0f0edd63	improvement(repo): separate realtime into separate app (#4262) ... * improvement(repo): restructuring to make realtime image narrower scoped * improvements * chore(repo): rebase fixes and quality improvements for realtime split Addresses merge-time issues and gaps from the realtime app split: - Retarget stale vi.mock paths to @sim/workflow-persistence/subblocks - Restore README branding, fix AGENTS.md script reference - Restore TSDoc on workflow-persistence subblocks helpers - Use toError() from @sim/utils/errors in save.ts - Add vitest config + local mocks so @sim/audit tests run standalone - Move socket.io-client to devDependencies in apps/realtime - Add missing package COPY steps to docker/app.Dockerfile - Add check:boundaries/check:realtime-prune scripts and wire into CI Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(security): consolidate crypto primitives into @sim/security Move general-purpose crypto primitives out of apps/sim into the @sim/security package so both apps/sim and apps/realtime can share them. @sim/security exports (all pure, dependency-free): ./compare safeCompare (constant-time HMAC-wrapped equality) ./encryption encrypt/decrypt (AES-256-GCM, iv:cipher:tag format) ./hash sha256Hex ./tokens generateSecureToken (base64url) Migrate apps/sim call sites to use these + @sim/utils helpers: crypto.randomUUID() -> generateId() from @sim/utils/id createHash('sha256').digest -> sha256Hex timingSafeEqual on hashed hex -> safeCompare new Promise(setTimeout) -> sleep from @sim/utils/helpers No behavior change: encryption format, digest output, and token length are preserved exactly. * refactor(copilot): use toError in remaining otel/finalize sites Replace the last two `error instanceof Error ? error : new Error(String(error))` patterns with toError from @sim/utils/errors. Completes the sweep of clean candidates — no behavior change. * refactor(security): consolidate HMAC-SHA256 primitives into @sim/security Adds hmacSha256Hex and hmacSha256Base64 to @sim/security/hmac and migrates 15 webhook providers plus 5 other hot paths (deployment token signing, outbound webhook requests, workspace notification delivery, notification test route, Shopify OAuth callback) off bare `createHmac` calls. Secret parameter accepts `string | Buffer` to cover base64-decoded Svix-style secrets (Resend) and MS Teams' HMAC scheme. AWS SigV4 signing in S3 and Textract tools intentionally retains direct `createHmac` usage — its multi-step key derivation chain doesn't fit a generic helper. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(packages): post-audit test + packaging polish - Add safeCompare unit tests (identity, length mismatch, hex-nibble diff). - Add Buffer-secret cases to hmac tests to lock in Svix/MS-Teams contract. - Declare `reactflow` as a peerDependency on @sim/workflow-types — only used for type imports. - Add a barrel export to @sim/workflow-persistence for consumers that prefer package-level imports; subpath exports retained. - Document the data-field invariant in load.ts for loop/parallel subflow patching. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(realtime): address PR review feedback - Remove redundant SOCKET_PORT=3002 env from Dockerfile runner stage (env.PORT already defaults to 3002 via zod schema). - Reorder PORT fallback so an explicitly-set SOCKET_PORT wins over the schema default for PORT; keeps SOCKET_PORT functional as an override instead of dead code. - Add dedicated type-check CI step for @sim/realtime so TS errors surface pre-deploy (the Dockerfile runs source TS via Bun and has no implicit build-time type check). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(realtime): remove unused SOCKET_PORT env var SOCKET_PORT has lived in the socket server since the June 2025 refactor but was never actually set in any deploy config — docker-compose.prod, helm values/templates, .env.example, and docs all use PORT or the 3002 default exclusively. No self-hoster was ever pointed at SOCKET_PORT, so removing it is safe. Simplifies realtime port resolution to `env.PORT` (zod-validated with a 3002 default) and drops the orphaned sim-side schema entry. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Waleed Latif <walif6@gmail.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-22 23:06:16 -07:00
Siddharth Ganesan	0aeab026a8	feat(observability): add mothership tracing (#4253)	2026-04-22 09:06:01 -07:00
WaleedandClaude Opus 4.7	45bf396968	fix(deps): bump drizzle-orm 0.45.2 + adopt MCP SDK 1.25.3 native types (#4252) ... * fix(deps): bump drizzle-orm to 0.45.2 (GHSA-gpj5-g38j-94v9) Resolves Dependabot alert #98. Drizzle ORM <0.45.2 improperly escaped quoted SQL identifiers, allowing SQL injection via untrusted input passed to APIs like sql.identifier() or .as(). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(mcp): adopt native SDK types after @modelcontextprotocol/sdk 1.25.3 bump Replace hand-written schema/annotation shapes with the SDK's exported Tool, JSONRPCResultResponse, and Tool['annotations'] types so changes upstream flow through automatically. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(types): use drizzle $inferSelect for row types Replace hand-written interfaces that duplicated schema shape with typeof table.$inferSelect aliases for webhook, workflow, and workspaceFiles rows. Also simplify metadata insert/update to use .returning() instead of field-by-field copies. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(uploads): fall through to INSERT if restore-deleted row races a hard delete If a hard delete races between the initial SELECT and the restore UPDATE, .returning() yields no row. Previously the function would return undefined and silently violate the Promise<FileMetadataRecord> contract. Now the function falls through to the INSERT path, which already handles uniqueness races via the 23505 catch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(uploads): align metadata.ts with global standards Replace dynamic uuid import with generateId() per @sim/utils/id convention, narrow the error catch off `any`, and convert the inline comment to TSDoc. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-21 19:52:15 -07:00
Waleed	9c1b0bc15f	improvement(ui): remove anti-patterns, fix follow-up auto-scroll, move CopyCodeButton to emcn (#4148) ... * improvement(ui): restore smooth streaming animation, fix follow-up auto-scroll, move CopyCodeButton to emcn * fix(ui): restore delayed animation, handle tilde fences, fix follow-up scroll root cause * fix(ui): extract useStreamingReveal to followup, keep cleanup changes * fix(ui): restore hydratedStreamingRef for reconnect path order-of-ops * fix(ui): restore full hydratedStreamingRef effect for reconnect path * fix(ui): use hover-hover prefix on CopyCodeButton callers to correctly override ghost variant * fix(logs): remove destructive color from cancel execution menu item * feat(logs): optimistic cancelling status on cancel execution * feat(logs): allow cancellation of pending (paused) executions * fix(hitl): cancel paused executions directly in DB Paused HITL executions are idle in the DB — they don't poll Redis or run in-process, so the existing cancel signals had no effect. The DB status stayed 'pending', causing the optimistic 'cancelling' update to revert on refetch. - Add PauseResumeManager.cancelPausedExecution: atomically sets paused_executions.status and workflow_execution_logs.status to 'cancelled' inside a FOR UPDATE transaction - Guard enqueueOrStartResume against resuming a cancelled execution - Include pausedCancelled in the cancel route success check * upgrade turbo * test(hitl): update cancel route tests for paused execution cancellation - Mock PauseResumeManager.cancelPausedExecution to prevent DB calls - Add pausedCancelled to all expected response objects - Add test for HITL paused execution cancellation path - Add missing auth/authz tests - Switch to vi.hoisted pattern for all mocks * fix(hitl): set endedAt when cancelling paused execution Without endedAt, the logs API running filter (isNull(endedAt)) would keep cancelled paused executions in the running view indefinitely. * fix(hitl): emit execution:cancelled event to canvas when cancelling paused execution Paused HITL executions have no active SSE stream, so the canvas never received the cancellation event. Now writes execution:cancelled to the event buffer and updates the stream meta so the canvas reconnect path picks it up and shows 'Execution Cancelled'. * fix(hitl): isolate cancelPausedExecution failure from successful cancellation Wrap cancelPausedExecution in try/catch so a DB error does not mask a prior successful Redis or in-process cancellation. Also move the resource-collapse side effect in home.tsx to a useEffect to avoid the stale closure on the resources array. * fix(hitl): add .catch() to fire-and-forget event buffer calls in cancel route	2026-04-13 22:04:13 -07:00
6 people	0abcc6e813	improvement(mothership): restructured stream, tool structures, code typing, file write/patch/append tools, timing issues (#4090) ... * fix build error * improvement(mothership): new agent loop (#3920) * feat(transport): replace shared chat transport with mothership-stream module * improvement(contracts): regenerate contracts from go * feat(tools): add tool catalog codegen from go tool contracts * feat(tools): add tool-executor dispatch framework for sim side tool routing * feat(orchestrator): rewrite tool dispatch with catalog-driven executor and simplified resume loop * feat(orchestrator): checkpoint resume flow * refactor(copilot): consolidate orchestrator into request/ layer * refactor(mothership): reorganize lib/copilot into structured subdirectories * refactor(mothership): canonical transcript layer, dead code cleanup, type consolidation * refactor(mothership): rebase onto latest staging * refactor(mothership): rename request continue to lifecycle * feat(trace): add initial version of request traces * improvement(stream): batch stream from redis * fix(resume): fix the resume checkpoint * fix(resume): fix resume client tool * fix(subagents): subagent resume should join on existing subagent text block * improvement(reconnect): harden reconnect logic * fix(superagent): fix superagent integration tools * improvement(stream): improve stream perf * Rebase with origin dev * fix(tests): fix failing test * fix(build): fix type errors * fix(build): fix build errors * fix(build): fix type errors * feat(mothership): add cli execution * fix(mothership): fix function execute tests * Force redeploy * feat(motheship): add docx support * feat(mothership): append * Add deps * improvement(mothership): docs * File types * Add client retry logic * Fix stream reconnect * Eager tool streaming * Fix client side tools * Security * Fix shell var injection * Remove auto injected tasks * Fix 10mb tool response limit * Fix trailing leak * Remove dead tools * file/folder tools * Folder tools * Hide function code inline * Dont show internal tool result reads * Fix spacing * Auth vfs * Empty folders should show in vfs * Fix run workflow * change to node runtime * revert back to bun runtime * Fix * Appends * Remove debug logs * Patch * Fix patch tool * Temp * Checkpoint * File writes * Fix * Remove tool truncation limits * Bad hook * replace react markdown with streamdown * Checkpoitn * fix code block * fix stream persistence * temp * Fix file tools * tool joining * cleanup subagent + streaming issues * streamed text change * Tool display intetns * Fix dev * Fix tests * Fix dev * Speed up dev ci * Add req id * Fix persistence * Tool call names * fix payload accesses * Fix name * fix snapshot crash bug * fix * Fix * remove worker code * Clickable resources * Options ordering * Folder vfs * Restore and mass delete tools * Fix * lint * Update request tracing and skills and handlers * Fix editable * fix type error * Html code * fix(chat): make inline code inherit parent font size in markdown headers Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improved autolayout * durable stream for files * one more fix * POSSIBLE BREAKAGE: SCROLLING * Fixes * Fixes * Lint fix * fix(resource): fix resource view disappearing on ats (#4103) Co-authored-by: Theodore Li <theo@sim.ai> * Fixes * feat(mothership): add execution logs as a resource type Adds `log` as a first-class mothership resource type so copilot can open and display workflow execution logs as tabs alongside workflows, tables, files, and knowledge bases. - Add `log` to MothershipResourceType, all Zod enums, and VALID_RESOURCE_TYPES - Register log in RESOURCE_REGISTRY (Library icon) and RESOURCE_INVALIDATORS - Add EmbeddedLog and EmbeddedLogActions components in resource-content - Export WorkflowOutputSection from log-details for reuse in EmbeddedLog - Add log resolution branch in open_resource handler via new getLogById service - Include log id in get_workflow_logs response and extract resources from output - Exclude log from manual add-resource dropdown (enters via copilot tools only) - Regenerate copilot contracts after adding log to open_resource Go enum * Fix perf and message queueing * Fix abort * fix(ui): dont delete resource on clearing from context, set resource closed on new task (#4113) Co-authored-by: Theodore Li <theo@sim.ai> * improvement(mothership): structure sim side typing * address comments * reactive text editor tweaks * Fix file read and tool call name persistence bug * Fix code stream + create file opening resource * fix use chat race + headless trace issues * Fix type issue * Fix mothership block req lifecycle * Fix build * Move copy reqid * Fix * fix(ui): fix resource tag transition from home to task (#4132) Co-authored-by: Theodore Li <theo@sim.ai> * Fix persistence --------- Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> Co-authored-by: Waleed Latif <walif6@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Theodore Li <theo@sim.ai> Co-authored-by: Theodore Li <theodoreqili@gmail.com>	2026-04-13 16:46:35 -07:00
Waleed	85f1d96859	feat(ee): enterprise feature flags, permission group platform controls, audit logs ui, delete account (#4115) ... * feat(ee): enterprise feature flags, permission group platform controls, audit logs ui, delete account * fix(settings): improve sidebar skeleton fidelity and fix credit purchase org cache invalidation - Bump skeleton icon and text from 16/14px to 24px to better match real nav item visual weight - Add orgId support to usePurchaseCredits so org billing/subscription caches are invalidated on credit purchase, matching the pattern used by useUpgradeSubscription - Polish ColorInput in whitelabeling settings with auto-prefix and select-on-focus UX * revert(settings): remove delete account feature * fix(settings): address pr review — atomic autoAddNewMembers, extract query hook, fix types and signal forwarding * chore(helm): add CREDENTIAL_SETS_ENABLED to values.yaml * fix(access-control): dynamic platform category columns, atomic permission group delete * fix(access-control): restore triggers section in blocks tab * fix(access-control): merge triggers into tools section in blocks tab * upgrade tubro * fix(access-control): fix Select All state when config has stale blacklisted provider IDs * fix(access-control): derive platform Select All from features list; revert turbo schema version * fix(access-control): fix blocks Select All check, filter empty platform columns * revert(settings): restore original skeleton icon and text sizes	2026-04-11 20:41:37 -07:00
Waleed	2504bfbaf8	feat(athena): add AWS Athena integration (#4034) ... * feat(athena): add AWS Athena integration * fix(athena): address PR review comments - Fix variable shadowing: rename inner `data` to `rowData` in row mapper - Fix first-page maxResults off-by-one: request maxResults+1 to compensate for header row - Add missing runtime guard for queryString in create_named_query - Move athena registry entries to correct alphabetical position * fix(athena): alphabetize registry keys and add type re-exports - Reorder athena_* registry keys to strict alphabetical order - Add type re-exports from index.ts barrel * fix(athena): cap maxResults at 999 to prevent overflow with header row adjustment The +1 adjustment for the header row on first-page requests could produce MaxResults=1001 when user requests 1000, exceeding the AWS API hard cap of 1000.	2026-04-07 20:20:53 -07:00
7 people	42fb434354	fix(encryption): specify authTagLength on all AES-GCM cipher/decipher calls (#3883) ... * fix: specify authTagLength in AES-GCM decipheriv calls Fixes missing authTagLength parameter in createDecipheriv calls using AES-256-GCM mode. Without explicit tag length specification, the application may be tricked into accepting shorter authentication tags, potentially allowing ciphertext spoofing. CWE-310: Cryptographic Issues (gcm-no-tag-length) * fix: specify authTagLength on createCipheriv calls for AES-GCM consistency Complements #3881 by adding explicit authTagLength: 16 to the encrypt side as well, ensuring both cipher and decipher specify the tag length. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: clean up crypto modules - Fix error: any → error: unknown with proper type guard in encryption.ts - Eliminate duplicate iv.toString('hex') calls in both encrypt functions - Remove redundant string split in decryptApiKey (was splitting twice) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * new turborepo version --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Lakee Sivaraya <71339072+lakeesiv@users.noreply.github.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com> Co-authored-by: NLmejiro <kuroda.k1021@gmail.com>	2026-04-01 10:46:58 -07:00
Waleed	7d4dd26760	fix(knowledge): fix document processing stuck in processing state (#3857) ... * fix(knowledge): fix document processing stuck in processing state * fix(knowledge): use Promise.allSettled for document dispatch and fix Copilot OAuth context - Change Promise.all to Promise.allSettled in processDocumentsWithQueue so one failed dispatch doesn't abort the entire batch - Add writeOAuthReturnContext before showing LazyOAuthRequiredModal from Copilot tools so useOAuthReturnForWorkflow can handle the return - Add consumeOAuthReturnContext on modal close to clean up stale context * fix(knowledge): fix type error in useCredentialRefreshTriggers call Pass empty string instead of undefined for connectorProviderId fallback to match the hook's string parameter type. * upgrade turbo * fix(knowledge): fix type error in connectors-section useCredentialRefreshTriggers call Same string narrowing fix as add-connector-modal — pass empty string fallback for providerId.	2026-03-30 20:35:08 -07:00
Waleed	8a481b612d	chore(config): clean up bun, turbo, and next.js config (#3788) ... * chore(config): clean up bun, turbo, and next.js config * chore(ci): bump bun to 1.3.11 in dockerfiles and workflows	2026-03-26 13:04:23 -07:00
Waleed	951c8fd5e9	feat(integrations): add integrationType and tags classification to all blocks (#3702) ... * feat(integrations): add integrationType and tags classification to all blocks * improvement(integrations): replace generic api/oauth tags with use-case-oriented tags * lint * upgrade turbo	2026-03-21 11:45:49 -07:00
Waleed	8837f14194	feat(home): expand template examples with 83 categorized templates (#3592) ... * feat(home): expand template examples with 83 categorized templates - Extract template data into consts.ts with rich categorization (category, modules, tags) - Expand from 6 to 83 templates across 7 categories: sales, support, engineering, marketing, productivity, operations - Add show more/collapse UI with category groupings for non-featured templates - Add connector-powered knowledge search templates (Gmail, Slack, Notion, Jira, Linear, Salesforce, etc.) - Add platform-native templates (document summarizer, bulk data classifier, knowledge assistant, etc.) - Optimize prompts for mothership execution with explicit resource creation and integration names - Add tags field for future cross-cutting filtering by persona, pattern, and domain - React 19.2.1 → 19.2.4 upgrade * fix(home): remove WhatsApp customer notifications template * fix(home): add aria-expanded to toggle button, skip popular in expanded view * fix(home): fix category display order, add aria-label to template cards	2026-03-14 17:53:12 -07:00
9 people	5b9f0d73c2	feat(mothership): mothership (#3411) ... * Fix lint * improvement(sidebar): loading * fix(sidebar): use client-generated UUIDs for stable optimistic updates (#3439) * fix(sidebar): use client-generated UUIDs for stable optimistic updates * fix(folders): use zod schema validation for folder create API Replace inline UUID regex with zod schema validation for consistency with other API routes. Update test expectations accordingly. * fix(sidebar): add client UUID to single workflow duplicate hook The useDuplicateWorkflow hook was missing newId: crypto.randomUUID(), causing the same temp-ID-swap issue for single workflow duplication from the context menu. * fix(folders): avoid unnecessary Set re-creation in replaceOptimisticEntry Only create new expandedFolders/selectedFolders Sets when tempId differs from data.id. In the common happy path (client-generated UUIDs), this avoids unnecessary Zustand state reference changes and re-renders. * Mothership block logs * Fix mothership block logs * improvement(knowledge): make connector-synced document chunks readonly (#3440) * improvement(knowledge): make connector-synced document chunks readonly * fix(knowledge): enforce connector chunk readonly on server side * fix(knowledge): disable toggle and delete actions for connector-synced chunks * Job exeuction logs * Job logs * fix(connectors): remove unverifiable requiredScopes for Linear connector * fix(connectors): remove legacy requiredScopes from Jira and Confluence connectors Jira and Confluence OAuth tokens don't return legacy scope names like read:jira-work or read:confluence-content.all, causing the 'Update access' banner to always appear. Set requiredScopes to empty array like Linear. * feat(tasks): add rename to task context menu (#3442) * Revert "fix(connectors): remove legacy requiredScopes from Jira and Confluence connectors" This reverts commit a0be3ff414. * fix(connectors): restore Linear connector requiredScopes Linear OAuth does return scopes in the token response. The previous fix of emptying requiredScopes was based on an incorrect assumption. Restoring requiredScopes: ['read'] as it should work correctly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(knowledge): pass workspaceId to useOAuthCredentials in connector card The ConnectorCard was calling useOAuthCredentials(providerId) without a workspaceId, causing the credentials API to return an empty array. This meant the credential lookup always failed, getMissingRequiredScopes received undefined, and the "Update access" banner always appeared. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix oauth link callback from mothership task * feat(connectors): add Fireflies connector and API key auth support (#3448) * feat(connectors): add Fireflies connector and API key auth support Extend the connector system to support both OAuth and API key authentication via a discriminated union (`ConnectorAuthConfig`). Add Fireflies as the first API key connector, syncing meeting transcripts via the Fireflies GraphQL API. Schema changes: - Make `credentialId` nullable (null for API key connectors) - Add `encryptedApiKey` column (AES-256-GCM encrypted, null for OAuth) This eliminates the `'_apikey_'` sentinel and inline `sourceConfig._encryptedApiKey` patterns, giving each auth mode its own clean column. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(fireflies): allow 0 for maxTranscripts (means unlimited) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Add context * fix(fireflies): correct types from live API validation (#3450) * fix(fireflies): correct types from live API validation - speakers.id is number, not string (API returns 0, 1, 2...) - summary.action_items is a single string, not string[] - Update formatTranscriptContent to handle action_items as string Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(fireflies): correct tool types from live API validation - FirefliesSpeaker.id: string -> number - FirefliesSentence.speaker_id: string -> number - FirefliesSpeakerAnalytics.speaker_id: string -> number - FirefliesSummary.action_items: string[] -> string - FirefliesSummary.outline: string[] -> string - FirefliesSummary.shorthand_bullet: string[] -> string - FirefliesSummary.bullet_gist: string[] -> string - FirefliesSummary.topics_discussed: string[] -> string Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat(knowledge): add connector tools and expand document metadata (#3452) * feat(knowledge): add connector tools and expand document metadata * fix(knowledge): address PR review feedback on new tools * fix(knowledge): remove unused params from get_document transform * refactor, improvement * fix: correct knowledge block canonical pair pattern and subblock migration - Rename manualDocumentId to documentId (advanced subblock ID should match canonicalParamId, consistent with airtable/gmail patterns) - Fix documentSelector.dependsOn to reference knowledgeBaseSelector (basic depends on basic, not advanced) - Remove unnecessary documentId migration (ID unchanged from main) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * lint * fix: resolve post-merge test and lint failures - airtable: sync tableSelector condition with tableId (add getSchema) - backfillCanonicalModes test: add documentId mode to prevent false backfill - schedule PUT test: use invalid action string now that disable is valid - schedule execute tests: add ne mock, sourceType field, use mockReturnValueOnce for two db.update calls - knowledge tools: fix biome formatting (single-line arrow functions) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fixes * Fixes * Clean vfs * Fix * Fix lint * fix(connectors): add rate limiting, concurrency controls, and bug fixes (#3457) * fix(connectors): add rate limiting, concurrency controls, and bug fixes across knowledge connectors - Add Retry-After header support to fetchWithRetry for all 18 connectors - Batch concurrent API calls (concurrency 5) in Dropbox, Google Docs, Google Drive, OneDrive, SharePoint - Batch concurrent API calls (concurrency 3) in Notion to match 3 req/s limit - Cache GitHub tree in syncContext to avoid re-fetching on every pagination page - Batch GitHub blob fetches with concurrency 5 - Fix GitHub base64 decoding: atob() → Buffer.from() for UTF-8 safety - Fix HubSpot OAuth scope: 'tickets' → 'crm.objects.tickets.read' (v3 API) - Fix HubSpot syncContext key: totalFetched → totalDocsFetched for consistency - Add jitter to nextSyncAt (10% of interval, capped at 5min) to prevent thundering herd - Fix Date consistency in connector DELETE route * fix(connectors): address PR review feedback on retry and SharePoint batching - Remove 120s cap on Retry-After — pass all values through to retry loop - Add maxDelayMs guard: if Retry-After exceeds maxDelayMs, throw immediately instead of hammering with shorter intervals (addresses validate timeout concern) - Add early exit in SharePoint batch loop when maxFiles limit is reached to avoid unnecessary API calls * fix(connectors): cap Retry-After at maxDelayMs instead of aborting Match Google Cloud SDK behavior: when Retry-After exceeds maxDelayMs, cap the wait to maxDelayMs and log a warning, rather than throwing immediately. This ensures retries are bounded in duration while still respecting server guidance within the configured limit. * fix(connectors): add early-exit guard to Dropbox, Google Docs, OneDrive batch loops Match the SharePoint fix — skip remaining batches once maxFiles limit is reached to avoid unnecessary API calls. * improvement(turbo): align turborepo config with best practices (#3458) * improvement(turbo): align turborepo config with best practices * fix(turbo): address PR review feedback * fix(turbo): add lint:check task for read-only lint+format CI checks lint:check previously delegated to format:check which only checked formatting. Now it runs biome check (no --write) which enforces both lint rules and formatting without mutating files. * upgrade turbo * improvement(perf): apply react and js performance optimizations across codebase (#3459) * improvement(perf): apply react and js performance optimizations across codebase - Parallelize independent DB queries with Promise.all in API routes - Defer PostHog and OneDollarStats via dynamic import() to reduce bundle size - Use functional setState in countdown timers to prevent stale closures - Replace O(n*m) .filter().find() with Set-based O(n) lookups in undo-redo - Use .toSorted() instead of .sort() for immutable state operations - Use lazy initializers for useState(new Set()) across 20 components - Remove useMemo wrapping trivially cheap expressions (typeof, ternary, template strings) - Add passive: true to scroll event listener * fix(perf): address PR review feedback - Extract IIFE Set patterns to named consts for readability in use-undo-redo - Hoist Set construction above loops in BATCH_UPDATE_PARENT cases - Add .catch() error handler to PostHog dynamic import - Convert session-provider posthog import to dynamic import() to complete bundle split * fix(analytics): add .catch() to onedollarstats dynamic import * improvement(resource): tables, files * improvement(resources): all outer page structure complete * refactor(queries): comprehensive TanStack Query best practices audit (#3460) * refactor: comprehensive TanStack Query best practices audit and migration - Add AbortSignal forwarding to all 41 queryFn implementations for proper request cancellation - Migrate manual fetch patterns to useMutation hooks (useResetPassword, useRedeemReferralCode, usePurchaseCredits, useImportWorkflow, useOpenBillingPortal, useAllowedMcpDomains) - Migrate standalone hooks to TanStack Query (use-next-available-slot, use-mcp-server-test, use-webhook-management, use-referral-attribution) - Fix query key factories: add missing `all` keys, replace inline keys with factory methods - Fix optimistic mutations: use onSettled instead of onSuccess for cache reconciliation - Replace overly broad cache invalidations with targeted key invalidation - Remove keepPreviousData from static-key queries where it provides no benefit - Add staleTime to queries missing explicit cache duration - Fix `any` type in UpdateSettingParams with proper GeneralSettings typing - Remove dead code: loadingWebhooks/checkedWebhooks from subblock store, unused helper functions - Update settings components (general, debug, referral-code, credit-balance, subscription, mcp) to use mutation state instead of manual useState for loading/error/success Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove unstable mutation object from useCallback deps openBillingPortal mutation object is not referentially stable, but .mutate() is stable in TanStack Query v5. Remove from deps to prevent unnecessary handleBadgeClick recreations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add missing byWorkflows invalidation to useUpdateTemplate The onSettled handler was missing the byWorkflows() invalidation that was dropped during the onSuccess→onSettled migration. Without this, the deploy modal (useTemplateByWorkflow) would show stale data after a template update. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add TanStack Query best practices to CLAUDE.md and cursor rules Add comprehensive React Query best practices covering: - Hierarchical query key factories with intermediate plural keys - AbortSignal forwarding in all queryFn implementations - Targeted cache invalidation over broad .all invalidation - onSettled for optimistic mutation cache reconciliation - keepPreviousData only on variable-key queries - No manual fetch in components rule - Stable mutation references in useCallback deps Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback - Fix syncedRef regression in use-webhook-management: only set syncedRef.current=true when webhook is found, so re-sync works after webhook creation (e.g., post-deploy) - Remove redundant detail(id) invalidation from useUpdateTemplate onSettled since onSuccess already populates cache via setQueryData Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address second round of PR review feedback - Reset syncedRef when blockId changes in use-webhook-management so component reuse with a different block syncs the new webhook - Add response.ok check in postAttribution so non-2xx responses throw and trigger TanStack Query retry logic Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use lists() prefix invalidation in useCreateWorkspaceCredential Use workspaceCredentialKeys.lists() instead of .list(workspaceId) so filtered list queries are also invalidated on credential creation, matching the pattern used by update and delete mutations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address third round of PR review feedback - Add nullish coalescing fallback for bonusAmount in referral-code to prevent rendering "undefined" when server omits the field - Reset syncedRef when queryEnabled becomes false so webhook data re-syncs when the query is re-enabled without component remount Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address fourth round of PR review feedback - Add AbortSignal to testMcpServerConnection for consistency - Wrap handleTestConnection in try/catch for mutateAsync error handling - Replace broad subscriptionKeys.all with targeted users()/usage() invalidation - Add intermediate users() key to subscription key factory for prefix matching - Add comment documenting syncedRef null-webhook behavior - Fix api-keys.ts silent error swallowing on non-ok responses - Move deployments.ts cache invalidation from onSuccess to onSettled Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: achieve full TanStack Query best practices compliance - Add intermediate plural keys to api-keys, deployments, and schedules key factories for prefix-based invalidation support - Change copilot-keys from refetchQueries to invalidateQueries - Add signal parameter to organization.ts fetch functions (better-auth client does not support AbortSignal, documented accordingly) - Move useCreateMcpServer invalidation from onSuccess to onSettled Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * ran lint * Fix tables row count * Update mothership to match copilot in logs * improvement(resource): layout * fix(knowledge): compute KB tokenCount from documents instead of stale column (#3463) The knowledge_base.token_count column was initialized to 0 and never updated. Replace with COALESCE(SUM(document.token_count), 0) in all read queries, which already JOIN on documents with GROUP BY. * improvement(resources): layout and items * feat(knowledge): add v1 knowledge base API, Obsidian/Evernote connectors, and docs (#3465) * feat(knowledge): add v1 knowledge base API, Obsidian/Evernote connectors, and docs - Add v1 REST API for knowledge bases (CRUD, document management, vector search) - Add Obsidian and Evernote knowledge base connectors - Add file type validation to v1 file and document upload endpoints - Update OpenAPI spec with knowledge base endpoints and schemas - Add connectors documentation page - Apply query hook formatting improvements * fix(knowledge): address PR review feedback - Remove validateFileType from v1/files route (general file upload, not document-only) - Reject tag filters when searching multiple KBs (tag defs are KB-specific) - Cache tag definitions to avoid duplicate getDocumentTagDefinitions call - Fix Obsidian connector silent empty results when syncContext is undefined * improvement(connectors): add syncContext to getDocument, clean up caching - Update docs to say 20+ connectors - Add syncContext param to ConnectorConfig.getDocument interface - Use syncContext in Evernote getDocument to cache tag/notebook maps - Replace index-based cache check with Map keyed by KB ID in search route * fix(knowledge): address second round of PR review feedback - Fix Zod .default('text') overriding tag definition's actual fieldType - Fix encodeURIComponent breaking multi-level folder paths in Obsidian - Use 413 instead of 400 for file-too-large in document upload - Add knowledge-bases to API reference docs navigation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(knowledge): prevent cross-workspace KB access in search Filter accessible KBs by matching workspaceId from the request, preventing users from querying KBs in other workspaces they have access to but didn't specify. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(knowledge): audit resourceId, SSRF protection, recursion depth limit - Fix recordAudit using knowledgeBaseId instead of newDocument.id - Add SSRF validation to Obsidian connector (reject private/loopback URLs) - Add max recursion depth (20) to listVaultFiles Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(obsidian): remove SSRF check that blocks localhost usage The Obsidian connector is designed to connect to the Local REST API plugin running on localhost (127.0.0.1:27124). The SSRF check was incorrectly blocking this primary use case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * improvement(resources): segmented API * fix(execution): ensure background tasks await post-execution DB status updates (#3466) The fire-and-forget IIFE in execution-core.ts for post-execution logging could be abandoned when trigger.dev tasks exit, leaving executions permanently stuck in "running" status. Store the promise on LoggingSession so background tasks can optionally await it before returning. * improvement(resource): sorting and icons * fix(resource): sorting * improvement(settings): fix mcp modal, add option to edit JSON and add Sim as an MCP client (#3467) * improvement(settings): fix mcp modal, add option to edit JSON and add Sim as an MCP client * added docs link in sidebar * ack comments * ack comments * fixed error msg * feat(mothership): billing (#3464) * Billing update * more billing improvements * credits UI * credit purchase safety * progress * ui improvements * fix cancel sub * fix types * fix daily refresh for teams * make max features differentiated * address bugbot comments * address greptile comments * revert isHosted * address more comments * fix org refresh bar * fix ui rounding * fix minor rounding * fix upgrade issue for legacy plans * fix formatPlanName * fix email dispay names * fix legacy team reference bugs * referral bonus in credits * fix org upgrade bug * improve logs * respect toggle for paid users * fix landing page pro features and usage limit checks * fixed query and usage * add unit test * address more comments * enterprise guard * fix limits bug * pass period start/end for overage * fix(sidebar): restore drag-and-drop for workflows and folders (#3470) * fix(sidebar): restore drag-and-drop for workflows and folders Made-with: Cursor * update docs, unrelated * improvement(tables): consolidation * feat(schedules): add schedule creator modal for standalone jobs Add modal to create standalone scheduled jobs from the Schedules page. Includes POST API endpoint, useCreateSchedule mutation hook, and full modal with schedule type selection, timezone, lifecycle, and live preview. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(schedules): add edit support with context menu for standalone jobs * style(schedules): apply linter formatting * improvement: tables, favicon * feat(files): inline file viewer with text editing (#3475) * feat(files): add inline file viewer with text editing and create file modal Add file preview/edit functionality to the workspace files page. Text files (md, json, txt, yaml, etc.) open in an editable textarea with Cmd/Ctrl+S save. PDFs render in an iframe. New file button creates empty .md files via a modal. Uses ResourceHeader breadcrumbs and ResourceOptionsBar for save/download/delete. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improvement(files): add UX polish, PR review fixes, and context menu - Add unsaved changes guard modal (matching credentials manager pattern) - Add delete confirmation modal for both viewer and context menu - Add save status feedback (Save → Saving... → Saved) - Add right-click context menu with Open, Download, Delete actions - Add 50MB file size limit on content update API - Add storage quota check before content updates - Add response.ok guard on download to prevent corrupt files - Add skeleton loading for pending file selection (prevents flicker) - Fix updateContent in handleSave dependency array Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): propagate save errors and remove redundant sizeDiff - Remove try/catch in TextEditor.handleSave so errors propagate to parent, which correctly shows save failure status - Remove redundant inner sizeDiff declaration that shadowed outer scope Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): remove unused textareaRef Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): move Cmd+S to parent, add save error feedback, hide save for non-text files - Move Cmd+S keyboard handler from TextEditor to Files so it goes through the parent handleSave with proper status management - Add 'error' save status with red "Save failed" label that auto-resets - Only show Save button for text-editable file types (md, txt, json, etc.) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improvement(files): add save tooltip, deduplicate text-editable extensions - Add Tooltip on Save button showing Cmd+S / Ctrl+S shortcut - Export TEXT_EDITABLE_EXTENSIONS from file-viewer and reuse in files.tsx instead of duplicating the list inline Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: extract isMacPlatform to shared utility Move isMacPlatform() from global-commands-provider.tsx to lib/core/utils/platform.ts so it can be reused by files.tsx tooltip without duplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(files): deduplicate delete modal, use shared formatFileSize - Extract DeleteConfirmModal component to eliminate duplicate modal markup between viewer and list modes - Replace local formatFileSize with shared utility from file-utils.ts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): fix a11y label lint error and remove mutation object from useCallback deps Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): add isDirty guard on handleSave, return proper HTTP status codes Prevents "Saving → Saved" flash when pressing Cmd+S with no changes. Returns 404 for file-not-found and 402 for quota-exceeded instead of 500. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): reset isDirty/saveStatus on delete and discard, remove deprecated navigator.platform - Clear isDirty and saveStatus when deleting the currently-viewed file to prevent spurious beforeunload prompts - Reset saveStatus on discard to prevent stale "Save failed" when opening another file - Remove deprecated navigator.platform, userAgent fallback covers all cases Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): prevent concurrent saves on rapid Cmd+S, add YAML MIME types - Add saveStatus === 'saving' guard to handleSave to prevent duplicate concurrent PUT requests from rapid keyboard shortcuts - Add yaml/yml MIME type mappings to getMimeTypeFromExtension Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(files): reuse shared extension constants, parallelize cancelQueries - Replace hand-rolled SUPPORTED_EXTENSIONS with composition from existing SUPPORTED_DOCUMENT/AUDIO/VIDEO_EXTENSIONS in validation.ts - Parallelize sequential cancelQueries calls in delete mutation onMutate Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): guard handleCreate against duplicate calls while pending Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): show upload progress on the Upload button, not New file Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): use ref-based guard for create pending state to avoid stale closure The uploadFile.isPending check was stale because the mutation object is excluded from useCallback deps (per codebase convention). Using a ref ensures the guard works correctly across rapid Enter key presses. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * cleanup(files): use shared icon import, remove no-op props, wrap handler in useCallback Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * improvement: tables, dropdown * improvement(docs): align sidebar method badges and polish API reference styling (#3484) * improvement(docs): align sidebar method badges and polish API reference styling * fix(docs): revert className prop on DocsPage for CI compatibility * fix(docs): restore oneOf schema for delete rows and use rem units in CSS * fix(docs): replace :has() selectors with direct className for reliable prod layout The API docs layout was intermittently narrow in production because CSS :has(.api-page-header) selectors are unreliable in Tailwind v4 production builds. Apply className="openapi-page" directly to DocsPage and replace all 64 :has() selectors with .openapi-page class targeting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(docs): bypass TypeScript check for className prop on DocsPage Use spread with type assertion to pass className to DocsPage, working around a CI type resolution issue where the prop exists at runtime but is not recognized by TypeScript in the Vercel build environment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(docs): use inline style tag for grid layout, revert CSS to :has() selectors The className prop on DocsPage doesn't exist in the fumadocs-ui version resolved on Vercel, so .openapi-page was never applied and all 64 CSS rules broke. Revert to :has(.api-page-header) selectors for styling and use an inline <style> tag for the critical grid-column layout override, which is SSR'd and doesn't depend on any CSS selector matching. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(docs): add pill styling to footer navigation method badges The footer nav badges (POST, GET, etc.) had color from data-method rules but lacked the structural pill styling (padding, border-radius, font-size). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(docs): use named grid lines instead of numeric column indices (#3487) Root cause: the fumadocs grid template has 3 columns in production but 5 columns in local dev. Our CSS used `grid-column: 3 / span 2` which targeted the wrong column in the 3-column grid, placing content in the near-zero-width TOC column instead of the main content column. Fix: use `grid-column: main-start / toc-end` which uses CSS named grid lines from grid-template-areas, working regardless of column count. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * improvement(resource): layout * improvement: icon, resource header options * improvement: icons * fix(files): icon * feat(tables): column operations, row ordering, V1 API (#3488) * feat(tables): add column operations, row ordering, V1 columns API, and OpenAPI spec Adds column rename/delete/type change/constraint updates to the tables module, row ordering via position column, UI metadata schema, V1 public API for column operations with rate limiting and audit logging, and OpenAPI documentation. Key changes: - Service-layer column operations with validation (name pattern, type compatibility, unique/required constraints) - Position column on user_table_rows with composite index for efficient ordering - V1 /api/v1/tables/{tableId}/columns endpoint (POST/PATCH/DELETE) with rate limiting and audit - Shared Zod schemas extracted to table/utils.ts using COLUMN_TYPES constant - Targeted React Query invalidation (row vs schema mutations) with consistent onSettled usage - OpenAPI 3.1.0 spec for columns endpoint with code samples - Position field added to all row response mappings for consistency - Sort fallback to position ordering when buildSortClause returns null Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tables): use specific error prefixes instead of broad "Cannot" match Prevents internal TypeErrors (e.g. "Cannot read properties of undefined") from leaking as 400 responses. Now matches only domain-specific errors: "Cannot delete the last column" and "Cannot set column". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tables): reject Infinity and NaN in number type compatibility check Number.isFinite rejects Infinity, -Infinity, and NaN, preventing non-finite values from passing column type validation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tables): invalidate table list on row create/delete for stale rowCount Row create and delete mutations now invalidate the table list cache since it includes a computed rowCount. Row updates (which don't change count) continue to only invalidate row queries. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tables): add column name length check, deduplicate name gen, reset pagination on clear - Add MAX_COLUMN_NAME_LENGTH validation to addTableColumn (was missing, renameColumn already had it) - Extract generateColumnName helper to eliminate triplicated logic across handleAddColumn, handleInsertColumnLeft, handleInsertColumnRight - Reset pagination to page 0 when clearing sort/filter to prevent showing empty pages after narrowing filters are removed Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: hoist tableId above try block in V1 columns route, add detail invalidation to invalidateRowCount - V1 columns route: `tableId` was declared inside `try` but referenced in `catch` logger.error, causing undefined in error logs. Hoisted `await params` above try in all three handlers (POST, PATCH, DELETE). - invalidateRowCount: added `tableKeys.detail(tableId)` invalidation since the single-table GET response includes `rowCount`, which becomes stale after row create/delete without this. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add position to all row mutation responses, remove dead filter code - Add `position` field to POST (single + batch) and PATCH row responses across both internal and V1 routes, matching GET responses and OpenAPI spec. - Remove unused `filterConfig`, `handleFilterToggle`, `handleFilterClear`, and `activeFilters` — dead code left over from merge conflict resolution. `handleFilterApply` (the one actually wired to JSX) is preserved. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: invalidateTableSchema now also invalidates table list cache Column add/rename/delete/update mutations now invalidate tableKeys.list() since the list endpoint returns schema.columns for each table. Without this, the sidebar table list would show stale column schemas until staleTime expires. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: replace window.prompt/confirm with emcn Modal dialogs Replace non-standard browser dialogs with proper emcn Modal components to match the existing codebase pattern (e.g. delete table confirmation). - Column rename: Modal with Input field + Enter key support - Column delete: Modal with destructive confirmation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * update schedule creation ui and run lint * improvement: logs * improvement(tables): multi-select and efficiencies * Table tools * improvement(folder-selection): folder deselection + selection order should match visual * fix(selections): more nested folder inaccuracies * Tool updates * Store tool call results * fix(landing): wire agent input to mothership * feat(mothership): resource viewer * fix tests * fix(streaming): smoother streaming with throttled rendering, ResizeObserver scroll, and batched updates (#3471) * fix(streaming): smoother streaming with throttled rendering, ResizeObserver scroll, and batched updates - Add useThrottledValue hook (100ms trailing-edge throttle) to gate DOM re-renders during streaming across all chat surfaces - Replace 100ms setInterval scroll polling with ResizeObserver-based auto-scroll, programmatic scroll timestamp tracking, and nested [data-scrollable] region handling - Extract processContentBuffer from inline content handler for cleaner code organization in copilot SSE handlers - Add RAF-based update batching (50ms max interval) to floating chat and home chat streaming paths - Add useProgressiveList hook for progressive rendering of long conversation histories via requestAnimationFrame Made-with: Cursor * ack PR comments * fix search modal * more comments * ack comments * count * ack comments * ack comment * improvement(mothership): worklfow resource * Fix tool call persistence in chat * Tool results * Fix error status * File uploads to mothership * feat(templates): landing page templates workflow states * improvement(mothership): chat stability * improvement(mothership): chat history and stability * improvement(tables): click-to-select navigation, inline rename, column resize (#3496) * improvement(tables): click-to-select navigation, inline rename, column resize * fix(tables): address PR review comments - Add doneRef guard to useInlineRename preventing Enter+blur double-fire - Fix PATCH error handler: return 500 for non-validation errors, fix unreachable logger.error - Stop click propagation on breadcrumb rename input * fix(tables): add rows-affected check in renameTable service Prevents silent no-op when tableId doesn't match any record. * fix(tables): useMemo deps + placeholder memo initialCharacter check - Use primitive editingId/editValue in useMemo deps instead of whole useInlineRename object (which creates a new ref every render) - Add initialCharacter comparison to placeholderPropsAreEqual, matching the existing pattern in dataRowPropsAreEqual * fix(tables): address round 2 review comments - Mirror name validation (regex + max length) in PatchTableSchema so validateTableName failures return 400 instead of 500 - Add .returning() + rows-affected check to renameWorkspaceFile, matching the renameTable pattern - Check response.ok before parsing JSON in useRenameWorkspaceFile, matching the useRenameTable pattern * refactor(tables): reuse InlineRenameInput in BreadcrumbSegment Replace duplicated inline input markup with the shared component. Eliminates redundant useRef, useEffect, and input boilerplate. * fix(tables): set doneRef in cancelRename to prevent blur-triggered save Escape → cancelRename → input unmounts → blur → submitRename would save instead of canceling. Now cancelRename sets doneRef like submitRename does, blocking the subsequent blur handler. * fix(tables): pointercancel cleanup + typed FileConflictError - Add pointercancel handler to column resize to prevent listener leaks when system interrupts the pointer (touch-action override, etc.) - Replace stringly-typed error.message.includes('already exists') with FileConflictError class for refactor-safe 409 status detection * fix(tables): stable useCallback dep + rename shadowed variable - Use listRename.startRename (stable ref) instead of whole listRename object in handleContextMenuRename deps - Rename inner 'target' to 'origin' in arrow-key handler to avoid shadowing the outer HTMLElement 'target' * fix(tables): move class below imports, stable submitRename, clear editingCell - Move FileConflictError below import statements (import-first convention) - Make submitRename a stable useCallback([]) by reading editingId and editValue through refs (matches existing onSaveRef pattern) - Add setEditingCell(null) to handleEmptyRowClick for symmetry with handleCellClick * feat(tables): persist column widths in table metadata Column widths now survive navigation and page reloads. On resize-end, widths are debounced (500ms) and saved to the table's metadata field via a new PUT /api/table/[tableId]/metadata endpoint. On load, widths are seeded from the server once via React Query. * fix type checking for file viewer * fix(tables): address review feedback — 4 fixes 1. headerRename.onSave now uses the fileId parameter directly instead of the selectedFile closure, preventing rename-wrong-file race 2. updateMetadataMutation uses ref pattern matching mutateRef/createRef 3. Type-to-enter filters non-numeric chars for number columns, non-date chars for date columns 4. renameValue only passed to actively-renaming ColumnHeaderMenu, preserving React.memo for other columns * fix(tables): position-based gap rows, insert above/below, consistency fixes - Fix gap row insert shifting: only shift rows when target position is occupied, preventing unnecessary displacement of rows below - Switch to position-based indexing throughout (positionMap, maxPosition) instead of array-index for correct sparse position handling - Add insert row above/below to context menu - Use CellContent for pending values in PositionGapRows (matching PlaceholderRows) - Add belowHeader selection overlay logic to PositionGapRows - Remove unnecessary 500ms debounce on column width persistence Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix cells nav w keyboard * added preview panel for html, markdown rendering, completed table --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(tables): one small tables ting (#3497) * feat(exa-hosted-key): Restore exa hosted key (#3499) Co-authored-by: Theodore Li <theo@sim.ai> * improvement(ui): consistent styling * styling alignment * improvements(tables): styling improvements * improve resizer for file preview for html files * updated document icon * fix(credentials): exclude regular login methods from credential sync * update docs * upgrade turbo * improvement: tables, chat * Fix table column delete * small table rename bug, files updates not persisting * Table batch ops * fix(credentials): block usage at execution layer without perms + fix invites * feat(hosted-key-services) Add hosted key for multiple services (#3461) * feat(hosted keys): Implement serper hosted key * Handle required fields correctly for hosted keys * Add rate limiting (3 tries, exponential backoff) * Add custom pricing, switch to exa as first hosted key * Add telemetry * Consolidate byok type definitions * Add warning comment if default calculation is used * Record usage to user stats table * Fix unit tests, use cost property * Include more metadata in cost output * Fix disabled tests * Fix spacing * Fix lint * Move knowledge cost restructuring away from generic block handler * Migrate knowledge unit tests * Lint * Fix broken tests * Add user based hosted key throttling * Refactor hosted key handling. Add optimistic handling of throttling for custom throttle rules. * Remove research as hosted key. Recommend BYOK if throtttling occurs * Make adding api keys adjustable via env vars * Remove vestigial fields from research * Make billing actor id required for throttling * Switch to round robin for api key distribution * Add helper method for adding hosted key cost * Strip leading double underscores to avoid breaking change * Lint fix * Remove falsy check in favor for explicit null check * Add more detailed metrics for different throttling types * Fix _costDollars field * Handle hosted agent tool calls * Fail loudly if cost field isn't found * Remove any type * Fix type error * Fix lint * Fix usage log double logging data * Fix test * Add browseruse hosted key * Add firecrawl and serper hosted keys * feat(hosted key): Add exa hosted key (#3221) * feat(hosted keys): Implement serper hosted key * Handle required fields correctly for hosted keys * Add rate limiting (3 tries, exponential backoff) * Add custom pricing, switch to exa as first hosted key * Add telemetry * Consolidate byok type definitions * Add warning comment if default calculation is used * Record usage to user stats table * Fix unit tests, use cost property * Include more metadata in cost output * Fix disabled tests * Fix spacing * Fix lint * Move knowledge cost restructuring away from generic block handler * Migrate knowledge unit tests * Lint * Fix broken tests * Add user based hosted key throttling * Refactor hosted key handling. Add optimistic handling of throttling for custom throttle rules. * Remove research as hosted key. Recommend BYOK if throtttling occurs * Make adding api keys adjustable via env vars * Remove vestigial fields from research * Make billing actor id required for throttling * Switch to round robin for api key distribution * Add helper method for adding hosted key cost * Strip leading double underscores to avoid breaking change * Lint fix * Remove falsy check in favor for explicit null check * Add more detailed metrics for different throttling types * Fix _costDollars field * Handle hosted agent tool calls * Fail loudly if cost field isn't found * Remove any type * Fix type error * Fix lint * Fix usage log double logging data * Fix test --------- Co-authored-by: Theodore Li <teddy@zenobiapay.com> * Fail fast on cost data not being found * Add hosted key for google services * Add hosting configuration and pricing logic for ElevenLabs TTS tools * Add linkup hosted key * Add jina hosted key * Add hugging face hosted key * Add perplexity hosting * Add broader metrics for throttling * Add skill for adding hosted key * Lint, remove vestigial hosted keys not implemented * Revert agent changes * fail fast * Fix build issue * Fix build issues * Fix type error * Remove byok types that aren't implemented * Address feedback * Use default model when model id isn't provided * Fix cost default issues * Remove firecrawl error suppression * Restore original behavior for hugging face * Add mistral hosted key * Remove hugging face hosted key * Fix pricing mismatch is mistral and perplexity * Add hosted keys for parallel and brand fetch * Add brandfetch hosted key * Update types * Change byok name to parallel_ai * Add telemetry on unknown models --------- Co-authored-by: Theodore Li <theo@sim.ai> * improvement(settings): SSR prefetch, code splitting, dedicated skeletons * fix: bust browser cache for workspace file downloads The downloadFile function was using a plain fetch() that honored the aggressive cache headers, causing newly created files to download empty. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(settings): use emcn Skeleton in extracted skeleton files * fix(settings): extract shared response mappers to prevent server/client shape drift Addresses PR review feedback — prefetch.ts duplicated response mapping logic from client hooks. Extracted mapGeneralSettingsResponse and mapUserProfileResponse as shared functions used by both client fetch and server prefetch. * update byok page * fix(settings): include theme sync in client-side prefetch queryFn Hover-based prefetchGeneralSettings now calls syncThemeToNextThemes, matching the useGeneralSettings hook behavior so theme updates aren't missed when prefetch refreshes stale cache. * fix(byok): use EMCN Input for search field instead of ui Input Replace @/components/ui Input with the already-imported EmcnInput for design-system consistency. * fix(byok): use ui Input for search bar to match other settings pages * fix(settings): use emcn Input for file input in general settings * improvement(settings): add search bar to skeleton loading states Skeletons now include the search bar (and action button where applicable) so the layout matches the final component 1:1. Eliminates layout shift when the dynamic chunk loads — search bar area is already reserved by the skeleton. * fix(settings): align skeleton layouts with actual component structures - Fix list item gap from 12px to 8px across all skeletons (API keys, custom tools, credentials, MCP) - Add OAuth icon placeholder to credential skeleton - Fix credential button group gap from 8px to 4px - Remove incorrect gap-[4px] from credential-sets text column - Rebuild debug skeleton to match real layout (description + input/button row) - Add scrollable wrapper to BYOK skeleton with more representative item count * chore: lint fixes * improvement(sidebar): match workspace switcher popover width to sidebar Use Radix UI's built-in --radix-popover-trigger-width CSS variable instead of hardcoded 160px so the popover matches the trigger width and responds to sidebar resizing. * revert hardcoded ff * fix: copilot, improvement: tables, mothership * feat: inline chunk editor and table batch ops with undo/redo (#3504) * feat: inline chunk editor and table batch operations with undo/redo Replace modal-based chunk editing/creation with inline editor following the files tab pattern (state-based view toggle with ResourceHeader). Add batch update API endpoint, undo/redo support, and Popover-based context menus for tables. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove icons from table context menu PopoverItems Icons were incorrectly carried over from the DropdownMenu migration. PopoverItems in this codebase use text-only labels. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: restore DropdownMenu for table context menu The table-level context menu was incorrectly migrated to Popover during conflict resolution. Only the row-level context menu uses Popover; the table context menu should remain DropdownMenu with icons, matching the base branch. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: bound cross-page chunk navigation polling to max 50 retries Prevent indefinite polling if page data never loads during chunk navigation across page boundaries. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: navigate to last page after chunk creation for multi-page documents After creating a chunk, navigate to the last page (where new chunks append) before selecting it. This prevents the editor from showing "Loading chunk..." when the new chunk is not on the current page. The loading state breadcrumb remains as an escape hatch for edge cases. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add duplicate rowId validation to BatchUpdateByIdsSchema Adds a .refine() check to reject duplicate rowIds in batch update requests, consistent with the positions uniqueness check on batch insert. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review comments - Fix disableEdit logic: use || instead of && so connector doc chunks cannot be edited from context menu (row click still opens viewer) - Add uniqueness validation for rowIds in BatchUpdateByIdsSchema - Fix inconsistent bg token: bg-background → bg-[var(--bg)] in Pagination Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove duplicate rowId uniqueness refine on BatchUpdateByIdsSchema The refine was applied both on the inner updates array and the outer object. Keep only the inner array refine which is cleaner. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address additional PR review comments - Fix stale rowId after create-row redo: patch undo stack with new row ID using patchUndoRowId so subsequent undo targets the correct row - Fix text color tokens in Pagination: use CSS variable references (text-[var(--text-body)], text-[var(--text-secondary)]) instead of Tailwind semantic tokens for consistency with the rest of the file Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove dead code and fix type errors in table context menu Remove unused `onAddData` prop and `isEmptyCell` variable from row context menu (introduced in PR but never wired to JSX). Fix type errors in optimistic update spreads by removing unnecessary `as Record<string, unknown>` casts that lost the RowData type. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: prevent false "Saved" status on invalid content and mark fire-and-forget goToPage calls ChunkEditor.handleSave now throws on empty/oversized content instead of silently returning, so the parent's catch block correctly sets saveStatus to 'error'. Also added explicit `void` to unawaited goToPage(1) calls in filter handlers to signal intentional fire-and-forget. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: handle stale totalPages in handleChunkCreated for new-page edge case When creating a chunk that spills onto a new page, totalPages in the closure is stale. Now polls displayChunksRef for the new chunk, and if not found, checks totalPagesRef for an updated page count and navigates to the new last page before continuing to poll. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Streaming fix -- need to test more * Make mothership block use long input instead of prompt input * improvement(billing): isAnnual metadata + docs updates (#3506) * improvement(billing): on demand toggling and infinite limits * store stripe metadata to distinguish annual vs monthly * udpate docs * address bugbot * Add piping * feat(clean-hosted-keys) Remove eleven labs, browseruse. Tweak firecrawl and mistral key impl (#3503) * Remove eleven labs, browseruse, and firecrawl * Remove creditsUsed output * Add back mistral hosting for mistral blocks * Add back firecrawl since they queue up concurrent requests * Fix price calculation, remove agent since its super long running and will clog up queue * Define hosting per tool * Remove redundant token finding --------- Co-authored-by: Theodore Li <theo@sim.ai> * Update vfs to handle hosted keys * improvement(tables): fix cell editing flash, batch API docs, and UI polish (#3507) * fix: show text cursor in chunk editor and ensure textarea fills container Add cursor-text to the editor wrapper so the whole area shows a text cursor. Click on empty space focuses the textarea. Changed textarea from h-full/w-full to flex-1/min-h-0 so it properly fills the flex container. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improvement(tables): fix cell editing flash, add batch API docs, and UI polish Fix stale-data flash when saving inline cell edits by using TanStack Query's isPending+variables pattern instead of manual cache writes. Also adds OpenAPI docs for batch table endpoints, DatePicker support in row modal, duplicate row in context menu, and styling improvements. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove dead resolveColumnFromEvent callback Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: unify paste undo into single create-rows action Batch-created rows from paste now push one `create-rows` undo entry instead of N individual `create-row` entries, so a single Ctrl+Z reverses the entire paste operation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: validate dates in inline editor and displayToStorage InlineDateEditor now validates computed values via Date.parse before saving, preventing invalid strings like "hello" from being sent to the server. displayToStorage now rejects out-of-range month/day values (e.g. 13/32) instead of producing invalid YYYY-MM-DD strings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: accept ISO date format in inline date editor Fall back to raw draft input when displayToStorage returns null, so valid ISO dates like "2024-03-15" pasted or typed directly are accepted instead of silently discarded. Date.parse still validates the final value. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add ISO date support to displayToStorage and fix picker Escape displayToStorage now recognizes YYYY-MM-DD input directly, so ISO dates typed or pasted work correctly for both saving and picker sync. DatePicker Escape now refocuses the input instead of saving, so the user can press Escape again to cancel or Enter to confirm — matching the expected cancel behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove dead paste boundary check The totalR guard in handlePaste could never trigger since totalR included pasteRows.length, making targetRow always < totalR. Remove the unused variable and simplify the selection focus calc. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * update openapi --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix dysfunctional unique operation in tables * feat(autosave): files and chunk editor autosave with debounce + refetch (#3508) * feat(files): debounced autosave while editing * address review comments * more comments * fix: unique constraint check crash and copilot table initial rows - Fix TypeError in updateColumnConstraints: db.execute() returns a plain array with postgres-js, not { rows: [...] }. The .rows.length access always crashed, making "Set unique" completely broken. - Add initialRowCount: 20 to copilot table creation so tables created via chat have the same empty rows as tables created from the UI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix signaling * revert: remove initialRowCount from copilot table creation Copilot populates its own data after creating a table, so pre-creating 20 empty rows causes data to start at position 21 with empty rows above. initialRowCount only makes sense for the manual UI creation flow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improvement: chat, workspace header * chat metadata * Fix schema mismatch (#3510) Co-authored-by: Theodore Li <theo@sim.ai> * Fixes * fix: manual table creation starts with 1 row, 1 column Manual tables now create with a single 'name' column and 1 row instead of 2 columns and 20 rows. Copilot tables remain at 0 rows, 0 columns. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: horizontal scroll in embedded table by replacing overflow-hidden with overflow-clip Cell content spans used Tailwind's `truncate` (overflow: hidden), creating scroll containers that consumed trackpad wheel events on macOS without propagating to the actual scroll ancestor. Replaced with overflow-clip which clips identically but doesn't create a scroll container. Also moved focus target from outer container to the scroll div for correctness. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix tool call ordering * Fix tests * feat: add task multi-select, context menu, and subscription UI updates Add shift-click range selection, cmd/ctrl-click toggle, and right-click context menu for tasks in sidebar matching workflow/folder patterns. Update subscription settings tab UI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(credentials): autosync behaviour cross workspace (#3511) * fix(credentials): autosync behaviour cross workspace * address comments * fix(api-key-reminder) Add reminder on hosted keys that api key isnt needed (#3512) * Add reminder on hosted keys that api key isnt needed * Fix test case --------- Co-authored-by: Theodore Li <theo@sim.ai> * improvement: sidebar, chat * Usage limit * Plan prompt * fix(sidebar): workspace header collapse * fix(sidebar): task navigation * Subagent tool call persistence * Don't drop suabgent text * improvement(ux): streaming * improvement: thinking * fix(random): optimized kb connector sync engine, rerenders in tables, files, editors, chat (#3513) * optimized kb connector sync engine, rerenders in tables, files, editors, chat * refactor(sidebar): rename onTaskClick to onMultiSelectClick for clarity Made-with: Cursor * ack comments, add docsFailed * feat(email-footer) Add "sent with sim ai" for free users (#3515) * Add "sent with sim ai" for free users * Only add prompt injection on free tier * Add try catch around billing info fetch --------- Co-authored-by: Theodore Li <theo@sim.ai> * improvement: modals * ran migrations * fix(mothership): fix hardcoded workflow color, tables drag line overflowing * feat(mothership): file attachment indicators, persistence, and chat input improvements - Show image thumbnails and file-icon cards above user messages in mothership chat - Persist file attachment metadata (key, filename, media_type, size) in DB with user messages - Restore attachments from history via /api/files/serve/ URLs so they survive refresh/navigation - Unify all chat file inputs to use shared CHAT_ACCEPT_ATTRIBUTE constant - Fix file thumbnail overflow: use flex-wrap instead of hidden horizontal scroll - Compact attachment cards in floating workflow chat messages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improvement: search modal * improvement(usage): free plan to 1000 credits (#3516) * improvement(billing): free plan to five dollars * fix comment * remove per month terminology from marketing * generate migration * remove migration * add migration back * feat(workspace): add workspace color changing, consolidate update hooks, fix popover dismiss - Add workspace color change via context menu, reusing workflow ColorGrid UI - Consolidate useUpdateWorkspaceName + useUpdateWorkspaceColor into useUpdateWorkspace - Fix popover hover submenu dismiss by using DismissableLayerBranch with pointerEvents - Remove passthrough wrapper for export, reuse Workspace type for capturedWorkspaceRef - Reorder log columns: workflow first, merge date+time into single column Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update oauth cred tool * fix(diff-controls): fixed positioning for copilot diff controls * fix(font): added back old font for emcn code editor * improvement: panel, special tags * improvement: chat * improvement: loading and file dropping * feat(templates): create home templates * fix(uploads): resolve .md file upload rejection and deduplicate file type utilities Browsers report empty or application/octet-stream MIME types for .md files, causing copilot uploads to be rejected. Added resolveFileType() utility that falls back to extension-based MIME resolution at both client and server boundaries. Consolidated duplicate MIME mappings into module-level constants, removed duplicate isImageFileType from copilot module, and replaced hardcoded ALLOWED_EXTENSIONS with composition from shared validation constants. Also switched file attachment previews to use shared getDocumentIcon utility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(home): prevent initial view from being scrollable Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * autofill fixes * added back integrations page, reverted secrets page back to old UI * Fix workspace dropdown getting cut off when sidebar is collapsed * fix(mothership): lint (#3517) * fix(mothership): lint * fix typing * fix tests * fix stale query * fix plan display name * Feat/add mothership manual workflow runs (#3520) * Add run and open workflow buttons in workflow preview * Send log request message after manual workflow run * Make edges in embedded workflow non-editable * Change chat to pass in log as additional context * Revert "Change chat to pass in log as additional context" This reverts commit e957dffb2f. * Revert "Send log request message after manual workflow run" This reverts commit 0fb92751f0. * Move run and workflow icons to tab bar * Simplify boolean condition --------- Co-authored-by: Theodore Li <theo@sim.ai> * feat(resource-tab-scroll): Allow vertical scrolling to scroll resource tab * fix(remove-speed-hosted-key) Remove maps speed limit hosted key, it's deprecated (#3521) Co-authored-by: Theodore Li <theo@sim.ai> * improvement: home, sidebar * fix(download-file): render correct file download link for mothership (#3522) * fix(download-file): render correct file download link for mothership * Fix uunecessary call * Use simple strip instead of db lookup and moving behavior * Make regex strip more strict --------- Co-authored-by: Theodore Li <theo@sim.ai> * improvement: schedules, auto-scroll * fix(settings): navigate back to origin page instead of always going home Use sessionStorage to store the return URL when entering settings, and use router.replace for tab switches so history doesn't accumulate. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(schedules): release lastQueuedAt lock on all exit paths to prevent stuck schedules Multiple error/early-return paths in executeScheduleJob and executeJobInline were exiting without clearing lastQueuedAt, causing the dueFilter to permanently skip those schedules — resulting in stale "X hours ago" display for nextRunAt. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(mothership): inline rename for resource tabs + workspace_file rename tool - Add double-click inline rename on file and table resource tabs - Wire useInlineRename + useRenameWorkspaceFile/useRenameTable mutations - Add rename operation to workspace_file copilot tool (schema, server, router) - Add knowledge base resource support (type, extraction, rendering, actions) - Accept optional className on InlineRenameInput for context-specific sizing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * revert: remove inline rename UI from resource tabs Keep the workspace_file rename tool for the mothership agent. Only the UI-side inline rename (double-click tabs) is removed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(mothership): knowledge base resource extraction + Resource/ResourceTable refactor - Extract KB resources from knowledge subagent respond format (knowledge_bases array) - Add knowledge_base tool to RESOURCE_TOOL_NAMES and TOOL_UI_METADATA - Extract ResourceTable as independently composable memoized component - Move contentOverride/overlay to Resource shell level (not table primitive) - Remove redundant disableHeaderSort and loadingRows props - Rename internal sort state for clarity (sort → internalSort, sortOverride → externalSort) - Export ResourceTable and ResourceTableProps from barrel Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(logs) Run workflows client side in mothership to transmit logs (#3529) * Run workflows client side in mothership to transmit logs * Initialize set as constant, prevent duplicate execution * Fix lint --------- Co-authored-by: Theodore Li <theo@sim.ai> * fix(import) fix missing file * fix(resource): Hide resources that have been deleted (#3528) * Hide resources that have been deleted * Handle table, workflow not found * Add animation to prevent flash when previous resource was deleted * Fix animation playing on every switch * Run workflows client side in mothership to transmit logs * Fix race condition for animation * Use shared workflow tool util file --------- Co-authored-by: Theodore Li <theo@sim.ai> * fix: chat scrollbar on sidebar collapse/open * edit existing workflow should bring up artifact * fix(agent) subagent and main agent text being merged without spacing * feat(mothership): remove resource-level delete tools from copilot Remove delete operations for workflows, folders, tables, and files from the mothership copilot to prevent destructive actions via AI. Row-level and column-level deletes are preserved. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: stop sidebar from auto-collapsing when resource panel appears (#3540) The sidebar was forcibly collapsed whenever a resource (e.g. workflow) first appeared in the resource panel during a task. This was disruptive on larger screens where users want to keep both the sidebar and resource panel visible simultaneously. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(mothership): insert copilot-created workflows at top of list (#3537) * feat(mothership): remove resource-level delete tools from copilot Remove delete operations for workflows, folders, tables, and files from the mothership copilot to prevent destructive actions via AI. Row-level and column-level deletes are preserved. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(mothership): insert copilot-created workflows at top of list * fix(mothership): server-side top-insertion sort order and deduplicate registry logic * fix(mothership): include folder sort orders when computing top-insertion position * fix(mothership): use getNextWorkflowColor instead of hardcoded color --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(stop) Add stop of motehership ran workflows, persist stop messages (#3538) * Connect play stop workflow in embedded view to workflow * Fix stop not actually stoping workflow * Fix ui not showing stopped by user * Lint fix * Plumb cancellation through system * Stopping mothership chat stops workflow * Remove extra fluff * Persist blocks on cancellation * Add root level stopped by user --------- Co-authored-by: Theodore Li <theo@sim.ai> * fix(autolayout): targetted autolayout heuristic restored (#3536) * fix(autolayout): targetted autolayout heuristic restored * fix autolayout boundary cases * more fixes * address comments * on conflict updates * address more comments * fix relative position scope * fix tye omission * address bugbot comment * Credential tags * Credential id field * feat(mothership): server-persisted unread task indicators via SSE (#3549) * feat(mothership): server-persisted unread task indicators via SSE Replace fragile client-side polling + timer-based green flash with server-persisted lastSeenAt semantics, real-time SSE push via Redis pub/sub, and dot overlay UI on the Blimp icon. - Add lastSeenAt column to copilotChats for server-persisted read state - Add Redis/local pub/sub singleton for task status events (started, completed, created, deleted, renamed) - Add SSE endpoint (GET /api/mothership/events) with heartbeat and workspace-scoped filtering - Add mark-read endpoint (POST /api/mothership/chats/read) - Publish SSE events from chat, rename, delete, and auto-title handlers - Add useTaskEvents hook for client-side SSE subscription - Add useMarkTaskRead mutation with optimistic update - Replace timer logic in sidebar with TaskStatus state machine (running/unread/idle) and dot overlay using brand color variables - Mark tasks read on mount and stream completion in home page - Fix security: add userId check to delete WHERE clause - Fix: bump updatedAt on stream completion - Fix: set lastSeenAt on rename to prevent false-positive unread Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback - Return 404 when delete finds no matching chat (was silent no-op) - Move log after ownership check so it only fires on actual deletion - Publish completed SSE event from stop route so sidebar dot clears on abort Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: backfill last_seen_at in migration to prevent false unread dots Existing rows would have last_seen_at = NULL after migration, causing all past completed tasks to show as unread. Backfill sets last_seen_at to updated_at for all existing rows. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: timestamp mismatch on task creation + wasSendingRef leak across navigation - Pass updatedAt explicitly alongside lastSeenAt on chat creation so both use the same JS timestamp (DB defaultNow() ran later, causing updatedAt > lastSeenAt → false unread) - Reset wasSendingRef when chatId changes to prevent a stale true from task A triggering a redundant markRead on task B Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: mark-read fires for inline-created chats + encode workspaceId in SSE URL Expose resolvedChatId from useChat so home.tsx can mark-read even when chatId prop stays undefined after replaceState URL update. Also URL-encode workspaceId in EventSource URL as a defensive measure. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: auto-focus home input on initial view + fix sidebar task click handling Auto-focus the textarea when the initial home view renders. Also fix sidebar task click to always call onMultiSelectClick so selection state stays consistent. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: auto-title sets lastSeenAt + move started event inside DB guard Auto-title now sets both updatedAt and lastSeenAt (matching the rename route pattern) to prevent false-positive unread dots. Also move the 'started' SSE event inside the if(updated) guard so it only fires when the DB update actually matched a row. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * modified tasks multi select to be just like workflows * fix * refactor: extract generic pub/sub and SSE factories + fixes - Extract createPubSubChannel factory (lib/events/pubsub.ts) to eliminate duplicated Redis/EventEmitter boilerplate between task and MCP pub/sub - Extract createWorkspaceSSE factory (lib/events/sse-endpoint.ts) to share auth, heartbeat, and cleanup logic across SSE endpoints - Fix auto-title race suppressing unread status by removing updatedAt/lastSeenAt from title-only DB update - Fix wheel event listener leak in ResourceTabs (RefCallback cleanup was silently discarded) - Fix getFullSelection() missing taskIds (inconsistent with hasAnySelection) - Deduplicate SSE_RESPONSE_HEADERS to spread from shared SSE_HEADERS - Hoist isSttAvailable to module-level constant to avoid per-render IIFE Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat(logs): add workflow trigger type for sub-workflow executions (#3554) * feat(logs): add workflow trigger type for sub-workflow executions * fix(logs): align workflow filter color with blue-secondary badge variant * feat(tab) allow user to control resource tabs * Make resources persist to backend * Use colored squares for workflows * Add click and drag functionality to resource * Fix expanding panel logic * Reduce duplication, reading resource also opens up resource panel * Move resource dropdown to own file * Handle renamed resources * Clicking already open tab should just switch to tab --------- Co-authored-by: Theodore Li <theo@sim.ai> * Fix new resource tab button not appearing on tasks * improvement(ui): dropdown menus, icons, globals * improvement: notifications, terminal, globals * reverted task logic * feat(context) pass resource tab as context (#3555) * feat(context) add currenttly open resource file to context for agent * Simplify resource resolution * Skip initialize vfs * Restore ff * Add back try catch * Remove redundant code * Remove json serialization/deserialization loop --------- Co-authored-by: Theodore Li <theo@sim.ai> * Feat(references) add at to reference sim resources(#3560) * feat(chat) add at sign * Address bugbot issues * Remove extra chatcontext defs * Add table and file to schema * Add icon to chip for files --------- Co-authored-by: Theodore Li <theo@sim.ai> * improvement(refactor): move to soft deletion of resources + reliability improvements (#3561) * improvement(deletion): migrate to soft deletion of resources * progress * scoping fixes * round of fixes * deduplicated name on workflow import * fix tests * add migration * cleanup dead code * address bugbot comments * optimize query * feat(sim-mailer): email inbox for mothership with chat history and plan gating (#3558) * feat(sim-mailer): email inbox for mothership with chat history and plan gating * revert hardcoded ff * fix(inbox): address PR review comments - plan enforcement, idempotency, webhook auth - Enforce Max plan at API layer: hasInboxAccess() now checks subscription tier (>= 25k credits or enterprise) - Add idempotency guard to executeInboxTask() to prevent duplicate emails on Trigger.dev retries - Add AGENTMAIL_WEBHOOK_SECRET env var for webhook signature verification (Bearer token) * improvement(inbox): harden security and efficiency from code audit - Use crypto.timingSafeEqual for webhook secret comparison (prevents timing attacks) - Atomic claim in executor: WHERE status='received' prevents duplicate processing on retries - Parallelize hasInboxAccess + getUserEntityPermissions in all API routes (reduces latency) - Truncate email body at webhook insertion (50k char limit, prevents unbounded DB storage) - Harden escapeAttr with angle bracket and single quote escaping - Rename use-inbox.ts to inbox.ts (matches hooks/queries/ naming convention) * fix(inbox): replace Bearer token auth with proper Svix HMAC-SHA256 webhook verification - Use per-workspace webhook secret from DB instead of global env var - Verify AgentMail/Svix signatures: HMAC-SHA256 over svix-id.timestamp.body - Timing-safe comparison via crypto.timingSafeEqual - Replay protection via timestamp tolerance (5 min window) - Join mothershipInboxWebhook in workspace lookup (zero additional DB calls) - Remove dead AGENTMAIL_WEBHOOK_SECRET env var - Select only needed workspace columns in webhook handler * fix(inbox): require webhook secret — reject requests when secret is missing Previously, if the webhook secret was missing from the DB (corrupted state), the handler would skip verification entirely and process the request unauthenticated. Now all three conditions are hard requirements: secret must exist in DB, Svix headers must be present, and signature must verify. * fix(inbox): address second round of PR review comments - Exclude rejected tasks from rate limit count to prevent DoS via spam - Strip raw HTML from LLM output before marked.parse to prevent XSS in emails - Track responseSent flag to prevent duplicate emails when DB update fails after send * fix(inbox): address third round of PR review comments - Use dynamic isHosted from feature-flags instead of hardcoded true - Atomic JSON append for chat message persistence (eliminates read-modify-write race) - Handle cutIndex === 0 in stripQuotedReply (body starts with quote) - Clean up orphan mothershipInboxWebhook row on enableInbox rollback - Validate status query parameter against enum in tasks API * fix(inbox): validate cursor param, preserve code blocks in HTML stripping - Validate cursor date before using in query (return 400 for invalid) - Split on fenced code blocks before stripping HTML tags to preserve code examples in email responses * fix(inbox): return 500 on webhook server errors to enable Svix retries * fix(inbox): remove isHosted guard from hasInboxAccess — feature flag is sufficient * fix(inbox): prevent double-enable from deleting webhook secret row * fix(inbox): null-safe stripThinkingTags, encode URL params, surface remove-sender errors - Guard against null result.content in stripThinkingTags - Use encodeURIComponent on all AgentMail API path parameters - Surface handleRemoveSender errors to the user instead of swallowing * improvement(inbox): remove unused types, narrow SELECT queries, fix optimistic ID collision Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(inbox): add keyboard accessibility to clickable task rows * fix(inbox): use Svix library for webhook verification, fix responseSent flag, prevent inbox enumeration - Replace manual HMAC-SHA256 verification with official Svix library per AgentMail docs - Fix responseSent flag: only set true when email delivery actually succeeds - Return consistent 401 for unknown inbox and bad signature to prevent enumeration - Make AgentMailInbox.organization_id optional to match API docs * chore(db): rebase inbox migration onto feat/mothership-copilot (0172 → 0173) Sync schema with target branch and regenerate migration as 0173 to avoid conflicts with 0172_silky_magma on feat/mothership-copilot. * fix(db): rebase inbox migration to 0173 after feat/mothership-copilot divergence Target branch added 0172_silky_magma, so our inbox migration is now 0173_youthful_stryfe. * fix(db): regenerate inbox migration after rebase on feat/mothership-copilot * fix(inbox): case-insensitive email match and sanitize javascript: URIs in email HTML - Use lower() in isSenderAllowed SQL to match workspace members regardless of email case stored by auth provider - Strip javascript:, vbscript:, and data: URIs from marked HTML output to prevent XSS in outbound email responses * fix(inbox): case-insensitive email match in resolveUserId Consistent with the isSenderAllowed fix — uses lower() so mixed-case stored emails match correctly, preventing silent fallback to workspace owner. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Kb args * refactor(resource): remove logs-specific escape hatches from Resource abstraction Logs now composes ResourceHeader + ResourceOptionsBar + ResourceTable directly instead of using Resource with contentOverride/overlay escape hatches. Removes contentOverride, onLoadMore, hasMore, isLoadingMore from ResourceProps. Adds ColumnOption to barrel export and fixes table.tsx internal import. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(sim-mailer): download email attachments and pass to LLM as multimodal content Attachments were only passed as metadata text in the email body. Now downloads actual file bytes from AgentMail, converts via createFileContent (same path as interactive chat), and sends as fileAttachments to the orchestrator. Also parallelizes attachment fetching with workspace context loading, and downloads multiple attachments concurrently via Promise.allSettled. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(connector): add Gmail knowledge base connector with thread-based sync and filtering Syncs email threads from Gmail into knowledge bases with configurable filters: label scoping, date range presets, promotions/social exclusion, Gmail search syntax support, and max thread caps to keep KB size manageable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(connector): add Outlook knowledge base connector with conversation grouping and filtering Syncs email conversations from Outlook/Office 365 via Microsoft Graph API. Groups messages by conversationId into single documents. Configurable filters: folder selection, date range presets, Focused Inbox, KQL search syntax, and max conversation caps. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * cleanup resource definition * feat(connectors): add 8 knowledge base connectors — Zendesk, Intercom, ServiceNow, Google Sheets, Microsoft Teams, Discord, Google Calendar, Reddit Each connector syncs documents into knowledge bases with configurable filtering: - Zendesk: Help Center articles + support tickets with status/locale filters - Intercom: Articles + conversations with state filtering - ServiceNow: KB articles + incidents with state/priority/category filters - Google Sheets: Spreadsheet tabs as LLM-friendly row-by-row documents - Microsoft Teams: Channel messages (Slack-like pattern) via Graph API - Discord: Channel messages with bot token auth - Google Calendar: Events with date range presets and attendee metadata - Reddit: Subreddit posts with top comments, sort/time filters All connectors validated against official API docs with bug fixes applied. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(inbox): fetch real attachment binary from presigned URL and persist for chat display The AgentMail attachment endpoint returns JSON metadata with a download_url, not raw binary. We were base64-encoding the JSON text and sending it to the LLM, causing provider rejection. Now we parse the metadata, fetch the actual file from the presigned URL, upload it to copilot storage, and persist it on the chat message so images render inline with previews. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * added agentmail domain for mailer * added docs for sim mailer * fix(resource) handle resource deletion deletion (#3568) * Add handle dragging tab to input chat * Add back delete tools * Handle deletions properly with resources view * Fix lint * Add permisssions checking * Skip resource_added event when resource is deleted * Pass workflow id as context --------- Co-authored-by: Theodore Li <theo@sim.ai> * update docs styling, add delete confirmation on inbox * Fix fast edit route * updated docs styling, added FAQs, updated content * upgrade turbo * fix(knowledge) use consistent empty state for documents page Replace the centered "No documents yet" text with the standard Resource table empty state (column headers + create row), matching all other resource pages. Move "Upload documents" from header action to table create row as "New documents". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(notifications): polish modal styling, credential display, and trigger filters (#3571) * fix(notifications): polish modal styling, credential display, and trigger filters - Show credential display name instead of raw account ID in Slack account selector - Fix label styling to use default Label component (text-primary) for consistency - Fix modal body spacing with proper top padding after tab bar - Replace list-card skeleton with form-field skeleton matching actual layout - Replace custom "Select a Slack account first" box with disabled Combobox (dependsOn pattern) - Use proper Label component in WorkflowSelector with consistent gap spacing - Add overflow badge pattern (slice + +N) to level and trigger filter badges - Use dynamic trigger options from getTriggerOptions() instead of hardcoded CORE_TRIGGER_TYPES - Relax API validation to accept integration trigger types (z.string instead of z.enum) - Deduplicate account rows from credential leftJoin in accounts API - Extract getTriggerOptions() to module-level constants to avoid per-render calls Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(notifications): address PR review feedback - Restore accountId in displayName fallback chain (credentialDisplayName || accountId || providerId) - Add .default([]) to triggerFilter in create schema to preserve backward compatibility - Treat empty triggerFilter as "match all" in notification matching logic - Remove unreachable overflow badge for levelFilter (only 2 possible values) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(settings): add spacing to Sim Keys toggle and replace Sim Mailer icon with Send Add 24px top margin to the "Allow personal Sim keys" toggle so it doesn't sit right below the empty state. Replace the Mail envelope icon for Sim Mailer with a new Send (paper plane) icon matching the emcn icon style. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * standardize back buttons in settings * feat(restore) Add restore endpoints and ui (#3570) * Add restore endpoints and ui * Derive toast from notification * Auth user if workspaceid not found * Fix recently deleted ui * Add restore error toast * Fix deleted at timestamp mismatch --------- Co-authored-by: Theodore Li <theo@sim.ai> * fix type errors * Lint * improvements: ui/ux around mothership * reactquery best practices, UI alignment in restore * clamp logs panel * subagent thinking text * fix build, speedup tests by up to 40% * Fix fast edit * Add download file shortcut on mothership file view * fix: SVG file support in mothership chat and file serving - Send SVGs as document/text-xml to Claude instead of unsupported image/svg+xml, so the mothership can actually read SVG content - Serve SVGs inline with proper content type and CSP sandbox so chat previews render correctly - Add SVG preview support in file viewer (sandboxed iframe) - Derive IMAGE_MIME_TYPES from MIME_TYPE_MAPPING to reduce duplication - Add missing webp to contentTypeMap, SAFE_INLINE_TYPES, binaryExtensions - Consolidate PREVIEWABLE_EXTENSIONS into preview-panel exports Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: replace image/* wildcard with explicit supported types in file picker The image/* accept attribute allowed users to select BMP, TIFF, HEIC, and other image types that are rejected server-side. Replace with the exact set of supported image MIME types and extensions to match the copilot upload validation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Context tags * Fix lint * improvement: chat and terminal --------- Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu> Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Theodore Li <teddy@zenobiapay.com> Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> Co-authored-by: Theodore Li <theodoreqili@gmail.com> Co-authored-by: Theodore Li <theo@sim.ai>	2026-03-13 21:02:08 -07:00