opensource/sim - sim - Gitea: Git with a cup of tea

mirror of https://github.com/simstudioai/sim.git synced 2026-09-24 15:45:35 +08:00

Author	SHA1	Message	Date
Waleed	23bc210bf8	fix(seo): noindex non-production sim.ai hosts, redirect indexed 404s (#5988 ) * fix(seo): noindex non-production sim.ai hosts, redirect indexed 404s Non-production deployments (dev.sim.ai, staging.sim.ai) serve the same build as www.sim.ai and were fully crawlable. Send X-Robots-Tag noindex for those hosts, and 301 seven marketing paths that an external SEO audit found returning 404. * fix(seo): parse only the first entry of a comma-joined forwarded host A multi-value x-forwarded-host survives the port split intact, so endsWith('.sim.ai') matched on the trailing entry and could apply noindex to the canonical site. Normalize with the same split(',')[0].trim() the rest of the repo uses for forwarded headers. * fix(seo): drop the /security redirect, complete the urls test mock security.txt advertises /security as its RFC 9116 Policy URI, so a permanent redirect to marketing would mislead that link and shadow a real policy page added later. urlsMock is installed globally and documents itself as carrying every real export, but was missing CANONICAL_SITE_HOST and isNonCanonicalSimHost — any test loading proxy.ts would have called undefined in track().	2026-07-27 16:33:22 -07:00
Theodore Li	334c7c81bc	fix(security): suppress the CodeQL password-hash alert on sha256Hex (#5982 ) CodeQL fails #5963 (the v0.7.46 release PR) with 2 high-severity js/insufficient-password-hash alerts on packages/security/src/hash.ts. Both are false positives that surfaced now because `d362e6816e` added sha256Base64Url to this file — touching it made every pre-existing taint path ending here count as 'new code changed by this pull request'. The flagged lines and all their callers are unchanged. Neither taint source is a human password: - hashApiKey passes an API key built from generateSecureToken(24) — 192 bits of randomBytes. A fast digest is the right construction for indexed lookup; a slow KDF would run on every authenticated request and buy nothing against an unsearchable keyspace. - passwordSlot passes an already-encrypted value, hashed to an 8-char discriminator that invalidates deployment auth tokens when the password changes — not credential storage. User credentials never reach this helper; Better Auth owns them and applies its own KDF. Suppress with the inline lgtm comments this repo already uses for the same class of judgement (see uuidV5 in ee/workspace-forking/lib/remap/block-identity.ts), and document the invariant on sha256Hex so a future caller knows a genuine password belongs in Better Auth, not here. Only the two flagged lines are suppressed — sha256Base64Url is left unannotated so a real finding there would still surface.	2026-07-27 18:03:36 -04:00
Theodore Li	ae49070bea	fix(slack): stop requesting the unapproved scopes, gate them behind an opt-in flag (#5977 ) * fix(slack): stop requesting the unapproved mention/assistant/DM scopes * feat(slack): gate the approval-pending scopes behind an opt-in env flag	2026-07-27 14:48:45 -04:00
Theodore Li	0dcbc56ef6	feat(tables): per-table mutation locks (schema/insert/update/delete) (#5960 ) * feat(tables): per-table mutation locks (schema/insert/update/delete) Adds four independent, admin-only locks to a table so a workspace can make it append-only, read-only, or schema-frozen. Enforced at the lib/table service layer rather than the routes, so the API, workflow blocks, and Mothership (which calls the services directly) are all covered by one assert. Violations return 423; changing a lock requires workspace admin and returns 403. * fix(tables): address review — CI snapshot, lock-aware jobs, UI gating - Add the drizzle meta snapshot the hand-written migration was missing, which made CI regenerate the lock columns as a phantom 0271 - Stop a queued or retried delete/update job that starts after its lock is enabled; a run that has already committed pages still finishes, since pages are never rolled back and aborting would leave an uncancelled half-done state - Map TableLockedError to 423 on the internal single-row DELETE (was a 500) - Gate the lock settings UI behind NEXT_PUBLIC_TABLE_LOCKS so it can't open onto a Save that 403s against the server-side flag - Respect the delete lock on column drops in the grid, matching assertColumnDestructive, instead of failing only on click * fix(tables): close lock gaps in async import and undo/redo - Assert insert (and delete for replace) at the start of the import worker. Replace deleted every row before the first insert assert, so an insert-locked table was wiped and then failed, leaving it empty. - Run the delete/update worker start checks through assertRowDelete / assertRowUpdate instead of reading the lock flags directly, so the worker and its enqueue site apply identical rules — including the workflow-column exemption a bulk update was being cancelled despite. - Make undo/redo verb mapping direction-aware for column ops: dropping or retyping a column needs the delete lock clear too, matching assertColumnDestructive, so those steps are skipped instead of stranded. * fix(tables): stop locks over-blocking workflow runs, duplicate, and imports - Run / Re-run / Stop no longer inherit the cell-edit gate: they write only workflow-output columns (which the update lock exempts) and Stop is a cancel - Duplicate needs only the insert lock — it inserts a full copied row, so it stays available on an append-only table - updateWorkflowGroup asserts the destructive rule only when a patch actually drops or remaps output columns; rename / autoRun / mapping edits need just the schema lock - Gate the import-async enqueue on insert (and delete for replace) so a locked table 423s instead of claiming the write-job slot and failing in the worker - Disable Import CSV under an insert lock, and wire the expanded cell editor and the previously-unused stable add-row handler to the lock-aware flags * fix(tables): keep column menu usable and let locks always be cleared - Pass the blocked-delete handler instead of undefined so ColumnOptionsMenu still mounts on a delete-locked table; withholding it hid the entire menu, including Insert column, which the delete lock must not affect - Restore ColumnHeaderMenu readOnly to permission-only: it swaps the header for a static label, which was also disabling pin, column-select and open-config - Assert the schema lock at the import-async enqueue when createColumns is set - Allow a locks PATCH that only clears locks even when the feature flag is off, and keep the settings entry reachable on a locked table, so flipping the kill switch can't strand a table with locks nobody can remove * fix(tables): scope the update-lock exemption and re-check locks per page - Make the workflow-output carve-out opt-in via `computedWrite`, set only by the cell-write path. It was caller-agnostic, so an ordinary API caller could PATCH a workflow-output column on an update-locked table - Re-read the lock before every page in the delete and update runners. The single worker-start check went stale immediately, so enabling a lock could not stop a job already deleting or overwriting rows. Committed pages still stay committed, exactly as with an explicit cancel. Import keeps its one-shot check: a half-imported table needs the deletes the lock now forbids - Route Insert column to the locked-action modal under a schema lock instead of leaving it live (regular header) or hiding the whole menu (group header) * fix(tables): revalidate locks in-transaction and unbreak backfill + partial unlock - Re-assert the lock inside each page-write transaction, under the same advisory lock updateTableLocks takes, so check-then-write is atomic against a lock change. The per-page check alone still left the page-selection await between the assert and the write - Pass computedWrite from the backfill runner: batchUpdateRows is the workflow-output backfill path, so scoping the exemption had made rebuilding outputs 423 on an update-locked table while live cell writes still worked - Gate the feature flag on a lock actually going off->on rather than on an all-false payload. The settings UI always submits all four flags, so with the flag off an admin could not clear one lock while another stayed on - Hoist the lock-kind -> flag map to lib/table/types as TABLE_LOCK_FLAGS * fix(tables): guard import batches in-transaction and split paste by verb - Re-assert the insert lock inside each import batch's insert transaction, under the same advisory lock updateTableLocks takes. Reversing the earlier "let the file finish" call: an admin can lift the lock to clean up a partial import, so honouring it beats letting the rest of the file land - Gate paste per verb. Overwriting existing rows is an update and extending past the last row is a full-row insert, so a paste-append still works on an append-only table while an overwrite explains itself. Refuses the whole paste rather than applying half of it - Space opens the same row editor as double-click, so it now follows the update lock instead of filling in a form that 423s on save * fix(tables): guard the remaining import writes and explain blocked Shift+Enter - Route the replace-mode wipe, the inferred-schema write and createColumns through the same guardBatch transaction as the batch inserts, so a delete or schema lock committed while the file is downloading or being sampled is seen instead of the job-start snapshot - Shift+Enter takes the manual-add path, so it now opens the lock modal like the Add row button instead of returning silently * improvement(tables): surface blocked lock actions as a toast, not a modal - Replace TableLockedModal with a warning toast carrying a "Lock settings" action button for admins. Being told you can't edit shouldn't cost a dismiss click, and the button still routes admins straight to the panel - Dedupe by id so repeated attempts on a locked cell replace one notice instead of stacking a column of them - Move the copy into lock-copy.ts alongside the rest of the lock vocabulary and tighten it for toast length * fix(tables): don't show the update-lock notice to users without write access Double-click checked the update lock before any permission check, so a read-only member on a locked table got "Editing rows is locked" — misleading (the lock isn't why they can't edit) and it swallowed the expanded-cell viewer, which is a legitimate read-only affordance. * improvement(tables): trim the update-lock toast copy * fix(tables): map append-import locks to 423 and stop conflating locks with permissions - The sync append branch returns instead of rethrowing, so the outer catch's mapper never saw a TableLockedError and every lock violation became a 500. Map it in that catch, with a regression test (replace mode already rethrows) - Stop mounting the workflow-group column menu for users without edit access: passing the blocked handlers unconditionally made it appear for read-only members and report a lock even on an unlocked table - Enter/F2 now raises the same lock notice as double-click and Space instead of silently doing nothing - guardBatch returns the freshly-read definition so addTableColumnsWithTx asserts live state; a schema lock cleared mid-import no longer fails the createColumns step. The snapshot pre-asserts now only run as the fallback for callers that pass no revalidator - Append-only no longer labels a table whose schema is also locked, which claimed columns were mutable when they weren't - Import dialog withholds Replace on a delete-locked table and create-column on a schema-locked one, instead of offering a configuration that only 423s * fix(tables): revalidate sync imports under the advisory lock, explain every blocked key - The sync append/replace paths asserted only the request-start snapshot, so a lock committed while the CSV was parsed still let the write through. Both now re-read under the schema advisory lock at the top of their own transaction, taken before acquireRowOrderLock so the order stays advisory -> rows_pos -> definitions - Delete/Backspace, Cmd+D, typeahead and cut raised no notice on an update-locked table; they now explain the lock, and stay silent for users without write access - The multipart import fetch threw a plain Error, dropping the 423 status, so the lock self-heal never ran and the stale detail cache survived. It throws ApiClientError now and both import-into-table hooks call handleTableLockRejection - Force append at submit when the delete lock landed while the dialog was open with Replace already selected * fix(tables): restore the workspace ownership check on copilot row deletes Switching deleteRow/deleteRowsByIds to take a TableDefinition dropped the workspaceId argument that previously scoped the query, and the two branches I added loaded the table without the ownership comparison every other operation in the tool performs — letting a caller delete rows from a table in another workspace. Both now reject a foreign table as not found.	2026-07-25 14:21:05 -04:00
Theodore Li	19c3b6f47d	feat(setup): setup wizard with browser-based Chat key handoff (#5911 ) * feat(setup): setup wizard with browser-based Chat key handoff Adds `bun run setup` and `bun run doctor` for local installs, and replaces the wizard's paste-your-Chat-key step with a browser handoff that never puts the key in a URL. * improvement(setup): drop the paste-a-key fallback, simplify consent copy The browser handoff is now the only path — the wizard waits on a spinner instead of racing a paste prompt. Consent card leads with "Connect your terminal" and moves the match-the-code disclaimer into the description. * fix(setup): pin kube context, keep secrets out of argv, validate reused keys Review findings from #5911: - helm/kubectl now run against the validated context instead of the ambient one - helm values are piped on stdin rather than passed as --set arguments - ENCRYPTION_KEY/API_ENCRYPTION_KEY are checked for the 64-hex format the app requires, not just length, so an unusable key is replaced rather than kept - the managed Redis container's published port is read back instead of assumed * refactor(copilot): one module for Chat API key operations list/generate/delete each repeated the same /api/validate-key envelope in their route. They now share callValidateKey in lib/copilot/server/api-keys.ts, which also keeps the display masking server-side so the full key can only ever leave at creation. * improvement(setup): reuse shared helpers, parallelize probes, drop dead code - PKCE verifier/state/pairing code now use generateSecureToken, generateRandomHex and generateShortId instead of hand-rolled randomBytes; the pairing loop's modulo was unbiased only because 256 % 32 == 0 - new sha256Base64Url in @sim/security/hash so both sides of the PKCE exchange derive the challenge from one implementation - isUsableSecret moved beside SECRET_KEYS so setup and doctor apply the same rule; doctor previously passed a key setup would replace - isTruthy narrowed to true/1, matching the app it claims to mirror — it accepted yes/on, so a flag could read on in doctor and off in the app - checkLive runs its five probes concurrently (~17s serial worst case) - detection overlaps the banner animation instead of queueing behind it - glyph.fail/glyph.warn at 13 sites that bypassed the constant; removed unused prompter exports, a dead ENV_PATHS re-export, and an unused export keyword * fix(setup): make doctor understand the compose env layout Compose writes a single root .env (what docker-compose reads via env_file) but the checks required the three per-app files, so a successful compose install was followed by doctor printing three failures and exiting 1 — and the whole coherence catalog was skipped because it keyed off apps/sim/.env existing. Layout is now derived from what's on disk and every check consults it: file and schema checks iterate the layout's targets, consistency reports skip when there's only one file to mirror, and coherence/live read the layout's primary file. The wizard's existing-config detection counts root for the same reason — a compose install used to read as unconfigured and re-run from scratch. * feat(cli-auth): device-authorization poll flow, drop the loopback listener The CLI no longer binds a local port. It generates a request id + poll secret, opens /cli/auth, and polls /api/cli/auth/poll over TLS while the user approves in the browser — so the flow works over SSH and inside containers, where the browser and terminal don't share a machine. - approve stores the approval keyed by request id (session-authed, userId from the session only); poll verifies the secret before an atomic claim, so an observer of the semi-public request id can neither mint nor cancel it - pairing code stays as the anti-phishing compare; no key ever crosses the browser; done page just confirms - removes the loopback listener, /token exchange, buildCliHandoffUrl, and validateCliCallbackUrl (+ its tests) — nothing hands a key to a URL anymore * fix(setup): reuse an existing managed Postgres container instead of colliding A running sim-postgres fell through to `docker run --name sim-postgres` and died on the name conflict; a stopped one failed with "no DATABASE_URL to reach it" because the generated password only lived in the env files a fresh clone lacks. Both facts are recoverable from Docker: the ladder now reads the published port and password back via `docker inspect` and reuses the container (starting it if stopped). A container that won't answer prompts before recreating, and never drops the data volume silently. * improvement(setup): audience-first run-mode hints Each run mode now names who it's for — compose for self-hosting/evaluating, dev for contributing to Sim, k8s for rehearsing a production deploy — with the live detection state (Docker/kube/VM) appended. * fix(cli-auth): retry a failed mint, port container/port fixes to Redis + k8s Review findings from #5911: - poll now reserves the mint with an atomic NX lock instead of deleting the approval up front, so a failed mint (e.g. mothership blip) is retried by the next poll instead of forcing a fresh browser approval; the lock still prevents a double-mint and its TTL frees the slot if the caller dies - setup reuses/recreates an unhealthy managed sim-redis instead of colliding on the name (Redis has no data volume, so it removes and recreates without a prompt) - k8s failure-path hints carry --context, matching the success-path hints, so a changed ambient context can't send diagnostics to the wrong cluster - compose port-free waits for a killed port to actually release before re-checking; SIGKILL is async, so the immediate re-check re-saw the port * fix(setup): harden mint cleanup, Windows browser, container detection, helm cwd Review findings from #5911: - a post-mint completeApproval failure no longer routes into releaseMint — the mint lock now outlives the approval (shared TTL), so a cleanup blip can't leave a re-mintable window and orphan a key; cleanup is best-effort after the key ships - compose doctor --fix writes the feature-flag twin to the layout's primary env (root .env on a compose install), not always apps/sim/.env - Windows opens the browser via `cmd /c start "" <url>` — `start` is a shell builtin, so spawning it directly ENOENT'd and the handoff never opened - managed-container detection filters loosely and pins the exact name in code; Docker's `name=^x$` anchor matches the internal `/x` form and often missed, skipping the reuse branch - the shared helm/kind run helper pins cwd to the repo root, matching helm test, so `helm upgrade --install ./helm/sim` works from any working directory * feat(chat-keys): standalone manage page, drop from settings nav, refresh README - Add /account/settings/chat-keys — a linkable page to view, create, and revoke Chat API keys - Remove Chat keys from the settings sidebar (account + unified nav) and its render branches - README: replace Docker Compose + Manual Setup with the bun run setup wizard; drop the manual COPILOT_API_KEY step, point to the manage page * fix(setup): per-key reason in the secret-replacement warning Cursor: the warn hardcoded '64-character hex key', but only ENCRYPTION_KEY/API_ENCRYPTION_KEY require that — BETTER_AUTH_SECRET/INTERNAL_API_SECRET only need length >= 32. Use the existing secretRequirement(key) helper so each replaced key reports its actual requirement. * fix(setup): compose doctor schema, cross-platform binary detection, quoted context hints - Doctor: for the compose (root) env layout, require only the secrets compose has no interpolation default for (BETTER_AUTH_SECRET/ENCRYPTION_KEY/INTERNAL_API_SECRET). DATABASE_URL/BETTER_AUTH_URL/NEXT_PUBLIC_APP_URL come from docker-compose ${VAR:-default}, so a healthy compose install no longer fails doctor. - Binary detection: use Bun.which instead of which (which is absent on Windows), so kubectl/helm/kind/docker resolve cross-platform. - k8s diagnostic hints: POSIX-quote the kube-context so a context with whitespace/metacharacters can't break or inject into a copied command. * fix(setup): quote kube-context in the helm uninstall tear-down hint too The tear-down hint used --kube-context ${context} raw while the sibling kubectl hints already used shq(); a context with whitespace/metacharacters could break or inject into the copied command. All copyable k8s hints now go through shq(context). * feat(setup): sim lifecycle CLI — start/stop/status/logs/down/reset Turn the setup entry into a 'sim' command umbrella so there's one place to run everything, not scattered docker/bun commands. Adds a global bin (bun link) + a bun run sim fallback. - Detects how you're running (compose file / managed dev containers / helm release) from disk + docker/helm state — no persisted mode. Ambiguous installs prompt. - start/stop/restart/logs work per mode; down removes containers (volumes kept); reset archives .env + wipes managed data; both destructive verbs confirm first. - status shows detected mode, container states, and app/realtime health. - Wizard outro + README now point at the sim commands and the one-time bun link. * feat(setup): 'bun run sim' is the primary entry; bare invocation prints help - Lead usage/wizard-outro/README with 'bun run sim <cmd>' (works with zero PATH setup); global bare 'sim' via bun link is an optional upgrade, with the ~/.bun/bin PATH caveat spelled out (Homebrew's bun omits it). - Bare 'sim' now prints help instead of launching the wizard; the wizard is 'sim setup'. The 'setup' npm script passes the keyword so 'bun run setup' is unchanged. * fix(setup): quote the auth URL for cmd /c start on Windows Cursor (High): cmd re-parses the command line and treats & in the query string as a command separator, so cmd /c start opened a URL truncated at the first &, breaking the key flow on win32 (the handoff URL always has request/challenge/pairing). Quote the URL and pass args verbatim so & stays literal. * fix(setup): verify kube-context is really local; lengthen CLI handoff wait - k8s: a context named like a local cluster (kind-, docker-desktop) can actually point at a remote API server. Verify the server host is loopback/docker-internal before defaulting the 'use this context?' confirm to yes; otherwise warn and default to no, so generated secrets can't ship to a remote cluster on a blind Enter. - cli-auth: bump the device-flow wait from 3 to 15 minutes so first-time users have time to sign up, wait for the email OTP, and approve before the terminal stops polling. The server-side approval record keeps its own short TTL, so a longer client wait only costs cheap rate-limited polls. fix(setup): only manage k8s lifecycle on a verified-local context Greptile: sim down/reset used the ambient kube-context, so switching context after setup could uninstall a same-named sim-dev release from the wrong cluster. Gate k8sInstall on the same locality check the wizard uses (API server is loopback/docker-internal) via a shared isLocalKubeContext helper — the wizard only ever deploys locally, so a remote current-context is never treated as a Sim install. * fix(setup): doctor skips placeholder secrets when seeding; reset names its target - checks: the missing-file autofix copied shared keys from apps/sim/.env whenever truthy, including .env.example placeholders — doctor --fix could seed unusable secrets into realtime/db env files. Skip placeholders, matching autofixForMissing. - lifecycle: reset now names the exact install (k8s context / compose file / dev containers) in its confirm, so a destructive reset can't silently hit the wrong same-named install after a context switch (down already names the context). * fix(cli-auth): size the poll rate limit to the poll cadence; honor Retry-After The poll route used the default public-IP bucket (10 burst, 5/min) but the CLI polls every 2s (30/min), so it 429'd within ~20s — worse behind a slow dev cold-compile. Give the endpoint a bucket matched to its cadence (60 burst, 60/min); it's not a brute-force surface (unknown request id returns pending, minting needs the 256-bit verifier). Also make the CLI honor Retry-After and back off on 429 so a shared-NAT per-IP limit degrades gracefully instead of hammering. * fix(setup): check ports before starting the dev server, not just compose Local dev auto-start spawned bun run dev:full with no port check, so it silently started a server that couldn't bind when 3000/3002 were already taken (e.g. another worktree's dev server). Extract compose's port-conflict resolver into a shared ensurePortsFree(ports) and run it before the dev start too — kill/recheck/leave, same as compose. Leaving the ports skips the auto-start with guidance instead of failing; compose still treats it as fatal. * fix(setup): verify the kube cluster is reachable, not just local A kubeconfig context can outlive its cluster — a kind cluster gets deleted or its Docker container stops (Docker/machine restart), but the context entry remains, pointing at a dead API-server port. The wizard checked the context looked local and handed it to helm, which failed with 'cluster unreachable'. Add a clusterReachable() liveness probe: only offer the current context when it actually answers; if a local context is dead, fall through to the kind path. There, if kind still knows 'sim' but it's stopped, start its node containers and wait for the API; if it's gone, create fresh. Either way the user gets a working cluster instead of a cryptic helm failure. * fix(helm): point appVersion at published image tags (v-prefixed, current) The chart's appVersion was "0.6.73", but CI publishes GHCR tags with a v prefix (its release-commit regex captures v0.7.45). Since sim.image defaults every image tag to Chart.AppVersion, a default helm install requested ghcr.io/simstudioai/{simstudio,realtime,migrations}:0.6.73 — a tag that has never existed — so app and realtime sat in ImagePullBackOff and helm --wait failed with 'progress deadline exceeded'. Any self-hoster installing with default values hit this, not just the setup wizard. Set appVersion to v0.7.45 (latest release on main; all three images verified present on ghcr) and bump the chart version to 1.1.1. Verified with helm lint, helm template (all images render as v0.7.45), and a live helm upgrade on a kind cluster where the new pods pull successfully while the old 0.6.73 pods remain in ImagePullBackOff. * Revert "fix(helm): point appVersion at published image tags (v-prefixed, current)" This reverts commit `28b6047d1d`. * chore(api-validation): rebaseline route count to 977 after staging merge Staging moved the baseline to 975; this branch's two CLI-auth routes (approve, poll) make 977. The clean merge absorbed the earlier +2 adjustment. * fix(settings): don't highlight a sibling nav item on nested settings pages /account/settings/chat-keys is a real page but deliberately not a nav item, so the sidebar's parseSettingsPathSection fell through to defaultSection ('general') and highlighted General — the page read as though it lived inside General. Resolve the sidebar's active item with a null default so an unmatched nested route highlights nothing, and widen SettingsSidebar's activeSection to string \| null. The section feeding the title/description provider keeps its default (pages override title/description anyway), and /account/settings/billing/credit-usage still correctly highlights Billing. * fix(setup,auth): manage explicitly-confirmed k8s contexts, fail loudly on reset, clear stale post-auth redirect - lifecycle: detection is now factual — a sim-dev release either exists on the current context or it doesn't. Gating on locality stranded a release the user explicitly confirmed during setup (status/start/stop/down/reset all claimed no k8s install). Locality is recorded instead and surfaced through describeInstall, which every destructive confirm renders, so acting on a non-local cluster is named and defaulted to no rather than silently blocked or silently allowed. - lifecycle: reset no longer discards helm uninstall's exit status. Env files are archived by that point, so claiming 'Reset complete' while the release still runs is the worst outcome — it now throws with retry/inspect commands. - auth: signup clears POST_AUTH_REDIRECT_STORAGE_KEY when it has no callbackUrl, and the verification-disabled path consumes it, so a stale CLI/invite destination can't leak into a later flow in the same tab.	2026-07-25 04:24:36 -04:00
Vikhyath Mondreti	8a2ae25d78	improvement(agent-streaming): add way to opt in for workflow executions (#5956 )	2026-07-24 20:10:06 -07:00
Vikhyath MondretiandClaude Fable 5	f43b52c569	fix(realtime): evict revoked collaborators from live workflow rooms (#5917 ) * fix(realtime): evict revoked collaborators from live workflow rooms via periodic read-access re-validation * fix * fix(realtime): close join/eviction race and make sweep cleanups independent Re-authorize immediately before socket.join so an in-flight join cannot reverse a sweep eviction, and run the sweep's best-effort cleanups independently so a room-state failure cannot skip the presence broadcast. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(realtime): single-flight role resolution and retry failed eviction cleanup Coalesce concurrent role resolutions per (user, workflow) so a slow stale read can never overwrite a recorded revocation, and defer failed eviction room-state cleanups into a per-sweep retry queue so collaborators are not left with a stale presence entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(realtime): scope room removal to the target workflow and detect swallowed cleanup failures Honor the workflowIdHint as the target room in both room managers so removing a stale room cannot clobber the mapping of a room the socket has since moved to, and confirm eviction cleanup via the returned workflowId plus an unswallowed mapping read so Redis failures actually defer into the retry queue. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(realtime): treat unconfirmed removals as failures and refresh role cache on fresh verify Treat any null removal result as a failed cleanup (the sweep always passes the target room, so null only means failure — including with expired mapping keys), move the same-room rejoin guard to a synchronous check immediately before the removal, and record verifyWorkflowAccess's fresh decision into the role cache so a re-granted user is not blocked by a stale cached revocation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(realtime): prefer a mid-flight recorded role decision over the in-flight query result If a fresh authoritative read (join-time verify) records a decision while a single-flighted resolution's query is in flight, keep the recorded decision instead of overwriting it with the potentially stale result. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(realtime): isolate the security scan from the Redis cleanup lane Run the revocation scan (local sockets + DB only) and the best-effort room-state cleanup as independently-guarded lanes so a hanging Redis command can stall only presence cleanup, never revocation enforcement. Evictions now enqueue cleanup instead of awaiting it, and the scan no longer reads presence for a fallback role. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(realtime): bound authorization waits in the revocation scan Race each socket's authorization check against a per-socket timeout and cap the whole pass with a budget below the sweep interval, so a hanging DB query skips that socket for the pass (never evicting on uncertainty) instead of wedging the scan lane and starving subsequent ticks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(realtime): round-robin the revocation scan so hung checks cannot starve later sockets Resume each scan pass after the last target the previous pass processed, so a fixed prefix of hanging authorization checks can never repeatedly consume the pass budget and leave sockets behind it unexamined. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-24 16:45:40 -07:00
Vikhyath Mondreti	17d77795b4	feat(providers): prompt caching capability + usage-based cache pricing (#5922 ) * improvement(providers): validation pass, and stream tool loop improvements * remove deploy options correctly * fix * feat(providers): prompt caching capability and usage-based cache pricing Replace the arbitrary cached-rate heuristic with a single cache-aware pricing function, and add prompt caching as an opt-in capability for Anthropic. Pricing: priceModelUsage in cost-policy.ts is now the only place cache arithmetic happens. Provider adapters normalize their wire shape into ModelUsage (input always excludes cache buckets); the pricing function never branches on provider. This removes five divergent behaviors, including the !!request.context heuristic that gave Router and Evaluator an unearned 10x input discount, and the overwrite that silently billed Anthropic cache reads and writes at zero. Also parses OpenAI cache_write_tokens, previously ignored. Caching: Anthropic gets a capability-gated advanced switch that places cache_control on the last tool and last system block; system is now always a TextBlockParam array. OpenAI gets a stable per-block prompt_cache_key with no UI, since its caching is automatic. * fix(providers): route OpenAI and Gemini block cost through cache-aware pricing Cache-aware pricing only reached trace segments. The billable block cost still called calculateCost on the cache-inclusive prompt total, so OpenAI cache hits and Gemini implicit-cache hits were charged at the full input rate and GPT-5.6+ cache writes went unbilled. Both providers now accumulate cache buckets and price through priceModelUsage, matching the Anthropic token convention where input excludes cache reads and writes. Cached counts are clamped to the prompt total so an over-reporting payload cannot bill more input than the request contained. * fix(streaming): redact tool payloads on selected outputs in public chat Redaction only ran on the empty-selection branch, but a deployment almost always selects outputs, so it was dead in the case it exists for. Selecting toolCalls streamed the raw arguments and results to a public chat client in a chunk frame, and providerTiming carried thinking content the same way. Both paths now extract from the sanitized block output rather than the raw log: the streamed selected output, which is the reachable vector, and the final envelope. Sanitizing the source rather than per selected path means a newly selectable field cannot reopen the hole. * refactor(providers): drop unreachable billing fallbacks Every provider pricing helper took a policy parameter no caller passed. Worse than dead: passing one would have double-applied the margin the central layer already applies. Removed, so providers can only price at list. Also removed guards that cannot fire. The central fallback normalized cache buckets no provider can reach it with (all three that report cache usage price themselves) and did so at a 1x write multiplier no vendor charges. priceModelUsage re-validated token counts the adapter had already clamped, and applyModelCostPolicy defaulted a required total field. Validation now happens once, in the adapter that parses the vendor payload and is the only layer that knows cache buckets are a subset of the prompt total.	2026-07-24 15:46:11 -07:00
Waleed	513292f17b	feat(sso): DNS domain verification gating org SSO registration (#5909 ) * feat(sso): DNS domain verification gating org SSO registration Add org-scoped domain ownership verification (DNS TXT challenge) as the security precondition for configuring SSO. Closes the first-come domain-claim vuln where any org could wire another company's domain to its own IdP. - New sso_domain table + migration 0266; existing org SSO domains are grandfathered as verified so live tenants are unaffected - Verified-domains settings UI (enterprise-gated) with add/verify/remove - Register route now requires a verified domain for org-scoped registration; personal SSO and already-grandfathered domains are unaffected - Self-host register script writes the verified sso_domain row directly, so script-driven registration stays backwards compatible * fix(sso): harden domain verification against concurrency + fix CI lint Addresses review findings on state invariants under concurrent/failed writes: - Add unique index on (organization_id, domain) so concurrent claims can't create duplicate pending rows; POST re-reads and stays idempotent on conflict - Verify flips the row only if it's still the exact pending challenge checked (guards deletion/token-rotation mid-DNS-lookup) and maps the partial unique index violation to 409 instead of an unhandled 500 - Wrap the self-host script's provider write + verified-domain upsert in a transaction so a failed ownership write can't leave a provider committed - Format 0266 snapshot/journal with biome (fixes @sim/db lint:check) * fix(sso): re-check domain verification before provider write (TOCTOU) The register gate checked the verified sso_domain row only at handler entry, then ran OIDC discovery before writing the provider. A verified row removed during that window could still complete registration. Extract the check into a closure and call it both as an entry fast-fail and authoritatively right before registerSSOProvider, alongside the existing domain-conflict re-check. * fix(sso): stop rotating verification token on idempotent re-add Re-adding a pending domain rotated its verification token, which invalidated a TXT record the admin may have already published and — under two concurrent re-adds — could return a token the racing write had already superseded, so the admin's DNS record would never verify. Return the existing row unchanged instead; the pending token is always shown in the UI, so it is never lost. * fix(sso): close register TOCTOU with compensating delete + harden edges Audit-driven hardening: - Close the residual register TOCTOU: registerSSOProvider is create-only (throws if the providerId exists), so a compensating delete after the write is provably safe — it can only remove the just-created row. If verification was revoked during the write, roll the provider back and 403. - Verify is now idempotent under concurrency: a same-org row already flipped to verified by a racing request returns 200, not a confusing 409. - Grandfather backfill + self-host script now match normalizeSSODomain's dominant transforms (lower + trim + strip leading wildcard) so a non-canonical legacy domain can't miss the runtime gate's lookup. Prod backfill result is unchanged. - Cleanup: drop dead default export, align card radius to sibling convention. * fix(sso): redact domain tokens from non-admins + fix script stale-update Round-5 review findings: - GET /domains redacted the pending TXT verification token (a management secret) to any org member. Now only owner/admins read it; members see the list and status without it. Non-Enterprise orgs get an empty list (entitlement flag only), never the domains/tokens. - Self-host script decided update-vs-insert from a read taken OUTSIDE the transaction; a provider deleted mid-flight made the UPDATE match zero rows silently while the verified-domain upsert still committed (orphaned domain). The decision now happens inside the transaction from the UPDATE's row count. * docs(sso): drop unshipped enforce-SSO / auto-join copy from verified domains Verified domains currently only gate SSO configuration. Remove the forward-looking references to enforcing SSO and auto-joining members (deferred to a later release) from the docs, settings copy, nav description, and schema comment so we don't promise unshipped features. * fix(sso): guard rollback to new providers only + Enterprise-gate domain removal Round-6 review findings: - The compensating provider rollback now only fires when the provider did not exist before this request (providerExistedBefore). registerSSOProvider is create-only today so reaching the rollback already implies a fresh create, but this makes the safety local and future-proof: if Better Auth ever allowed updating an existing provider, a revoked-verification rollback must not delete that pre-existing row. - DELETE /domains now requires an Enterprise plan like add/list/verify, so all domain mutations share one entitlement (the UI already hides removal from non-Enterprise orgs). Adds a delete-route test. * fix(sso): roll back the SSO provider by row id, not logical keys The compensating rollback deleted by (providerId, orgId). providerId is unique, so if this request's row were deleted and recreated by a concurrent registration in the narrow window before the rollback, the logical-key delete would remove that other request's provider. Delete by the primary-key id registerSSOProvider returns instead, so only the exact row this request created is ever removed. * chore(sso): final-review polish — trim script read, unify copy, doc migration edge Cosmetic cleanup from a final 4-track adversarial review (no bugs found in the new logic): - Self-host script: narrow the pre-transaction existence read to select({ id }) instead of SELECT * (it only feeds a log line now). - Unify invalid-domain copy ("for example acme.com") and the verified-elsewhere 409 wording ("is already verified by another organization") across routes. - p-3 shorthand on the domain row card. - Document the migration's rare two-orgs-share-a-domain grandfather behavior (login unaffected; validated no such duplicates in prod). * fix(sso): apply attribute mapping + make SSO edit work; drop dead guard Two pre-existing SSO bugs the final review surfaced (prod has one SSO org, RVW, script-registered with the default mapping, so neither change affects it): - Attribute mapping was passed at the top level of the register payload, which Better Auth ignores — it reads oidcConfig.mapping / samlConfig.mapping. Nest it so custom mappings actually apply. (Default mapping is unchanged, so existing logins are unaffected.) - Editing an SSO provider was broken: registerSSOProvider is create-only and threw on the existing providerId → generic 500. Route now detects a provider the caller already owns and updates it via Better Auth's updateSSOProvider, and surfaces Better Auth's own error status/message instead of a blanket 500. Also drops the now-unnecessary providerExistedBefore guard (the rollback deletes by the created row's primary-key id and register is create-only) and the earlier final-review polish (script read, unified copy, migration edge note). Smoke-test SSO login + edit on staging before merge (auth-path change). * fix(sso): require null org on personal-mode provider lookups (gate bypass) The personal branch of both provider-ownership lookups keyed on (providerId, userId) without requiring organizationId IS NULL. Because org providers store userId = their creator and providerId is globally unique, an org admin could send a personal-mode request (no orgId) — which skips the membership check and the domain-verification gate — yet still match, and then via the new update path move, their org's provider to an unverified domain. Add isNull(organizationId) to the personal branch of both clauses so it can only match a genuinely personal provider, matching the route's own isOwnedByCaller. Found by an adversarial review of the update path added in 394bda9f7. * fix(sso): script updates the observed provider by id, not providerId Inside the registration transaction the script updated WHERE providerId — the logical key. If the observed provider was deregistered and a replacement created with the same providerId before the transaction ran, that update would clobber the replacement's config and ownership. Update the specific observed row by its primary-key id instead; if it's gone we insert, which fails cleanly on the providerId unique constraint rather than overwriting the replacement. * fix(sso): script upserts provider via delete-then-insert (no unique constraint) sso_provider.provider_id is a plain (non-unique) index and prod holds legitimate duplicates, so the previous "update by id, else insert" could create a duplicate provider when the observed row was deregistered and replaced before the transaction — the fallback insert would succeed. Delete every row for the providerId then insert exactly one, inside the transaction, so the providerId ends up as exactly this config atomically. Linked accounts key on the providerId string (not the row id), so existing logins are unaffected. * fix(sso): guard compensating-delete row id so rollback can't silently no-op * chore(sso): regenerate migration as 0268 after merging staging Staging landed migrations 0266/0267, colliding with our 0266. Removed our migration, merged staging, and regenerated cleanly with drizzle-kit as 0268_sso_domain_verification (identical sso_domain table + indexes), then re-appended the grandfather backfill. api-validation baseline reconciled to 973 (staging 970 + our 3 domain routes). Also make the register-route test's registerSSOProvider mock return an id so the guarded compensating delete runs. * refactor(sso): share normalizeSSODomain via @sim/utils so script matches gate The self-host script canonicalized SSO domains with a minimal inline transform (lower+trim+wildcard) that diverged from the app's full normalizeSSODomain (protocol, port, path, trailing dot, email local part) — equivalent spellings could store a different ownership key than the runtime gate looks up. Move normalizeSSODomain into @sim/utils/sso-domain (a pure function) so the register route, the domain-claim route, and the script all use the identical canonicalizer. The script now skips the verified-domain record when SSO_DOMAIN isn't a valid registrable domain instead of storing a malformed key.	2026-07-24 00:34:16 -07:00
Vikhyath Mondreti Claude Fable 5 Waleed Latif	6dcc65be89	feat(skills): add skill editors (#5705 ) * feat(skills): permissions layer * chore(db): drop skill_member migration 0261 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0262 on latest staging Same DDL as the dropped 0261 (skill_member table, enums, indexes, skill.workspace_shared) plus the hand-written write-user backfill, renumbered after staging's 0261. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0263 after staging merge Staging claimed 0262 (strong_storm); same DDL plus the hand-written write-user backfill, renumbered on the merged snapshot chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(db): drop skill_member migration 0263 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0264 after staging merge Staging claimed 0263 (workflow_fork_sync_excluded); same DDL plus the hand-written write-user backfill, renumbered on the merged snapshot chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * make editing skills full page * fix disclaimer * edit access msg * fix lint * chore(db): drop skill_member migration 0264 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0265 after staging merge Staging claimed 0264 (fat_ikaris); same DDL plus the hand-written write-user backfill, renumbered on the merged snapshot chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix tests * simplify system * fix * fix lint * add mship skills docs * chore(db): drop skill_member migration 0265 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0266 after staging merge Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deps): override zod to 4.3.6 to dedupe nested copies breaking type-check better-auth 1.6.23 and fumadocs-mdx resolve ^4.3.6 to a nested zod 4.4.3, which makes @sim/auth's inferred betterAuth types non-portable (TS2883) and split docs onto a second zod instance. Both ranges accept the repo-wide pinned 4.3.6, so a single hoisted copy satisfies everything. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix lint * feat(skills,tools): fullscreen skill create + shared custom tool editor Moves the rich-markdown and custom-tool editing surfaces out of modals and onto full-page surfaces, and collapses the duplicated chrome behind shared components. Skills - Add /skills/new, a full-page create surface mirroring the skill detail page (CredentialDetailLayout + DetailSection + unsaved-changes guard). "Add to Sim" navigates there instead of opening a modal. - Import moves to a header action (SkillImportButton) backed by a shared readSkillFile helper; the GitHub-URL import and its /api/skills/import route are removed. - Skill name validation is now one shared validateSkillName, replacing three copies of the kebab-case rule and its messages. - The skill editor roster renders through the shared MemberRow instead of re-deriving its identity block, with a locked role control and a lock-reason tooltip explaining inherited workspace-admin access. Custom tools - Extract the canvas modal's schema/code editors into a shared custom-tool-editor module (fields, wand generation, schema helpers), cutting custom-tool-modal.tsx by ~900 lines. - Settings > Custom tools gains a full-page detail sub-view (SettingsPanel + SettingsSection + saveDiscardActions), deep-linkable via ?custom-tool-id. Rows are clickable; delete now lives only in the detail view. - Replace legacy Button/Input/Badge/Label with the chip family, move chip-field chrome into CodeEditor behind an error prop, and delete its dead wand button. Rich markdown field - maxHeight is now opt-in: omit it on a page and the editor grows with its content so the page owns the only scrollbar. Modals pass explicit caps. - The field variant drops to font-weight 400 to match adjacent chip fields. * fix(skills): address review round on create navigation, 409 copy, and editor audit - Skill create navigated using the first element of the upsert response, but that endpoint returns the caller's whole skill list (built-ins prepended) — match the new skill by its workspace-unique name instead. - The suggested-skill 409 toast claimed the skill existed but was not shared and told the user to ask a skill admin. Every workspace member can already see and use every skill, so a 409 only means the name is taken. - Adding an editor emitted the skill_shared event and SKILL_MEMBER_ADDED audit even when onConflictDoNothing skipped the insert on a concurrent add. Gate both on the insert actually returning a row. * chore: format skills-resolver test import * fix(skills,tools): audit fixes — autocomplete boundary, resize clipping, error routing Two real regressions introduced while simplifying the extracted editor: - The schema-param autocomplete's trigger was rewritten to match a trailing identifier, but the completion still split on separators. The two disagreed, so typing `data.ci` opened the menu and selecting replaced `data.ci` whole — eating the member-access prefix. Both now share one SCHEMA_PARAM_WORD regex. - The uncapped markdown field measured its height only on value change while always setting overflow-hidden, so any width change that re-wrapped lines clipped the tail with no scrollbar to reach it. Now re-measures via ResizeObserver. Also from the audit: - Generation writes bypass the code field's change handler, so an open autocomplete stayed over a disabled streaming editor; close it on busy. - Delete failures rendered in the Schema section's error slot on both custom tool surfaces; route them to a toast instead. - Skill create navigated away while still dirty, stranding the unsaved-changes guard's history sentinel so Back landed on an empty create form. - The Description field on skill create never received its error border. - Drop a double-applied opacity-50 (the editor already dims when disabled), a dead try/catch around a non-throwing call that also shadowed the error prop, and a stale reference to a /tools page that does not exist. - Docs still described the removed GitHub-URL import and the old Add Skill dialog; rewrite for the create page and file/paste import. * feat(tools): read-only tool detail, create lands on the new tool, drop dead wand prompt API - Viewers without edit rights could not open a custom tool at all, while the equivalent skill and custom-block surfaces both offer a read-only view. The detail page now takes `readOnly`: editors inert, no Save/Discard/Delete, no Generate. Creating still requires edit rights. - Creating a tool bounced back to the list while creating a skill lands on the new skill. Tools now do the same. The upsert returns the workspace's whole tool list (newest first) rather than just the new row, so the id is matched by title instead of by index — the same trap that produced the skill-create navigation bug. - Remove `openPrompt`/`closePrompt` from useWand. `closePrompt`'s last callers went away with the custom-tool-modal extraction and `openPrompt` had none before it; nothing reads `isPromptVisible` any more either. * fix(tools): read-only editors, design-system wrench, skills-matching tool identity - readOnly never reached the editors: the prop gated actions and Generate but the schema and code fields were still typable for viewers without edit rights. Wire disabled through both fields into CodeEditor. - The row icon used lucide's Wrench (strokeWidth 2) where @sim/emcn/icons ships one drawn for this system (1.55, tuned viewBox), and it inherited body text colour instead of --text-icon. Swap it. - Give the tool detail page the same identity heading as skill detail: tile, name, and description at the top left, instead of only a header title. - Extract ResourceTile so the skills and tools tiles share one definition (SkillTile now composes it), and add an opt-in `iconFilled` to SettingsResourceRow so the tools list tile matches the skills gallery. Both default to today's behaviour for every existing consumer. * fix(mentions): use the product's own glyph for every @ mention kind The `@` menu and the inserted chip mapped kinds to arbitrary lucide icons — `Sparkles` for a skill, a generic `File` for every file — while the rest of the product has a settled glyph per resource. Mirror CHAT_CONTEXT_KIND_REGISTRY, which Chat's `@` menu already renders from: - skill now uses AgentSkillsIcon, the same glyph SkillTile shows everywhere - workflow / folder / table / knowledge use the @sim/emcn/icons set the sidebar and the chat registry use - file derives its icon from the filename extension, so a .pdf and a .csv are distinguishable, matching the file list and Chat's context chips - integration keeps the block's brand icon from the registry Also drop the generic placeholder. `kind` is untrusted — the node schema defaults it to `''` and a hand-written `sim:` link can carry anything — but an unrecognized kind now yields no icon instead of a meaningless box, which is what the chat registry does. The menu already guarded a missing icon; the chip now does too, so this cannot crash on a malformed link. * chore(db): drop skill_member migration 0266 for regeneration on latest staging * feat(db): regenerate skill_member migration as 0267 after staging merge --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-07-23 19:52:00 -07:00
Bill Leoutsakos Bill Leoutsakos Cursor Vikhyath Mondreti	d24bc7eccb	feat(agent-stream): thinking and tool streaming (#5671 ) * feat(agent-stream): add agent-events thinking/tool streaming for chat and canvas Ship the agent-events-v1 protocol with provider tool loops, dual-gated chat thinking, DeepSeek/Groq/OpenAI reasoning wiring, and ChatGPT-like thinking chrome. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): clear stuck streaming UI and format db snapshot Biome was failing CI on migrations/meta/0261_snapshot.json. Also settle assistant streaming/tool flags when SSE ends without a terminal frame, without clobbering Stop's finalized content. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): satisfy biome format and import order Auto-format the sim package for CI lint:check, and repair the Anthropic streaming tool-loop payload after an unsafe delete-to-undefined rewrite. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): keep drained answer on abort and update migration journal test Treat AbortError from reader.cancel as a cancelled pump result so soft-complete retains answerText. Point the workspace storage migration journal assertion at 0261_chat_include_thinking. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(chat): keep Stop notice when server emits cancel error Ignore terminal SSE error frames after the user aborts so "Client cancelled request" cannot overwrite "Response stopped by user". Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(chat): ChatGPT-style thinking shimmer and stick-to-bottom scroll Add left-to-right shimmer on live thinking label/body, keep scroll working by shimmering an inner node, and follow the answer only while near the bottom. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): stop pump on client disconnect; soft-complete agents only Abort the agent stream pump when the projected HTTP body is cancelled so provider work does not continue after disconnect. Limit AbortError soft-success to Agent blocks so Function/HTTP cancels still fail in logs. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): persist includeThinking across pause snapshots Paused chat runs with Include thinking enabled were dropping the flag when serializing the pause snapshot, so resume always rebuilt streams without thinking/tool SSE frames. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): keep drained answer text when stream times out Persist pump answerText onto the streaming execution before throwing on timeout, and carry that partial content into the failed block output so logs match what the client already saw. Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(chat): auto-collapse tools chrome when tool streaming ends Match thinking UX: open while tools run, collapse when finished, and keep the panel open only if the user manually reopens it. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): settle canvas stream chrome on failure paths Clear agentStreamActive and settle running tool chips when blocks error, timeouts cancel runs, or execution ends without stream:done so the output panel does not stay on live Thinking/Using tools chrome. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lint): organize imports in terminal console store Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): mark open tools cancelled on HITL pause Pause can interrupt a tool loop without tool end events; settling those chips as success incorrectly showed unfinished tools as complete. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(db): drop branch-local 0261 migration ahead of staging merge * chore(db): regenerate include_thinking migration as 0266 post staging merge * fix(providers): resolve type errors in streaming tool loop call sites * fix(agent-stream): gate agent events opt-in and correct provider loop behavior - streamToolCalls and provider thinking requests now require run-level agentEvents opt-in (canvas on, chat dual-gated, API off) so existing runs keep pre-agent-events behavior exactly - OpenAI reasoning summaries opt-in + strip-and-retry on unverified-org 400 - streaming loops run tool postProcess again (firecrawl/exa async results) - bedrock live loop falls back to silent path for responseFormat - deepseek: reasoning_content pass-back unconditional, 'none' sends disabled - groq: x_groq.usage fallback, reasoning params gated, qwen none disables - gemini: functionCall parts echoed verbatim, local ids only for events - truncated turns (max_tokens/length) no longer execute partial tool calls - MAX_TOOL_ITERATIONS exit flushes last turn text as final answer - iterations reports actual model calls; shared loop plumbing extracted * refactor(agent-stream): consolidate protocol, dedupe client/server plumbing, hygiene - canonical ChatStreamFrame union + type guards consumed by server emitters and the chat client; stream_error restored to legacy log-only handling - strip thinking/tool args from providerTiming on public final envelopes - shared tool-chip lifecycle module for chat, canvas, and console store - shared sink-to-execution-events forwarder replaces the copy-pasted adapter in the execute route and HITL manager; LIVE_ONLY event set shared - stream:thinking payload field renamed data->text; canvas thinking batched - abort reasons carried as AbortError DOMExceptions so raw fetch consumers classify correctly; thinking cap renamed to chars and scope-documented - kimi wired for agent events like the other compat providers - deleted dead exports/step-N comments; fixtures match real wire shapes; loop tests use explicit mocks instead of importOriginal * test(agent-stream): cover the dual-gated execution path and typed abort reasons - chat route tests assert agentEvents reaches executeWorkflow only when policy and protocol header agree - execution-limits tests assert AbortError-typed reasons - executor metadata type carries agentEvents * fix(deploy-modal): align include-thinking spacing with the modal's 6.5px rhythm * docs(agent-stream): autogenerate per-model thinking/tool stream support on the Agent block page - capabilities.thinking.streamed ('full' \| 'summary' \| 'none') on models.ts, explicit for the Anthropic family where visibility varies per generation; getThinkingStreamVisibility exposes the derivation for docs and UI alike - scripts/sync-agent-stream-docs.ts regenerates the support tables between markers in workflows/blocks/agent.mdx from the model registry and STREAMING_TOOL_CALL_PROVIDERS; --check fails on drift or missing metadata - wired agent-stream-docs:check into CI next to the other sync gates * feat(anthropic): request summarized thinking display for omitted-default Claude models The newest Claude generations (Fable 5, Sonnet 5, Opus 4.8/4.7) default thinking.display to omitted — empty thinking blocks, no deltas. On agent-events runs Sim now opts back in with display: 'summarized', driven by the registry's streamed metadata; legacy runs keep the exact pre-agent-events request shape. Registry, generated docs, and the family capability table updated accordingly. * docs(skills): cover thinking.streamed and agent-stream docs sync in model skills * chore(deps): upgrade @anthropic-ai/sdk to 0.114.0 and adopt official types - adaptive thinking, display, and output_config are now SDK-typed; the only remaining custom payload field is output_format (beta-header structured outputs, which the SDK models as output_config.format instead) - anthropic stream events narrow on the SDK's discriminated unions instead of anonymous casts; compat deltas type content/tool_calls from the OpenAI SDK with vendor reasoning fields as an explicit optional extension - @sim/auth exposes an explicit VerifyAuth contract so its declarations no longer reference better-auth's nested zod instance (TS2883 under fresh install layouts); realtime consumer aligned - docs app zod pinned to the repo's exact 4.3.6 so ai SDK types bind the same zod instance (docs type-check was latently broken) - knowledge embedding tests made hermetic against local .env keys and hosted rotation fallback * refactor(providers): replace legacy as-any stream casts with annotated typed casts * refactor(providers): finish provider audit — remove dead byte-stream helper, annotate remaining legacy casts Audit of all 26 providers for the agent-events feature confirmed every streaming execution declares agent-events-v1 and every adapter emits AgentStreamEvent objects. Cleanup from the audit: the unconsumed legacy createOpenAICompatibleStream byte helper is deleted, and the remaining streamResponse-as-any casts (xai, nvidia, kimi, meta, zai, sakana) are annotated typed casts matching the groq/deepseek fix. * feat(streaming): stream answer text live during tool loops via turn_end protocol The live tool loops buffered all answer text per model turn (classification of intermediate vs final is only known at turn end), so gated surfaces saw thinking stream, then dead air with the thinking chrome stuck open, then the whole answer at once. Loops now emit text deltas live as `turn: 'pending'` plus a `turn_end` event per turn. The pump buffers pending text and projects it to the byte path (answerText/logs/memory/legacy clients) only on a final turn_end, so all settled semantics are unchanged. Gated surfaces render the pending text as it streams and reconcile with a reset when a turn resolves to tools: - public chat: live `chunk` frames from the sink + dual-gated `chunk_reset`; byte-path frame emission is suppressed to avoid duplicates (kept for response-format transformed streams via clientStreamTransformed) - canvas: forwarder emits live `stream:chunk` + `stream:chunk_reset`; the execute route and HITL resume readers stop re-emitting byte chunks; panel chat tracks per-block segments and replaces content on flush - chat client: per-block text segments, chunk_reset handling, and thinking chrome now settles on tool start as well as first answer chunk * fix(streaming): address validated review findings across provider gating and reset reconciliation Three-reviewer pass over the branch, findings validated against staging: - agent-handler forwards agentEvents to executeProviderRequest — the flag was computed but dropped in the field-by-field copy, so provider-side thinking requests (OpenAI summaries, Gemini includeThoughts, Anthropic summarized display) never activated on opted-in runs - openai: restore summary:'auto' alongside explicit reasoning effort — staging always paired them; gating summary purely on agentEvents changed legacy payloads - gemini: Gemini 2 + tools + responseFormat falls back to the silent path; the live loop never applied the deferred responseSchema for AUTO tools - openai-compat loop: malformed tool-argument JSON fails the call instead of executing with defaulted {} args (staging parsed inside the execution try) - openai-compat parser: a vendor id arriving after a synthesized start no longer renames the call (start/end ids stayed consistent) - stream-pump: abort closes the byte projection so a drain blocked on backpressure cannot deadlock teardown - chunk_reset removes the block from the client text order (deployed chat + panel chat) so a reset block re-registers at arrival position — fixes separator/order corruption when parallel blocks stream around a reset - resume route echoes the negotiated X-Sim-Stream-Protocol response header (parity with the chat route); docs: [DONE] wire shape + final-vs-error terminal semantics corrected * chore(deps): exempt pinned @anthropic-ai/sdk 0.114.0 from the release-age gate CI's bun install --frozen-lockfile blocks 0.114.0 (published 2026-07-23, younger than the 7-day supply-chain gate). The pin is exact and was vetted for the agent-events streaming work; following the existing bunfig pattern, the exclusion ages out on 2026-07-30 and should be dropped then. * chore(providers): fix double-cast-allowed annotation placement for the strict boundary audit The audit only recognizes the annotation on the line directly above the cast; two annotations had drifted behind intervening code lines (groq stream params, deepseek loop messages) and the OpenAI reasoning-summary widening cast was never annotated. No behavior change. * fix(chat): settle straggler tool chips as error when final reports failure A failed run can still terminate with a `final` frame carrying success: false; running chips previously settled green regardless of the outcome. * fix(canvas): wire agent stream chrome into run-from-block Run-from-block executions emit the same live stream:thinking/stream:tool events as full runs but registered none of the handlers, so the terminal never showed thinking or tool chips on that path. The per-run chrome (batched thinking writes + tool chip lifecycle + settlement on stream done, block error, and every terminal execution state) is extracted into a shared createAgentStreamChrome factory consumed by both paths. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>	2026-07-23 19:39:03 -07:00
Theodore Li	96c67e9123	feat(sandbox): add Daytona as a manual-flip failover for E2B (#5860 ) * feat(sandbox): add Daytona as a manual-flip failover for E2B E2B was a hard single point of failure: lib/execution/e2b.ts had no retry and no fallback, so a failed Sandbox.create() killed Python function blocks, JS-with-imports, shell, doc generation and the Pi cloud agent outright. Extract a SandboxRunner boundary (lib/execution/remote-sandbox) with an E2B runner and a Daytona runner, selected once per execution by the sandbox-provider-daytona AppConfig flag. Everything above the provider boundary — marker parsing, mount materialization, file export, corruption handling — is unchanged. Selection resolves before create() and never mid-execution, since user code has side effects. Each sandbox kind fails closed when its snapshot id is unset. Notes on the Daytona adapter: - language binds at create(), not per call: Daytona applies it as a sandbox label and silently runs JS through Python if passed to codeRun - Python routes via CodeInterpreter for its {name,value,traceback} error shape, which matches E2B's and keeps formatE2BError's line offsets correct - timeouts convert ms to seconds - the streaming path delivers env via the filesystem API, as SessionExecuteRequest has no env field and secrets must not reach a command line Drops the dead E2BExecutionResult.images field (populated, never consumed). * improvement(sandbox): select the provider by SANDBOX_PROVIDER env var Replaces the boolean sandbox-provider-daytona feature flag with a SANDBOX_PROVIDER env var naming the provider ('e2b' default, or 'daytona'). A boolean doesn't scale to a third adapter; a keyed registry does. - PROVIDERS is a Record<SandboxProviderId, SandboxProvider>, so adding an adapter is one entry plus one id-union member — an unhandled provider is a compile error, not a runtime surprise - resolveProvider() reads env synchronously and throws on an unknown value (fail fast) instead of an async feature-flag lookup - drops the sandbox-provider-daytona flag and SANDBOX_PROVIDER_DAYTONA fallback Verified end-to-end: a Python function block through the running app routes to Daytona (Creating Daytona sandbox, kind: code) with SANDBOX_PROVIDER=daytona. * fix(sandbox): gate remote execution by provider availability, not E2B Addresses the review round on #5860. - Availability was gated on isE2bEnabled / isE2BDocEnabled, so a Daytona-only deployment (E2B_ENABLED unset) had its Python/shell/JS-with-imports and doc paths rejected before the provider-neutral sandbox call could run. Replace both with provider-aware flags (isRemoteSandboxEnabled / isDocSandboxEnabled) derived from the selected SANDBOX_PROVIDER's own credentials + image. E2B behavior is unchanged (the E2B branch mirrors the old definitions exactly). - Make the function-block gate error messages provider-neutral. - Daytona's streaming runCommand (Pi) returned empty stdout/stderr and delivered output only via callbacks, so the Pi cloud flow — which parses markers from stdout and formats errors from stderr — saw nothing. Accumulate the streamed chunks and return them while still forwarding to the callbacks. Renames the env-flag exports (and the @sim/testing mock) to match. Adds a conformance test that the streamed Pi output lands in stdout/stderr. * fix(sandbox): resolve SANDBOX_PROVIDER case-insensitively Addresses the round-2 review on #5860. env-flags lowercased SANDBOX_PROVIDER for the availability gate, but resolveProvider looked up the raw value in a lowercase-keyed map — so 'Daytona' passed the gate then threw Unknown SANDBOX_PROVIDER at create. resolveProvider now normalizes casing identically. * fix(sandbox): use getErrorMessage in build/verify scripts check:utils flagged the inline `error instanceof Error ? error.message : ...` pattern in the two new scripts. Use getErrorMessage from @sim/utils/errors, matching the repo convention the check enforces. * fix(sandbox): fall back to stdout for Daytona failure text Daytona merges both streams into stdout and returns an empty stderr, but the shell-error, base64-export, and URL-mount error builders read only result.stderr — so Daytona failures surfaced a generic 'Process exited with code N' / 'base64 failed' / 'curl exited N' instead of the real command output that the API and agents rely on. Fall back to stdout before the generic message (provider-agnostic: E2B still populates stderr). Strengthens the shell-error conformance test to assert the real output surfaces. * fix(sandbox): enforce timeout on Daytona streaming; drop leftover E2B copy - Daytona's streaming path (Pi) started the command with runAsync:true and then awaited getSessionCommandLogs with no bound, so a hung command never timed out the way E2B's commands.run({ timeoutMs }) does. Race the log stream against the timeout; on expiry return exit 124 with the accumulated output, and the finally's deleteSession terminates the still-running command. - Two user-facing strings still named E2B after the provider-neutral rename (the isolated-vm sandboxPath remediation and the disabled-xlsx message). Made both provider-neutral. Adds a streaming-timeout conformance test. * fix(sandbox): handle orphaned stream promise + preserve error detail Two regressions from the previous timeout fix: - When the timeout won the race, the abandoned getSessionCommandLogs promise would reject on deleteSession with no handler (unhandledRejection). Attach a .catch that records the error and yields an 'error' outcome, so a late rejection is always handled. - The streaming catch dropped the thrown error, so failures before any chunks (env write, executeSessionCommand, missing cmdId) surfaced as empty output. Fall back to getErrorMessage(error) when nothing streamed. Adds conformance tests for the stream-reject and start-throw paths.	2026-07-23 19:19:00 -04:00
Waleed	3796e9db4f	improvement(access-control): edit group details, filter by status, and colocate chat deploy auth (#5902 ) * improvement(access-control): editable group details, status filters, block tooltips, chat auth colocation - General tab gains editable Name and Description fields wired into the existing dirty buffer, so the header Save/Discard chips and the unsaved-changes guard cover them. Save only sends changed fields and surfaces the route's duplicate-name 409. - Blocks, Model Providers, and Platform tabs gain an All/Enabled/Disabled filter beside their search field, evaluated against the editing buffer. - Every block row carries an Info badge with the block's description. The badge sits outside the label/expand button so it never toggles the row. - The chat deploy toggle moves out of Deploy Tabs into the Chat section alongside its allowed-auth-modes dropdown, mirroring the Files section. * improvement(access-control): polish group detail filters, tooltips, and details fields Follows the cleanup review: - reconcile the post-save baseline from the server response instead of local values, matching the scope/default writes - pin the status-filter dropdown width so the search field stops resizing - restore flex-1 on the block-name button and move the row hover surface to the wrapper so Info badges align in a column - add empty states for filter-empty lists and neutralize Select All there - surface a 'Name is required' message next to the disabled Save - align hint text on the field-hint tokens and drop a redundant TSDoc * refactor(access-control): keep chat and files toggles in the platform registry The first pass pulled hideDeployChatbot out of the declarative platformFeatures array and hand-rolled a Chat section beside the existing bespoke Files one. That forfeited search, status filtering, Select All, category grouping and the Info hint, and the replacement platformSectionVisible re-implemented two of those with different semantics — searching 'deploy' or 'deployment' hid the very control named Deployment, and Select All silently skipped both toggles. Both toggles are now ordinary registry entries under their own Chat and Files categories, with an id-keyed featureExtras map supplying the nested auth-mode dropdown. Search, filtering, Select All, hints and the empty state are correct by construction, and the parallel filter pipeline is gone. Also from the review: - index the allow-lists into Sets so per-row membership checks are O(1) - split the search and status passes so the common 'all' filter returns the searched list by reference and a checkbox toggle no longer re-sorts ~180 rows - extract StatusFilterChip and AuthModeField instead of stamping out the dropdowns three and two times - derive nameChanged/descriptionChanged once instead of repeating the comparisons in the save payload - lock the config key-order invariant the dirty check depends on with a test * polish(access-control): apply the second cleanup round - indent the nested auth-mode field so it lines up under its toggle's label instead of reading as a sibling row, and label the dropdown for screen readers - order Chat right after Deploy Tabs so the three deploy targets stay adjacent - flush the trailing Select All chips on the providers and platform rows - drop the doubled margin on the name error (SettingRow already gaps it) - hoist PLATFORM_FEATURES and PLATFORM_CATEGORY_ORDER to module scope - drop the useCallback on the two save/discard handlers; nothing observes their identity and their deps changed on every keystroke anyway - fix a comment that still pointed at a 'Hide Chat' toggle that no longer exists * fix(access-control): keep the block disclosure chevron inside its toggle button Splitting the chevron out so the Info badge could sit beside the name left the chevron with no click handler — the visible expansion affordance did nothing. It goes back inside the button; Info stays outside it, since an Info trigger is itself a button and cannot nest. * chore(access-control): use structuredClone in the config key-order test check:utils bans JSON.parse(JSON.stringify(...)). The clone only needs to hand the schema a distinct object; structuredClone preserves key order the same way. * fix(access-control): trim descriptions so a padded value can't wedge the form descriptionChanged compared a trimmed draft against the raw saved description, so a group whose stored description carried padding opened dirty and could never be cleared — Discard restored the same padded string, and the unsaved-changes guard then blocked navigation until a save rewrote it. The contract now trims description on create and update, matching what name already did, and the dirty check trims both sides so existing padded rows behave too. * improvement(access-control): seed the description buffer trimmed Keeps the editing buffer and the dirty baseline normalized the same way, so a legacy row with padding no longer shows stray whitespace in the input and the buffer never round-trips padding a save would strip anyway. * fix(emcn): forward aria-label/aria-labelledby from ChipDropdown to its trigger ChipDropdown destructures only its known props, so an aria-labelledby passed by a consumer never reached the trigger button — AuthModeField's wiring to its visible label was silently dropped and the control had no accessible name. Both attributes are now explicit, typed props forwarded to the trigger. Kept as two named props rather than a rest spread so the component still owns its chrome and consumers can't smuggle arbitrary attributes onto the button. * fix(access-control): correct the nested field indent, platform row hover, and aria name - aria-labelledby REPLACES the content-derived name, so naming the auth-mode dropdown with its caption alone dropped the selected value from the accessible name — worse than no attribute. It now references caption + trigger, which needed ChipDropdown to forward id as well. - The nested field's pl-[30px] assumed a 14px checkbox; the default is 16px, so the caption sat 2px left of the label it hangs under while the dropdown (not flush, so mx-0.5) sat at 32. Both are pl-8 + flush now. - Platform feature rows kept their hover surface on the label, so the highlight stopped 20px short of the row edge while the Blocks tab ran flush. Moved to the wrapper, matching the core-blocks cell. - Split the platform search and status passes like the provider and block lists, so a toggle no longer recomputes three chained memos. - Dropped an unreachable allLabel and a comment that would go stale on merge. * fix(access-control): trim the name comparison the same way as description nameChanged compared a trimmed buffer against an untrimmed baseline — the exact asymmetry already fixed for description. A group stored before the name schema gained .trim() opens permanently dirty: Save/Discard visible and the unsaved-changes modal firing on back, until a save rewrites it. Both buffers now seed trimmed and compare trimmed on both sides. Also dims the Info badge along with the row it belongs to when the block is disallowed.	2026-07-23 15:44:13 -07:00
Waleed	78fb2c0679	chore(deps): bump next to 16.2.11 to clear security advisories (#5890 ) * chore(deps): bump next to 16.2.11 to clear security advisories Patches SSRF, cache confusion, DoS, and middleware-bypass advisories (GHSA-89xv-2m56-2m9x et al.) affecting next < 16.2.11 across apps/sim, apps/docs, and packages/emcn. Excludes next/@next/env from the minimum-release-age gate until the 7-day window elapses on 2026-07-28. * chore(deps): drop aged-out typescript entries from release-age excludes typescript and @typescript/typescript6 passed the 7-day minimum-release-age gate (aged out 2026-07-15 and 2026-07-13), so their exclusions are no longer needed. Keeps @typescript/native-preview (permanent nightly builds) and the Pi packages (age out 2026-07-24).	2026-07-23 10:47:07 -07:00
Theodore Li	0199508706	fix(custom-blocks): stop image icons blowing up on class-sized surfaces (#5878 )	2026-07-23 04:11:45 -04:00
Waleed	227cd65f41	fix(mcp): bound and retry OAuth start so a transient stall recovers instead of a blank popup (#5874 ) * fix(mcp): bound and retry OAuth start so a transient stall recovers instead of a blank popup Empirically root-caused a blank/stuck authorize popup: the provider (planetscale) and our guarded OAuth fetch are both fast (120/120 legs clean from staging), and /oauth/start uses local AES encryption with no Redis lock in its path — so the intermittent hang is the same transient headers-then-stalled-body class we've documented for CDN-fronted MCP hosts (a per-connection stall a fresh attempt dodges), which /oauth/start had no server-side bound against. - Bound every /oauth/start step with the shared timedStep helper (extracted from the callback route, now used by both) + an entry log, so a stalled step surfaces as a labeled error instead of hanging the request (and the browser popup) to the client's 30s timeout. - Retry mcpAuthGuarded once on a bounded 12s timeout: a fresh attempt gets a fresh connection and recovers from the transient stall automatically (two 12s attempts stay under the client's 30s deadline). McpOauthRedirectRequired (the success signal) and DCR-unsupported errors are never retried. Adds OauthStepTimeoutError + makeTimedStep to the shared oauth barrel and test mocks. * fix(mcp): drop the unsafe OAuth-start retry; fail fast without error-logging success Review fixes on the bound+retry change: - Removed the mcpAuthGuarded auto-retry. timedStep can't cancel the loser, so a lingering first attempt shares this server's OAuth row and could overwrite the retry's PKCE verifier / state after the client already got the second authorize URL, breaking the callback. Recovery is now fail-fast (504) → the user re-clicks, which is a clean fresh flow (fresh connection dodges the transient stall) with no shared-state race. - Catch McpOauthRedirectRequired (the success signal) INSIDE the bounded step and return it as a value, so a successful authorize is no longer error-logged as 'OAuth step failed'. - Tighten step budgets (5s DB x3 + 12s auth = 27s) to stay under the client's 30s /oauth/start deadline. * fix(mcp): route all bounded-step timeouts to the 504 handler Move OauthStepTimeoutError handling to the outer catch so a DB-step timeout (loadServer/getOrCreateOauthRow/loadPreregisteredClient) returns the same fast 504 'try again' as the auth step, not a generic 500. Documents that the fresh retry is race-safe: the callback correlates on the state nonce, so a lingering timed-out attempt overwriting the row's state only yields a clean invalid_state on the user's fresh authorize URL — never silent corruption. * fix(mcp): bound the setOauthRowUser write too so no step escapes the budget The user-stamp write was the one DB op left unbounded on the start path; wrap it in timedStep(DB_STEP_MS) so every step stays inside the sub-30s budget and its timeout routes to the same 504. * fix(mcp): shrink OAuth-start step budgets to fit the 30s client deadline with 4 DB steps Bounding setOauthRowUser added a fourth possible DB step, so 4x5+12=32s exceeded the client's 30s /oauth/start abort. Lower DB steps to 4s and auth to 10s: 4x4+10=26s worst case, leaving margin for middleware/network. Comment corrected.	2026-07-22 20:31:24 -07:00
Waleed	dda602a367	improvement(tests+ci): phase 3 — shared-mock convergence completion and CI runner-minute cuts (#5875 ) * chore(ci): cut redundant runner minutes — dedup promotion-PR test runs, companion-pr-check concurrency, right-size trivial jobs - ci.yml: new dedup-promotion gate skips the pull_request test-build on staging/main-headed promotion PRs only when the merge tree provably equals the head tree (empty base delta over the merge base) AND the push-event run at the same sha passed its test jobs (polled). Fail-open on any error/ timeout/failure, job-level skip only (skipped job reports Success); verified no required status checks are configured on main/staging rulesets. Measured 39 duplicate PR runs / 5.15 days (~227/mo) at ~7.1 min each on 8vcpu (~57 vcpu-min), probe costs ~9 vcpu-min worst case on 2vcpu. - companion-pr-check.yml: per-PR concurrency group with cancel-in-progress so superseded synchronize/edit runs stop; no paths filter (check depends on PR body + cross-repo state, not changed files). - detect-version and check-docs-changes: 4vcpu -> 2vcpu Blacksmith runners (pure shell / depth-2 checkout + path filter only). * improvement(testing): complete stateful shared mocks for env, urls, redis-config, environment-utils Shared mock infrastructure for vitest isolate:false convergence: - packages/testing/src/mocks/env.mock.ts: stateful envMock (live env proxy, setEnv/resetEnvMock, process.env fallback) - packages/testing/src/mocks/urls.mock.ts: complete urlsMock with real-behavior default impls + resetUrlsMock - packages/testing/src/mocks/redis-config.mock.ts: adds getRedisConnectionDefaults + resetRedisConfigMock - packages/testing/src/mocks/environment-utils.mock.ts: new environmentUtilsMock + fns + reset - contract tests: env.mock.test.ts, urls.mock.test.ts, redis-config.mock.test.ts, environment-utils.mock.test.ts - packages/testing/src/mocks/index.ts: barrel exports - apps/sim/vitest.setup.ts: global installs for env, urls, redis, environment/utils - real-module tests unmocked: lib/core/config/env.test.ts, lib/core/config/redis.test.ts, lib/core/utils/urls.test.ts, tools/index.test.ts (urls) - stubEnv/process.env fallout migrated to setEnv: lib/webhooks/providers/{revenuecat,rootly,instantly}.test.ts, app/api/auth/oauth2/authorize/route.test.ts * improvement(tests): drop redundant local mocks in executor/tools/providers and misc dirs (shared-worker readiness) * improvement(tests): drop redundant local mocks in app routes (shared-worker readiness) * improvement(tests): drop redundant local mocks in lib (shared-worker readiness) * fix(ci+testing): live base-tip recheck before dedup skip; prod-aware urls mock fallbacks - the dedup gate re-verifies merge-tree equivalence against the LIVE base tip at decision time, closing the window where the base branch gains real commits during the poll (frozen BASE_SHA check alone was stale) - the urls mock's getBaseUrl protocol prefix and getBaseDomain parse fallback now follow the shared isProd flag, mirroring the real module * fix(ci+testing): fail-closed nojobs fallback in dedup gate; TLS-aware redis defaults mock - the dedup gate no longer infers coverage from overall run conclusion when no 'Test and Build /' jobs match — a renamed or skipped test job now runs the tests instead of skipping them - the shared getRedisConnectionDefaults mock mirrors the real TLS resolution (rediss:// to a raw IP requires REDIS_TLS_SERVERNAME and yields tls.servername) * fix(ci): keep polling while nested test jobs have not appeared yet An in-progress push run lists its reusable-workflow jobs only after the caller starts; nojobs is now terminal (fail closed) only once the run has completed without them.	2026-07-22 19:04:17 -07:00
Waleed	9d8e14ce9f	improvement(tests): converge env-flags mocks onto a complete shared mock (#5871 ) * improvement(tests): converge env-flags mocks onto a complete shared mock * fix(tests): drop the repo's only bare vi.mock automock A bare vi.mock('drizzle-orm') automock colliding with factory mocks of the same module in a shared worker corrupts vitest's mock registry (upstream vitest-dev/vitest#10290 / #10145, reproduced in isolation). The global factory mock already covers this suite. * chore(tests): drop defensive resets in non-mutating suites, merge sequential setEnvFlags calls * fix(tests): run providers/utils cases sequentially over shared env-flags state	2026-07-22 17:31:18 -07:00
Waleed	2b5a92a3c8	feat(auth): org session policies — lifetime/idle limits, org-wide revocation (#5862 ) * feat(auth): org session policies — lifetime/idle limits, org-wide revocation, cookie-cache versioning * refactor(auth): consolidate session-policy clamp semantics, shared security-policy version module, canonical bounds, docs * polish(session-policy): cleanup pass — muted field labels, spinner reset, state tracker, response-seeded baseline, comment trims * fix(session-policy): govern member sessions by membership (closes revoke cookie-cache hole), normalize createdAt, remount on org switch, sync audit mock * fix(session-policy): clamp pre-join sessions on invite acceptance, normalize expiresAt, sync unified nav test * fix(session-policy): invalidate membership cache on removal/transfer, spare impersonator sessions in revoke-all, raise idle floor to 2x cookie window * fix(session-policy): resolve governing org by membership only — activeOrganizationId goes stale across transfer/leave * fix(session-policy): atomic policy save + eager clamp, asymmetric membership TTL, admin-add cache invalidation * fix(session-policy): org-scoped cookie version string, atomic revoke delete+bump * fix(session-policy): plan-gate effective policy so downgraded orgs stop enforcing automatically * chore(session-policy): drop dead bumpSecurityPolicyVersion helper — call sites bump transactionally * fix(session-policy): unify join paths on applySessionPolicyToNewMember; final audit polish (dead exports, response bound, test name)	2026-07-22 16:42:59 -07:00
WaleedandMarcus Chandra	8cce661a37	feat(api): proxyUrl for residential/custom proxy egress on the API block (#5867 ) * feat(api): add proxyUrl for residential/custom proxy egress on the API block The HTTP/API block egresses from the app runtime's fixed datacenter IPs via secureFetchWithPinnedIP, so targets behind Cloudflare/WAF that block datacenter IPs (e.g. state .gov license portals) return 403/429 even when the identical request works from a browser. There was no way to route a request through a residential/custom proxy. Add an optional `proxyUrl` field (Advanced) to the API block. When set, the request routes through the given http:// proxy so it egresses from that proxy's IP. Security: - validateAndPinProxyUrl resolves the proxy host's DNS and blocks private/reserved/loopback IPs (same SSRF guard as target URLs), then pins the connection by rewriting the host to the resolved IP (creds/port preserved), closing the DNS-rebinding window. - Restricted to the http: proxy scheme (https/socks rejected) so host pinning is safe without breaking TLS-to-proxy SNI. - Target-IP pinning is intentionally bypassed when a proxy is active (the proxy resolves the target); target URL validation still runs. Threaded block field -> http tool param -> formatRequestParams -> executeToolRequest (validate + pin) -> secureFetchWithPinnedIP, which swaps its pinned Node agent for HttpsProxyAgent/HttpProxyAgent (keyed off target protocol) when proxyUrl is set. * docs(api): document the Proxy URL advanced field and steer proxy credentials to env vars * fix(api): reject loopback/private proxy hosts unconditionally, closing the self-hosted rebinding gap * chore(api): tighten proxy-path inline comments --------- Co-authored-by: Marcus Chandra <mzxchandra@gmail.com>	2026-07-22 16:24:31 -07:00
Waleed	52659d4a89	improvement(tests): db-mock migration tranche 2 — copilot, mothership, workspaces, connectors, mcp (#5863 ) * improvement(tests): db-mock migration tranche 2 — copilot, mothership, workspaces, connectors, mcp * fix(testing): drain unconsumed ...Once overrides in resetDbChainMock vi.clearAllMocks clears call history only — a ...Once override queued by a previous test but never consumed survived into the next test. resetDbChainMock now mockReset()s every shared spy and stable wrapper, which restores the original implementation AND drains once-queues.	2026-07-22 15:17:43 -07:00
Waleed	62a8ce4953	improvement(tests): db-mock migration tranche 1 — knowledge, billing/org, workflows/background (#5861 ) * improvement(tests): migrate knowledge, billing/org, and workflows/background suites off private @sim/db factories * improvement(tests): db-mock migration tranche 1 — knowledge, billing/org, workflows/background - migrate 19 suites off private vi.mock('@sim/db') factories onto the shared dbChainMock + queueTableRows API (net ~-1,260 lines of bespoke chain plumbing); resolves the known shared-worker rival pairs (knowledge processing-queue vs api utils; billing polluters; persistence/utils vs schedules/deploy) - add .for() to the mock's limit builder (drizzle .limit(1).for('update')) with a contract test - document the join-table queue fallback footgun on queueTableRows	2026-07-22 14:53:17 -07:00
Waleed	c083be9def	improvement(auth): bump better-auth to 1.6.23 and add trusted-proxy client IP resolution (#5857 ) * improvement(auth): bump better-auth to 1.6.23 and add trusted-proxy client IP resolution * chore(billing): record checkout-scope mirror re-verification against @better-auth/stripe 1.6.23 * chore(deploy): expose AUTH_TRUSTED_PROXIES in docker-compose.prod and Helm chart	2026-07-22 14:41:02 -07:00
Waleed	3215a12da9	improvement(testing): consolidate @sim/db mocks into one table-aware chain mock (#5856 ) * improvement(testing): consolidate @sim/db mocks into one table-aware chain mock - back databaseMock and dbChainMock with the SAME db instance so a module bound to either export hits identical chain fns — rival-mock divergence between the two @sim/testing db mocks is structurally impossible now - add queueTableRows(table, rows): FIFO per-table select routing keyed by schema-mock table identity, consumed at where() materialization and resolved by every downstream terminal (limit/orderBy/groupBy/for/joins) - delete createMockDb (duplicate chain implementation, no external users) - migrate the five suites that hand-rolled table routing + databaseMock delegation (billing plan/usage/usage-log, admin dashboard-organizations, workspaces/utils) onto queueTableRows; net -295 lines - add a contract test for the mock itself and a test script to @sim/testing so its tests actually run under turbo * fix(testing): harden table routing — join-table queues, direct-await from, mutation isolation - track the chain's tables as a list (from + joins) so rows queued for a join-only table route correctly; from-table queue checked first - make the from/join builder a lazy thenable so awaiting a select with no where clause resolves queued rows (dequeue at await, never double-consumed) - update/delete/set clear the routing context so a mutation's where() can never consume rows queued for a select - document the left-to-right chain-construction assumption; contract tests for all three behaviors * fix(testing): close routing over each chain's own tables for direct-await builders * refactor(testing): move all chain routing state into per-chain closures - shared dbChainMockFns entries become pure spy/override ports: their default implementation returns a sentinel that chain-local builders replace, while any mock* override on the spy wins verbatim - each select().from() captures its own immutable table list; where(), joins, terminals, and direct awaits all resolve through that closure, so partially-built chains for different tables interleave without cross-talk - no module-level routing state remains * fix(testing): lazy queue consumption at resolution and wrapper restore on reset - each chain holds one lazy rows supplier: the queued set is dequeued only when a default thenable actually resolves, so a chain answered by a per-test terminal override leaves its queued rows for the next chain - resetDbChainMock also mockReset()s the stable db entry-point wrappers so direct overrides on databaseMock.db.* cannot outlive a suite	2026-07-22 14:15:44 -07:00
Waleed	fad9728cbd	improvement(tests): make vitest suites state-safe with auto-unstub of env/global stubs (#5853 ) * improvement(tests): make vitest suites state-safe with auto-unstub of env/global stubs - add unstubEnvs/unstubGlobals to vitest config so vi.stubEnv/vi.stubGlobal are restored after every test - move module-scope fetch/crypto/window stubs into beforeEach across executor, data-drains, and knowledge suites so they hold under auto-unstub (several suites were silently hitting live Datadog/BigQuery/Snowflake/OpenAI/Anthropic endpoints when their module-scope stubs were cleared) - converge billing/workspaces/tools/hooks suites onto the shared @sim/testing mock instances and namespace spies instead of rival file-local vi.mock factories, with explicit beforeEach setup and afterAll restore - give lib/core suites private module instances via vite query-suffix imports where module-level state bakes env at import time - stub auth-client/NEXT_PUBLIC_APP_URL in suites that crashed at collection without a local .env Groundwork for eventually running the suite with isolate: false; no CI behavior change (isolation stays on). * fix(tests): complete databaseMock restore and route selectDistinctOn - restoreGlobalDb now mockReset()s entries whose original implementation was undefined instead of leaving the chain-mock delegation installed - add selectDistinctOn to the setup-level createMockDb and to the dashboard suite's delegated key set so the distinct-on path routes under either binding * fix(tests): run knowledge utils cases sequentially over shared per-test state	2026-07-22 13:15:15 -07:00
Theodore LiandClaude Fable 5	70814bc4a2	feat(db): role-keyed dbFor clients for cleanup and exec workloads (#5583 ) * feat(db): role-keyed dbFor clients for cleanup and exec workloads Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NNy9Fzpfc1FdHAzYyb6Ycy * fix(db): keep cleanup-invoked helpers and snapshot reads on their role pools Route markLargeValuesDeleted / pruneLargeValueMetadata (optional dbClient) and chat-cleanup's file collection through the cleanup pool, and getSnapshot through the exec pool, so the cleanup and inline-execution workloads stop borrowing the process-wide pool for these queries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NNy9Fzpfc1FdHAzYyb6Ycy * fix(db): dbFor falls back to the process-role URL, not the base URL With DATABASE_URL_WEB/TRIGGER set (as in prod) and the sub-pool URLs unset, falling back to the base URL would silently shift execution-log and cleanup traffic to a different PgBouncer endpoint on deploy. Chain the fallback through the URL the process itself resolved so the rollout stays inert. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NNy9Fzpfc1FdHAzYyb6Ycy * feat(db): log which connection each dbFor sub-pool resolved to One line per role at first use: the dedicated DATABASE_URL_<ROLE> when set, otherwise an explicit fallback message naming the process connection it shares — so a missing/typo'd env var is visible at rollout instead of silent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NNy9Fzpfc1FdHAzYyb6Ycy * feat(db): route pause/resume and large-value metadata persistence to the exec pool Coverage scan follow-up: the paused_executions / resume_queue / workflow_execution_logs transactions in human-in-the-loop-manager.ts and the large-value owner/reference registration writes on the execution path now use dbFor('exec'), matching the completion writes in the execution logger. All are self-contained; billing calls remain outside the moved transactions on the default client. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NNy9Fzpfc1FdHAzYyb6Ycy --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-22 14:35:42 -04:00
Waleed	93fbf584a2	feat(chat): soft-delete sidebar chats with restore from Recently Deleted (#5830 ) * feat(chat): soft-delete sidebar chats with restore from Recently Deleted * fix(chat): review round 1 — restore workspace authz, purge recheck, archived-list invalidation * test(chat): update SSE handler assertions for workspaceLists invalidation * fix(chat): bump updatedAt on restore, recheck retention cutoff in task cleanup * fix(chat): drop explicit feedback delete in task cleanup — chat FK cascade covers it * test(chat): cover restore route; guard legacy copilot delete from hard-deleting mothership chats * chore: revert unintended bun.lock drift from worktree install * fix(cleanup): recheck workflow archive cutoff on delete; export deletedAt in chat drain	2026-07-21 20:10:13 -07:00
Waleed	a360cd4d7c	fix(workflow): align condition/router handles with their rows (#5831 )	2026-07-21 18:00:58 -07:00
Vikhyath Mondreti	37628f1566	fix(workflow-persistence): make persistMigratedBlocks' updated_at guard millisecond-tolerant (#5790 ) * fix(workflow-persistence): make persistMigratedBlocks' updated_at guard millisecond-tolerant * improvement(sockets): state improvements * address comments	2026-07-21 14:36:11 -07:00
Waleed	94b23344e6	improvement(emcn): consistent hugging tab bars via ChipModalTabs (#5815 ) - Pin ChipModalTabs to w-fit so the segmented pill always hugs its tabs instead of stretching full-width when dropped into a flex column - Migrate the mothership settings tab bar off a hand-rolled full-width underline bar onto ChipModalTabs - Fix undefined --border-secondary token (fell back to currentColor) in mothership -> --border, matching the admin settings precedent	2026-07-21 13:28:42 -07:00
Theodore Li	2f31981fbf	improvement(blocks): two-tier sunset schema (legacy amber / deprecated red) (#5793 ) * improvement(blocks): two-tier sunset schema (legacy amber / deprecated red) * fix(blocks): require replacedBy on legacy sunset blocks in registry check * improvement(blocks): reword legacy badge tooltip	2026-07-20 23:17:59 -04:00
Theodore Li	53c2716391	feat(blocks): surface deprecated block and model warnings on canvas (#5785 ) * feat(blocks): surface deprecated block and model warnings on canvas * improvement(blocks): simplify deprecation derivation, drop unused getModelReplacement * fix(blocks): make deprecation badge keyboard-accessible	2026-07-20 20:22:22 -04:00
Waleed	ae6a363289	improvement(ux): submit on Enter in chip modals and advance table rows (#5781 ) * improvement(ux): submit on Enter in chip modals and advance table rows * fix(emcn): stop Enter double-submit and close registration gap - stopPropagation on field Enter-submit so a parent onKeyDown can't re-fire the primary (double OAuth connect, etc.) - register primary via useLayoutEffect so it's set before paint (no null-Enter window) - drop now-redundant parent Enter handlers in connect-oauth/create-workspace/rename-document modals * fix(table): suppress auto-opened tag dropdown when Enter advances cells Focusing an empty cell auto-opens the tag dropdown; without closing it a follow-up Enter inserts a tag instead of navigating down the column. * fix(table): clean up cell refs on unmount and guard Enter advance - delete inputRefs/overlayRefs entries when a cell detaches so deleting a row can't leave stale position-keyed entries - guard Enter advance with isConnected so a stale ref can never steal focus into a detached node	2026-07-20 16:38:56 -07:00
Theodore Li	61de0b5808	feat(pii): custom user-supplied regex patterns for redaction (#5732 ) * feat(pii): custom user-supplied regex patterns for redaction * fix(pii): enforce custom-regex syntax + safety at the boundary schema * improvement(pii): always wrap custom-pattern redaction token in angle brackets * chore(pii): register guardrails_validate in the dev minimal tool registry * fix(pii): coerce empty guardrails entity-type checkbox (null) so the contract accepts it * fix(pii): keep detect-all when a custom pattern is added; custom patterns win overlaps	2026-07-17 15:20:58 -04:00
Vikhyath Mondreti	86e6e1d26c	feat(forking): excluded workflows (#5727 ) * feat(forking): excluded workflows * improve sync preview	2026-07-16 20:23:26 -07:00
Vikhyath Mondreti	442eabaf27	improvement(checkout): enforce team/enterprise-only org subscription, block double-covered personal checkouts (#5715 ) * improvement(checkout): enforce team/enterprise-only org subscriptions, block double-covered personal checkouts, and drop renewal-triggered workspace detach * close race with personal pro / org inclusions * address comments * fix(billing): compute org coverage independently of the personal-sub lookup and fail closed on unverifiable plan writes	2026-07-16 19:16:44 -07:00
Siddharth Ganesan Claude Opus 4.8 Emir Karabeg Justin Blumencranz Vikhyath Mondreti	f03b4337fa	feat(mothership): mixture of models, search agent, persistent subagents, fork chat, inline questions (mothership v0.8) (#5410 ) * feat(scout): add scout agent * fix(contracts): update contracts to include scout agent * feat(copilot): search agent (research+scout merge) + read-only table/KB tool handlers Mirrors mothership dev f90f9b05: - regenerated tool-catalog/tool-schemas mirrors (search trigger replaces research + scout; QueryUserTable / SearchKnowledgeBase entries) - queryUserTableServerTool / searchKnowledgeBaseServerTool: read-only wrappers delegating to the full user_table / knowledge_base handlers with hard operation allowlists (and outputPath export rejection on query_user_table) - display maps: 'search' agent label/title/icon added; research + scout entries retained so historical transcripts keep rendering - Search.id replaces Research.id in LONG_RUNNING_TOOL_IDS (it inherits research's long crawls) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(copilot): run_code compute-only handler; docs lint fix Mirrors mothership dev db60da94: run_code is the compute-only variant of function_execute for the search agent — same sandbox and inputs, no outputs.files / outputTable, so it cannot create or overwrite workspace resources. Wrapper handler hard-rejects the write vectors and delegates to executeFunctionExecute; run_code is deliberately absent from OUTPUT_PATH_TOOLS and the table output post-processor, so the name gating blocks writes even for leaked args. Added to LONG_RUNNING_TOOL_IDS, display title/icon maps, and the regenerated catalog/schema mirrors. Also removes two ineffective biome suppression comments in the docs workflow-preview (the rule doesn't fire in the docs app config). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(copilot): failed tool calls must surface their error in terminal data A failed handler result that carried a defined-but-empty output (the app-tool executor's 'Tool not found' ships output: {}) won the priority race in getToolCallTerminalData, so the resume payload's data — the only thing the model reads — was a bare {} with the error text dropped. The search agent retried run_code 20+ times blind against a stale server because every failure rendered as empty instead of 'Tool not found'. Failed calls now always carry error in their terminal data: merged into object outputs, wrapped alongside non-object outputs, preserved when the output already has an error field. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(chat): render inline question tags from the agent in chat * fix(chat): let inert multi-step questions browse all prompts * improvement(chat): guard question answer formatting against sparse arrays * chore(copilot): drop user_memory from generated contracts and tool display Companion to mothership 8ae32e97 (user_memory tool removed — the feature no longer exists). Regenerates the mothership contract mirrors via generate-mship-contracts.ts, which also picks up the pending telemetry contract additions (gen_ai.agent.name labels, llm.client.context_tokens, llm.client.compactions, llm.request.compaction_trigger, llm.compaction.pause, gen_ai.usage.context_tokens), and removes the user_memory display title. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * improvement(chat): answered question card becomes the user turn; two select types only UI ordering: answering a question card no longer echoes a duplicate user bubble. The combined answer still goes on the wire as a user message, but the chat pairs it back to its card (strict 'Prompt — Answer' match, now uniform for single questions too) and renders the card as the answered recap — the card IS the user turn, and the next assistant message streams below it. The pairing is derived from the transcript, so live and reloaded renders are identical; a dismissed card followed by an unrelated typed message does not match and renders normally. Messages ending with a question card also drop the copy/thumbs actions row — the card is an input surface, not a reactable assistant turn. Question types are now single_select and multi_select only: text is removed (the free-text 'Something else' row covers it) and confirm collapses into single_select with Yes/No options. multi_select rows toggle with a check and the free-text row's arrow submits the step; answers are comma-joined labels plus any typed entry. Agent-supplied catch-all options ('Other', 'Something else', 'None of the above') are stripped at parse — the card always provides its own free-text row; a question left with no real options is invalid. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * improvement(chat): question cards are single_select only Removes multi_select (and its toggle/check UI). The card is one shape: pick one option or type into the always-present 'Something else' row. Catch-all stripping and the transcript pairing/recap behavior are unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * improvement(chat): bring back multi_select question cards Re-adds multi_select with a reworked interaction: option rows carry real checkboxes (emcn Checkbox chrome) instead of numbers and arrows, an option-styled Submit row confirms the step, and the "Something else" row reads as a plain option until clicked — then it becomes the focused text box, auto-checks, and can be unchecked without losing the typed text (blur with nothing typed reverts it). single_select behavior, catch-all stripping, and the transcript pairing/recap format are unchanged; multi_select answers are the checked labels comma-joined. * chore(copilot): regenerate mothership contract mirror (chat blob span attrs) * chore(copilot): regenerate mothership contract mirror (chat blob metrics) * feat(secrets): make output of generate api key a secret * feat(cli): add mkdir, mv, cp to mship tool set * feat(fork-chat): add fork chat to mothership * fix(fork-chat): fix messageid handling in fork chat * feat(credentials): agent-initiated oauth credential reconnect (#5488) * feat(credentials): agent-initiated oauth credential reconnect * fix(credentials): address reconnect review findings * improvement(credentials): log when connect draft name lookups degrade * fix(conflicts): remove migration * fix(conflicts): fix conflicts * fix(fork-chat): add migrations back * fix(ci): fix lint * fix(ci): fix bad import * fix(vfs): fix 500 char limit in vfs for skills and custom tools * feat(copilot): gate user skills to explicit slash-attach (#5536) Stop the mothership from adopting a workspace user-skill on its own: - Remove the load_user_skill tool and its three payload callers (chat payload, mothership execute route, inbox executor); delete lib/mothership/skills.ts + its test. Skills no longer autoload as the agent's own instructions. - Rename the workspace "## Skills" inventory to "## Agent Block Skills — NOT FOR YOU" with a one-line guardrail so a skill's description (e.g. "respond like a pirate") is not treated as an instruction. Skills reach the model as behavior only via explicit /-attach. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(lots-of-things): lots of things * feat(subagents): add persistent subagents * fix(copilot): let edit_workflow set knowledge-base tag filters, and stop it clearing them (#5546) * fix(copilot): persist KB tag subblocks as JSON strings from edit_workflow The edit_workflow tool normalizes array-with-id subblocks (via normalizeArrayWithIds) but only re-stringifies the keys listed in JSON_STRING_SUBBLOCK_KEYS. `tagFilters` (knowledge-tag-filters) and `documentTags` (document-tag-entry) were missing, so agent-authored tag filters were stored as raw JSON arrays while those UI components read their value with JSON.parse (expecting a string). The result: an agent edit to a Knowledge block's tag filter persisted correctly but rendered as an empty filter in the editor (JSON.parse on an array throws -> []). - Add `tagFilters` and `documentTags` to JSON_STRING_SUBBLOCK_KEYS so edit_workflow stores them in the same shape the UI writes. - Make both components' parsers tolerate an already-parsed array on read, self-healing values already persisted in the broken (array) shape. Search execution was unaffected (parseTagFilters accepts arrays), so the value was never lost — only the editor render and round-trip were broken. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(copilot): expose KB tag definitions in VFS meta.json Surface each knowledge base's defined tags (displayName -> tagSlot) inline in its meta.json via serializeKBMeta, loaded in one batched query (loadKbTagDefinitions), so the agent can bind a knowledge-tag filter to a real tag slot instead of guessing a tag name it cannot otherwise see. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(copilot): stringify KB tag subblocks on the nested-node edit path The nested-node merge path normalized array-with-id subblocks but never re-serialized the JSON_STRING_SUBBLOCK_KEYS, so editing a block nested in a loop/parallel container still persisted tagFilters/documentTags (and conditions/routes) as raw arrays -- the exact shape the subblock components cannot JSON.parse. Route all four write paths through a single normalizeSubblockValue helper so the normalize and re-stringify steps cannot drift apart again, and extract the duplicated string-or-array read logic into parseJsonArrayValue. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(copilot): tighten subblock serialization helpers Derive KbTagDefinitionSummary from the canonical TagDefinition instead of restating its fields, make parseJsonArrayValue generic so callers drop their `as T[]` casts, and unexport the three builders helpers that no longer have consumers outside the module now that normalizeSubblockValue fronts them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(copilot): stop stripping tagFilters/documentTags from the agent's workflow view sanitizeForCopilot dropped `tagFilters` and `documentTags` from the workflow state the agent reads (workflows/{name}/state.json), while edit_workflow is allowed to write both. The field was therefore write-only: on a follow-up edit the agent read back an absent field, concluded no filter was set, and cleared the user's tag filter. The redaction was introduced for workflow export (#1628) and is already enforced there by sanitizeWorkflowForSharing's key list. The duplicate in the copilot-only sanitizeSubBlocks was redundant for export and destructive for the agent. Removes it and pins the contract with a regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(copilot): reject malformed KB tag values instead of clearing the filter `knowledge-tag-filters` and `document-tag-entry` had no arm in the `edit_workflow` input validator, so they fell through to the pass-through default. Any non-array value the agent supplied -- a double-encoded JSON string, an object, an unparseable string -- reached `normalizeSubblockValue`, where `normalizeArrayWithIds` coerces unparseable input to `[]`. The write path then persisted `"[]"` over the tag filter the user had configured. `condition-input` and `router-input` already guard against exactly this and return an actionable error to the model. Extend that arm to cover the two KB subblock types. It keys on subblock type, so the unrelated `tagFilters` short-input on the Algolia block is unaffected. `null`/`undefined` and empty arrays still clear the field, so intentional clears keep working. Also wrap `loadKbTagDefinitions` in try/catch. Tag definitions are an optional meta.json enrichment, but the query ran inside the top-level `Promise.all`, so a transient failure would reject the entire workspace VFS materialize and leave the agent unable to read any file. Now it degrades to a meta.json without tag definitions, matching the sibling materializers. Adds regression tests for both, plus the first tests for `parseJsonArrayValue`, the helper that keeps pre-fix raw-array rows readable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(copilot): collapse duplicate JSON-array parsing in edit-workflow builders `normalizeArrayWithIds` and `normalizeConditionRouterIds` each hand-rolled the same "accept a raw array or the JSON string these subblocks persist" parse. Extract `parseJsonArray`, which returns null when the value is neither, so each caller keeps its own distinct fallback: `[]` for the former, the untouched original value for the latter. Behavior-preserving. An empty array is truthy, so `[]` and `"[]"` still parse through rather than hitting either fallback. `validation.ts` has a third copy, but `builders.ts` already imports from it, so sharing the helper across the two would introduce an import cycle. Left as is. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(copilot): specify tag name and legal operators in KB meta.json `tagDefinitions` exposed `displayName`, but a `tagFilters` entry must carry the key `tagName`. An entry written with `displayName` passes validation and persists, then filters nothing -- a silent failure. Rename the field at the serializer boundary; the DB column is untouched. Also emit the operators legal for each tag's `fieldType`, reusing `getOperatorsForFieldType`. `between` is valid for number and date but not for text or boolean, and the agent has no way to infer that. An unrecognized fieldType yields an empty list rather than throwing. Still unspecified, and deliberately out of scope: a filter entry's value key is `tagValue` (but `value` on documentTags), and `between` needs `valueTo`. Those describe the subblock entry shape, not the knowledge base, so meta.json is the wrong place for them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(copilot): pass a nullish subblock clear through instead of serializing "[]" `validateValueForSubBlockType` accepts null as an explicit clear, but `normalizeSubblockValue` then ran it through `normalizeArrayWithIds`, which coerces any non-array to `[]`, and persisted the string "[]". No data is lost either way -- "[]" and an absent field both mean "no filters". But it left the field present when the caller asked for it to be unset, so `sanitizeForCopilot` showed the agent an empty filter rather than an absent one, contradicting the absent-means-unset invariant the sanitizer documents. It also made Algolia's `if (params.tagFilters)` see a set value, since "[]" is truthy. An explicitly empty array still serializes to "[]" -- clearing with a value is distinct from clearing by omission. Reported by Cursor Bugbot on #5546. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(changes): huge changes * fix(subagents): lanes * fix(mship): transcript stuff * fix(subagents): thinking lanes * fix(superagent): fix superagent tools and checkpoints * fix(scope): scope subagent tools * fix(lint): fix lint * chore(db): regenerate workspace_files.message_id migration as 0260 on staging base * fix(superagent): fix superagent integration tools * improvement(questions): make something else a placeholder * chore(copilot): regenerate mothership contract mirror after staging rebase * feat(mship): add external mcps to mship * fix(ci): fix dev build * fix(stream): show thinking text * fix(ci): force redeploy * fix(mothership): keep chat forks outside workspace storage billing Preserve the product invariant that Mothership chat files are not charged as workspace file storage after the billing storage merge. * fix(uploads): restore listWorkspaceFiles throwOnError option dropped in rebase * fix(subagent-streaming): remove italics * fix(mothership): treat subagent lanes closed by subagent_end as settled so the between-steps thinking indicator isn't suppressed * fix(ui): thinking loader and rool names * fix(ui): add file * fix(thinking): show thinking during subagents * fix(chat): drop dead thinking-channel ternary after lanes skip thinking blocks * fix(thinking): remove thinking text * improvement(function execute): add timeout to function execute and stop showing text in subagents * fix(subagents): hide thinking text * fix(ff): move ff to go * improvement(superagent): nuke superagent * feat(main-agent): superagent into main agent * chore(db): regenerate workspace_files.message_id migration as 0262 on staging base * fix(credentials): restore reconnect params on shared createConnectDraft * fix(migrations): rebase with staging --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu> Co-authored-by: Justin Blumencranz <96924014+j15z@users.noreply.github.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>	2026-07-16 11:24:46 -07:00
Waleed	58e4b754f9	fix(emcn): remove brand-accent focus ring from TagInput default variant (#5696 )	2026-07-15 18:43:25 -07:00
Vikhyath Mondreti	54b35a4f0e	improvement(deployments): bugfixes for run-block, airtable + external sub management (#5680 ) * improvement(webhooks): external subscription management * ui/ux * remove test file * fix tests * address comments * address comments * update to grain v2 api * improvement(grain): hide auto-registered webhook URL on v2 triggers * Revert "improvement(grain): hide auto-registered webhook URL on v2 triggers" This reverts commit `c89660cc3e`. * address comments * address comments * address rollback * fix grain v2 * fix more comments	2026-07-15 18:01:53 -07:00
Waleed	3ac5e90853	fix(emcn): stop wiping virtualized code viewer row measurements on content changes (#5649 )	2026-07-13 16:19:32 -07:00
Theodore Li	de934b9979	feat(script): migrate agent/router/evaluator LLM keys to BYOK by model prefix (#5607 ) * feat(byok): migrate agent/router/evaluator LLM keys to BYOK by model prefix * fix(script): address review — cover translate/guardrails, trim provider ids, reject mixed mapping, add key-prefix guard * fix(script): include pi block in LLM-family set, trim key-prefix value * fix(script): add router_v2 to LLM-family set, canonicalize provider ids to lowercase * fix(script): resolve overlapping model prefixes deterministically (longest match wins)	2026-07-13 18:28:12 -04:00
Vikhyath Mondreti	2d41360807	improvement(concurrency): limits configurable, docs updates (#5640 ) * improvement(concurrency): limits configurable, docs updates * remove dead tests * limits self hosted vars	2026-07-13 13:11:05 -07:00
Vikhyath Mondreti	ef7c8e24b2	feat(platform): settings permissions, admin, billing attribution (#5545 ) * fix(invites): preserve active organization for external access Keep organization activation server-owned so failed membership checks cannot clear a valid session context. * feat(admin, billing, settings): cleanup settings visibility, billing actor resolution, new admin routes * address comments * chore(db): reset pending migrations before staging merge Remove locally generated migrations so they can be regenerated against the latest staging schema without preserving stale snapshots or numbering. * regen migrations * address comments * chore(db): reset generated migrations before staging merge Remove this branch's generated migrations so they can be regenerated against the latest staging schema with fresh numbering. * upgrade global work * fix lint * address comments * legacy callbacks correctness * address comments * update * guardrail attribution	2026-07-13 00:53:49 -07:00
Theodore Li	0cc6ed61d3	improvement(emcn): multi-select selectors + Wizard on ChipModal (#5614 ) * feat(sub-block): support multi-select in channel/user selector fields * improvement(emcn): migrate Wizard primitive to ChipModal * fix(emcn): wizard height sizes whole dialog; restore dialog description	2026-07-11 22:25:41 -04:00
Waleed	9ee499e9f7	fix(canvas): raw tooltip shows up when hovering block params (#5589 ) * fix(canvas): replace native title tooltip with styled overflow tooltip on block params Hovering a truncated subblock value or block name on the canvas popped the browser's raw native tooltip with the full untruncated content (including raw code). Replace the `title` attribute with the cursor-following styled Tooltip, shown only when the text is actually clipped. * fix(canvas): drop unused ResizeObserver in OverflowSpan useFloatingTooltip's canShow already receives the hovered element, so measuring overflow via useIsOverflowing was redundant — every canvas row was paying for a ResizeObserver and resize listener it never used.	2026-07-10 21:58:08 -07:00
Theodore Li	7962236719	improvement(custom-blocks): hardened delete with usage count + per-input required option (#5575 ) * improvement(custom-blocks): usage visibility + type-to-confirm delete * feat(custom-blocks): per-input required option * improvement(custom-blocks): replace usage tab with delete-confirmation usage count * fix(custom-blocks): escape LIKE wildcards in usage scan + fresh count on delete modal * fix(custom-blocks): explicit ESCAPE clause on usage-scan LIKE prefilter	2026-07-10 20:16:53 -04:00
Theodore LiandClaude Opus 4.8	f4d47ed826	feat(slack): reusable custom bot credentials, slack_v2 block (preview), redesigned trigger (#5323 ) * feat(slack): enable assistant-agent tools via assistant:write scope Add assistant:write, app_mentions:read, and im:history to the Slack bot OAuth scopes so the Set Assistant Status / Title / Suggested Prompts tools (assistant.threads.) work with users' existing Slack credentials — no new app or credentials required. Restore the action_assistant trigger capability (scope assistant:write) in the manifest generator. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WpeT8J5yVCrrNQB9Hzm9uS Add slack trigger * fix channel picker in slack trigger * improvement(slack-trigger): reorder app type, gate account to sim mode, add channel-id input * fix(slack-trigger): drop unmapped events from filter, resolve oauth token for reaction text + file downloads * fix(slack-trigger): empty operation selection fires nothing; resolve token via credential owner not execution actor * fix(slack-trigger): ignore message edit/delete/system subtypes; prefer channel picker over stale manual ids * feat(slack-trigger): single-event model with contextual filters and full event catalog * fix(slack-trigger): apply event/channel/bot filters on custom-app path too * fix(slack-trigger): don't drop edit/delete events when channel_type is absent * feat(slack): reusable custom bot credentials, slack_v2 block, interactivity triggers - Custom bot as a workspace service-account credential (set up once, shared ingest URL /api/webhooks/slack/custom/{credentialId}, reused across triggers and actions) - slack_v2 action block: credential-based Custom Bot auth alongside Sim OAuth; v1 hidden from toolbar - Interaction triggers (block_actions / view_submission) with optional action/callback id filter; settings.interactivity in generated manifests - Setup wizard: name + description, full permissions by default with ChipDropdown customization; reconnect mode rotates secrets in place - Centralized service-account token resolution (unknown provider fails loudly) - Shared Slack webhook fan-out dispatcher for native + custom ingest routes * chore(api-validation): bump route baseline to 924 after staging merge * feat(slack): preview-gate slack_v2 and the custom-bot credential surfaces slack_v2 (block + hosted slack_oauth trigger) ships preview: true — hidden from all discovery until revealed via block-visibility AppConfig or PREVIEW_BLOCKS. v1 stays toolbar-visible with the legacy slack_webhook trigger until v2 GAs. The integrations-page custom-bot setup surface rides the same flag via isHiddenUnder(slack_v2); placed instances, existing credentials, and ingest/execution paths are never gated. * fix(slack): v1 keeps slack_webhook trigger subblocks; handle object-form event channels - v1 spread had been swapped to slack_oauth's trigger subblocks (shared with v2), leaving its slack_webhook deploy path without signing-secret config (Bugbot high). v1 now carries the legacy trigger set again; v2 swaps them for slack_oauth's. - resolveSlackEventChannel reads channel.id for channel_created/channel_rename payloads, so channel filters no longer drop every rename event. * fix(slack): default absent appType to custom at deploy; deactivate custom-bot webhooks on credential delete - appType is hidden and seeded 'custom' by value(), which only covers editor-created blocks; defaultValue now persists it via buildProviderConfig and the deploy fallback flips to custom (the only exposed mode this ship) - deleting a slack-custom-bot credential now also deactivates provider='slack' webhooks routed by that credential id, not just native slack_app rows * fix(slack): resolve credential owner for deploy-time team_id lookup A teammate deploying a trigger wired to a shared Slack credential isn't the credential owner; refreshAccessTokenIfNeeded only loads tokens for the owning user. Resolve the account owner first, mirroring the runtime formatInput path. * chore(slack): reconcile staging merge - nullable webhook.path coalesced at correlation/payload/tiktok boundaries - slack dispatch delegates to staging's dispatchResolvedWebhookTarget (shared preprocess/deployment/filter/enqueue lifecycle), keeping the skip-reason diagnostics; route tests reworked around that seam - api-validation route baseline 924 -> 926 * fix(slack): workspace-scope bot credentials at deploy; recreate webhooks on routing transitions - a bot credential id is semi-public (embedded in Slack Request URLs), so the custom deploy branch now rejects credentials outside the workflow's workspace - needsRecreation also compares path/routingKey, so a row from an older routing model can't survive redeploy as a stale delivery surface * test(slack): pin fail-closed behavior for empty/missing event selection * fix(slack): 409 on custom-bot name collision instead of silently returning the existing credential The service-account dedupe matches on displayName, which defaults to the Slack team name — shared by every bot in that workspace. A second unnamed bot create returned the first credential as success, orphaning the new id already pasted into the Slack Request URL. Same-id replays stay idempotent; different-id collisions now fail loudly so the wizard prompts for a distinct name. * fix(slack): reconnect surfaces Atlassian error codes and persists name/description edits - PUT credential route now returns the Atlassian provider code (providerErrorCode -> code) so reconnect failures map to specific token/domain messages, matching create - Google/Atlassian reconnect send + seed displayName/description (parity with Slack); edits are no longer silently discarded, and empty fields don't clobber existing values * fix(slack): require bot name; propagate rotated bot_user_id to webhooks on reconnect - the setup wizard now requires a bot name (canAdvance), so the credential name, manifest app name, and uniqueness key all use the user's choice instead of the shared Slack team-name fallback that collided for a second bot in one workspace - reconnect that changes the bot user id (recreated Slack app) now updates the bot_user_id cached in each bound webhook's providerConfig, so reaction self-drop keeps working instead of letting the bot's own reactions re-enter --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-10 19:43:52 -04:00
Waleed	fac8ec0c51	fix(workflow-edges): enforce edge/block validation server-side, not just client-side (#5571 ) * fix(workflow-edges): enforce edge/block validation server-side, not just client-side Dragging a connection that creates a cycle correctly refused to render client-side, but the cyclic edge was still queued for realtime persistence and written to the DB, so it reappeared after refresh. Root cause: cycle detection (and several other edge/block rules) only lived in the client Zustand store and was never enforced by the realtime persistence layer, which is the actual source of truth on reload. - Move wouldCreateCycle, edge scope-boundary, annotation-only-block, duplicate-edge, and block-name-conflict checks into @sim/workflow-types so the client store, the collaborative queueing layer, and apps/realtime's DB write path all share one implementation - Wire these into apps/realtime/database/operations.ts's edge-add and block-rename handlers as the authoritative gate - Client-side behavior is unchanged (same call sites, same error messages, same rule ordering) — verified via existing + new test coverage * fix(workflow-edges): address review findings on realtime edge validation - Select triggerMode when fetching blocks for edge-add validation — isKnownWorkflowTriggerBlock checked block.triggerMode but the column was never fetched from the DB, so trigger-mode blocks could still receive an incoming edge (Cursor Bugbot) - Make filterUniqueWorkflowEdges incremental, so two duplicate edges within the same BATCH_ADD_EDGES payload are also deduped instead of both surviving (Greptile) * fix(workflow-edges): normalize empty-string handles in duplicate-edge check filterUniqueWorkflowEdges compared handles with ??, so a `sourceHandle: ''` edge wasn't recognized as a duplicate of an existing null-handle edge — even though both get persisted as the same null value at insert time (edge.sourceHandle \|\| null). Falsy-coalesce in the comparison so '' and null/undefined are treated as the same "no handle" state everywhere. (Greptile) * improvement(workflow-edges): dedup realtime edge-add validation, reuse block-name-conflict helper /simplify pass on the already-merged-quality PR before sign-off: - Extract filterEdgesForPersist in apps/realtime/src/database/operations.ts: the single-edge ADD and batch BATCH_ADD_EDGES handlers hand-inlined the same six-step validation pipeline (missing block, protected target, annotation-only, trigger-target, scope boundary, duplicate, cycle) and each independently re-fetched blocksById/existingEdgesForCycleCheck. Two copies of one rule in the same file was exactly the drift risk this PR otherwise closes across client/server. One shared helper now backs both. Net -63 lines despite the new shared function. - Fix a real bug this surfaced: droppedCounts was keyed by the free-text, block-id-bearing scope-boundary message, so it could never aggregate across edges/runs. Now keyed by a stable 'scope boundary' reason. - use-collaborative-workflow.ts's collaborativeUpdateBlockName still hand-rolled the empty/reserved/duplicate block-name-conflict checks this PR centralized as getWorkflowBlockNameConflict (already adopted by store.ts). Switched it to the shared helper, which also fixes a latent check-order mismatch between the two (this pre-check ran reserved before duplicate; the store's real gate — after this PR's own store.ts change — runs duplicate before reserved, so they could disagree on which toast a name that was both reserved and duplicate would surface). * docs(workflow-edges): document the per-workflow write-serialization invariant No behavior change. Address Greptile P1 on filterEdgesForPersist ('concurrent duplicate writes can persist without a per-workflow write guard') by documenting, at the actual mechanism, why the concern doesn't apply here: persistWorkflowOperation's leading 'UPDATE workflow SET updatedAt ... WHERE id = workflowId' already takes a row lock that serializes every operation (including edge adds) for a given workflowId — a second concurrent call blocks on that UPDATE until the first transaction commits or rolls back, so the validate-then-insert sequence in filterEdgesForPersist can never interleave across two writers on the same workflow. Verified empirically, not just by reading: ran two concurrent transactions against a throwaway local Postgres against the exact statement shape used here (UPDATE the parent row, sleep to simulate the read/validate window, insert, commit). The second transaction's UPDATE blocked for the full duration of the first's transaction and only proceeded once the first committed — confirming the row lock, not any application-level guard, already provides the serialization Greptile flagged as missing. Added a comment at the lock site (not a second, redundant advisory lock) so a future change can't silently break this invariant by making the UPDATE conditional/skippable as a perceived no-op optimization. * fix(workflow-edges): validate edges in BATCH_ADD_BLOCKS before persisting Real gap Cursor's PR summary flagged ('BATCH_ADD_BLOCKS edge inserts... not fully covered by the new server pipeline'), confirmed by reading the code: this handler bulk-inserted the edges from a block-paste/duplicate/import payload directly into workflowEdges with zero validation — no missing-block, protected-target, annotation-only, trigger-target, scope-boundary, duplicate, or cycle check. A client sending edges through this operation instead of BATCH_ADD_EDGES could bypass every rule this PR otherwise enforces server-side, exactly the class of gap the PR exists to close. Routes it through the same filterEdgesForPersist used by the other two edge-add handlers. Runs after the block insert in this same handler, so the shared helper's blocksById lookup also sees the blocks this same batch just inserted (a transaction observes its own prior writes).	2026-07-10 15:25:25 -07:00
Bill Leoutsakos Cursor Bill Leoutsakos Vikhyath Mondreti	5d3809a0e3	feat(tiktok): add tiktok trigger, block (#5504 ) * feat(tiktok): add TikTok integration Adds TikTok as a full OAuth-based integration: provider registration (with TikTok's comma-separated scope and client_key requirements), 9 tools covering profile info, video listing/querying, creator info, direct video/photo posting (URL or file upload), inbox drafts, and post status polling, plus the TikTok block, icon, and generated docs. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(tiktok): add avatarFile output to Get User Info Adds a file-typed avatarFile output (sourced from the largest available avatar URL) alongside the existing string avatar fields, so the profile picture can be materialized as a UserFile and chained into file-consuming blocks (e.g. attached to an email), per PR review feedback. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(tiktok): lower upload memory cap, drop redundant avatar string outputs Cap the file-upload video buffer at 250MB instead of TikTok's 4GB ceiling — relaying that much through this server's memory per request isn't safe under concurrent load, and larger files can still go through the PULL_FROM_URL path, which never buffers on our server. Also drop the now-redundant avatarUrl/avatarUrl100/avatarLargeUrl string outputs from Get User Info in favor of the file-typed avatarFile output alone, since the feature is unreleased and the raw URL is still reachable via avatarFile.url. Cover image URLs on List/Query Videos are confirmed to be signed, expiring TikTok CDN links; left as strings (no file-output conversion path exists for fields nested inside array items) but documented the expiry behavior more clearly. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(ci): bump API validation route-count baseline for TikTok publish-video route The TikTok integration adds one new Zod-backed internal API route (app/api/tools/tiktok/publish-video), which trips the route-count ratchet in check-api-validation-contracts.ts. Bumping totalRoutes and zodRoutes from 917 to 918 (nonZodRoutes stays 0) to acknowledge the new route is properly validated. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(tiktok): drop unused avatar_url_100 from default user fields After removing the avatar string outputs, avatar_url_100 was still requested from TikTok's user info endpoint but never surfaced anywhere. Removed it from the default field list and the field descriptions, and noted that avatar_url/avatar_large_url feed the avatarFile output. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(tiktok): stop returning raw 'credential' subBlock id from tools.config.params The block's params function built a local `credential` variable from params.oauthCredential and returned it under the key `credential` in every switch case. That literal token is the raw subBlock id, which is deleted after canonical transformation into `oauthCredential` — the blocks.test.ts canonical-param-validation suite flags any params function that still references it. It was also redundant: oauthCredential is already part of the base resolved inputs, which the executor merges into the tool call before config.params overrides are applied, so the OAuth token resolution (which reads contextParams.oauthCredential) worked regardless. Removed the explicit credential plumbing, matching the convention already used by other OAuth blocks like dropbox.ts. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(tiktok): send empty JSON body on Query Creator Info POST query_creator_info had no request.body function, and formatRequestParams() only attaches a body when tool.request.body is defined at all — so despite sending Content-Type: application/json, the request went out with no body whatsoever. Added body: () => ({}), matching the convention already used by other parameterless-POST tools in this codebase (Google Vault, Supabase, Square, Gmail, etc.). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(tiktok): stop dropping valid zero values in optional numeric fields cursor, photoCoverIndex, and videoCoverTimestampMs all used a truthy check (params.x && {...}) to decide whether to include an optional numeric override, which drops a legitimate 0 (first page has no cursor issue aside, photoCoverIndex 0 is TikTok's own default cover photo, and timestamp 0 is a valid first-frame cover). Switched to explicit undefined/empty-string checks, matching the !== undefined convention the underlying tools already use. In today's resolution pipeline these fields always arrive as strings (even chained block references get stringified by the template resolver), and a non-empty string like "0" is truthy, so this wasn't actively broken end-to-end - but it was relying on that subtlety rather than being correct by construction, and was inconsistent with the tools' own undefined checks. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(tiktok): accept newline-separated video IDs in Query Videos videoIds is a long-input (multiline textarea), the same widget used for the newline-separated photoImages field on this block, but its parser only split on commas. Entering one ID per line - the natural pattern for a multiline field, and the one already used elsewhere on this block - produced a single concatenated garbage string instead of an array, so TikTok's query would fail or return nothing. Now splits on commas or newlines, and updated the placeholder/description to reflect both formats. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(tiktok): add app-level webhook ingress and triggers * fix(tiktok): only count actually queued webhook executions Co-authored-by: Cursor <cursoragent@cursor.com> * chore(tiktok): bump API validation baseline for staging merge Co-authored-by: Cursor <cursoragent@cursor.com> * cleanup code * fix type issues * misc code cleanup * remove photos and add upload for videos * move shared video output properties to types.ts so docs generation resolves them Co-authored-by: Cursor <cursoragent@cursor.com> * hide TikTok from toolbar and docs until the integration is ready to ship Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ci): ratchet API validation baseline to 924 after staging merge Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>	2026-07-10 15:08:24 -07:00
Waleed	aa5b1d5569	fix(suggested-actions): swap unaudited filled icons for EMCN outline set (#5548 ) * fix(suggested-actions): swap unaudited filled icons for EMCN outline set Suggested-action template icons on the home page mixed filled/solid icons in with the app's outline icon convention. Swapped them for consistent outline icons and removed the unused filled Card icon. - gmail.ts: Card -> ClipboardList - clickhouse.ts, sftp.ts: Trash -> TrashOutline - ssh.ts: TerminalWindow (emcn) -> SshTerminalIcon (moved to components/icons.tsx alongside the other block/brand icons) - deleted unused packages/emcn/src/icons/card.tsx - regenerated docs * fix(docs): revert jira.mdx regen regression generate-docs.ts is dropping the Configuration and generic-webhook Output tables for Jira triggers even though the trigger schemas still define those fields (caught by Greptile review). Unrelated to the icon changes in this PR, so reverting jira.mdx to its prior content rather than debugging the generator here. * fix(build): remove last Card icon consumer in playground gallery apps/sim/app/playground/page.tsx imported Card from the top-level @sim/emcn barrel for the icon showcase grid, which I missed when auditing @sim/emcn/icons consumers. Broke the production build after card.tsx was deleted. Removed the import and its gallery entry. Verified with a local `bun run build`.	2026-07-09 15:55:58 -07:00

1 2 3 4 5 ...