mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
444c415a0baa192ef71a8d2cf2e6dddf2a98fa7e
5630
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Theodore Li
|
444c415a0b |
improvement(data-retention): docs for overrides + PII redaction, fix wedged saves (#5905)
* fix(data-retention): clamp sub-day retention values so saves aren't wedged A stored value under 12 hours rounded to '0' days on load and was re-sent as 0, which the contract rejects (min 24) — blocking every save on the page, including unrelated fields. Clamp hours->days into the contract's range on read, and throw instead of emitting 0/NaN on write. * improvement(docs): rewrite data retention for workspace overrides + PII redaction - Document PII redaction (Logs / Workflow input / Block outputs stages, entity types, languages, custom regex patterns) - Replace the stale 'no per-workspace overrides' section with the retention-policies list and override inheritance - Correct log retention (also covers background job logs) and soft deletion (adds Chat conversations, KB documents) - Add PII + override screenshots, refresh the main one |
||
|
Vikhyath MondretiandSim Pi Agent
|
e6eef4a4bf |
feat(library): What Is an MCP Server? (#5919)
Co-authored-by: Sim Pi Agent <pi@sim.ai> |
||
|
Vikhyath Mondreti
|
7120cdc901 |
fix(providers): final regenerated stream must not re-call tools (empty chat answers) (#5915)
* fix(providers): stop the final regenerated stream from re-calling tools and clobbering the answer
After the silent tool loop settles, OpenAI Responses and Gemini re-issue a
streaming request purely to stream the answer as prose — but with tools still
attached and auto tool choice, a reasoning model can re-decide to call a tool
there. Streamed calls are never executed on this path, so the run ends with a
dead function call, an empty streamed answer, and the stream callback
clobbering the tool loop's settled text with '' (deployed chat rendered
{"content": ""}).
Force tool_choice 'none' / functionCallingConfig NONE on the regeneration and
keep the tool loop's settled answer whenever the stream ends without text.
* feat(openai): settled tool chips on the regenerated answer stream
The silent Responses tool loop has no live stream while tools run, so opted-in
consumers saw no tool chips at all for OpenAI. The loop now records each
executed call and prepends settled tool_call_start/end pairs (name + status
only) to the agent-events stream ahead of the regenerated answer. Runs without
a sink never see these events, so legacy output is unchanged.
* fix(providers): apply the regeneration guard fleet-wide
Audit of every silent tool loop for the same race fixed for OpenAI/Gemini
(final regenerated stream re-calls a tool that is never executed, ending with
an empty answer that clobbers the settled text):
- anthropic (both implementations): tool_choice {type:'none'} on the
regeneration (tools must stay — history carries tool_use blocks) + keep the
settled answer when the stream ends without text
- groq: was re-applying the ORIGINAL tool_choice, so forced-tool runs
re-forced the tool on the regeneration — guaranteed dead call; now 'none'
+ fallback
- deepseek, mistral, cerebras, azure-openai (legacy chat path), openrouter,
xai: 'auto' -> 'none' + fallback
- bedrock: fallback only — Bedrock's ToolChoice has no 'none' and toolConfig
is required when history carries toolUse blocks
Already guarded (no change): meta, sakana, nvidia, vllm, litellm, baseten,
together, fireworks, kimi, zai, ollama.
|
||
|
Waleed
|
bee45621ec |
refactor(skills): one definition of the skill fields across every surface (#5916)
* refactor(skills): one definition of the skill fields across every surface
The Name / Description / Content trio was implemented three times — once in the
canvas modal and once each in the create and detail pages — and had already
drifted: the name hint appeared on create, was missing entirely on detail, and
sat in a different slot in the modal; the content editor was 260px on the pages
and 200px in the modal; only the modal marked the fields required.
Extract SkillFields, the DetailSection form both full-page surfaces render, and
move the shared copy into skill-copy.ts. The modal keeps ChipModalField — that
is required inside a ChipModalBody, so it cannot share the pages' JSX — but now
reads the same placeholder, hint, and max-length constants, so the wording can
no longer diverge.
The detail page's local FieldLockTooltip moves into SkillFields and is now
available to both pages, and the dynamic rich-editor import drops from three
copies to two.
No behavioral change beyond the drift being resolved: the name hint now shows on
the detail page as it already did on create.
* refactor(skills): derive modal saving state, collapse the field message line
From the cleanup passes over this branch:
- SkillModal mirrored `mutation.isPending` into a local `saving` useState, which
.claude/rules/sim-hooks.md forbids. It also carried a latent bug: the
`finally { setSaving(false) }` ran after `onSave()` had closed the modal, so it
set state on an unmounting component, and a throw from `onSave()` would leave
the modal stuck in a saving state. Derived from the two mutations instead.
- The hint/error line under a field was written three times in SkillFields, in
three slightly different shapes. One FieldMessage helper now renders all three.
- The name hint is an authoring instruction, so it no longer shows under a field
that cannot be edited (a built-in skill, or a viewer who is not an editor).
|
||
|
|
6dcc65be89 |
feat(skills): add skill editors (#5705)
* feat(skills): permissions layer * chore(db): drop skill_member migration 0261 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0262 on latest staging Same DDL as the dropped 0261 (skill_member table, enums, indexes, skill.workspace_shared) plus the hand-written write-user backfill, renumbered after staging's 0261. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0263 after staging merge Staging claimed 0262 (strong_storm); same DDL plus the hand-written write-user backfill, renumbered on the merged snapshot chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(db): drop skill_member migration 0263 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0264 after staging merge Staging claimed 0263 (workflow_fork_sync_excluded); same DDL plus the hand-written write-user backfill, renumbered on the merged snapshot chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * make editing skills full page * fix disclaimer * edit access msg * fix lint * chore(db): drop skill_member migration 0264 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0265 after staging merge Staging claimed 0264 (fat_ikaris); same DDL plus the hand-written write-user backfill, renumbered on the merged snapshot chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix tests * simplify system * fix * fix lint * add mship skills docs * chore(db): drop skill_member migration 0265 for regeneration on latest staging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): regenerate skill_member migration as 0266 after staging merge Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deps): override zod to 4.3.6 to dedupe nested copies breaking type-check better-auth 1.6.23 and fumadocs-mdx resolve ^4.3.6 to a nested zod 4.4.3, which makes @sim/auth's inferred betterAuth types non-portable (TS2883) and split docs onto a second zod instance. Both ranges accept the repo-wide pinned 4.3.6, so a single hoisted copy satisfies everything. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix lint * feat(skills,tools): fullscreen skill create + shared custom tool editor Moves the rich-markdown and custom-tool editing surfaces out of modals and onto full-page surfaces, and collapses the duplicated chrome behind shared components. Skills - Add /skills/new, a full-page create surface mirroring the skill detail page (CredentialDetailLayout + DetailSection + unsaved-changes guard). "Add to Sim" navigates there instead of opening a modal. - Import moves to a header action (SkillImportButton) backed by a shared readSkillFile helper; the GitHub-URL import and its /api/skills/import route are removed. - Skill name validation is now one shared validateSkillName, replacing three copies of the kebab-case rule and its messages. - The skill editor roster renders through the shared MemberRow instead of re-deriving its identity block, with a locked role control and a lock-reason tooltip explaining inherited workspace-admin access. Custom tools - Extract the canvas modal's schema/code editors into a shared custom-tool-editor module (fields, wand generation, schema helpers), cutting custom-tool-modal.tsx by ~900 lines. - Settings > Custom tools gains a full-page detail sub-view (SettingsPanel + SettingsSection + saveDiscardActions), deep-linkable via ?custom-tool-id. Rows are clickable; delete now lives only in the detail view. - Replace legacy Button/Input/Badge/Label with the chip family, move chip-field chrome into CodeEditor behind an error prop, and delete its dead wand button. Rich markdown field - maxHeight is now opt-in: omit it on a page and the editor grows with its content so the page owns the only scrollbar. Modals pass explicit caps. - The field variant drops to font-weight 400 to match adjacent chip fields. * fix(skills): address review round on create navigation, 409 copy, and editor audit - Skill create navigated using the first element of the upsert response, but that endpoint returns the caller's whole skill list (built-ins prepended) — match the new skill by its workspace-unique name instead. - The suggested-skill 409 toast claimed the skill existed but was not shared and told the user to ask a skill admin. Every workspace member can already see and use every skill, so a 409 only means the name is taken. - Adding an editor emitted the skill_shared event and SKILL_MEMBER_ADDED audit even when onConflictDoNothing skipped the insert on a concurrent add. Gate both on the insert actually returning a row. * chore: format skills-resolver test import * fix(skills,tools): audit fixes — autocomplete boundary, resize clipping, error routing Two real regressions introduced while simplifying the extracted editor: - The schema-param autocomplete's trigger was rewritten to match a trailing identifier, but the completion still split on separators. The two disagreed, so typing `data.ci` opened the menu and selecting replaced `data.ci` whole — eating the member-access prefix. Both now share one SCHEMA_PARAM_WORD regex. - The uncapped markdown field measured its height only on value change while always setting overflow-hidden, so any width change that re-wrapped lines clipped the tail with no scrollbar to reach it. Now re-measures via ResizeObserver. Also from the audit: - Generation writes bypass the code field's change handler, so an open autocomplete stayed over a disabled streaming editor; close it on busy. - Delete failures rendered in the Schema section's error slot on both custom tool surfaces; route them to a toast instead. - Skill create navigated away while still dirty, stranding the unsaved-changes guard's history sentinel so Back landed on an empty create form. - The Description field on skill create never received its error border. - Drop a double-applied opacity-50 (the editor already dims when disabled), a dead try/catch around a non-throwing call that also shadowed the error prop, and a stale reference to a /tools page that does not exist. - Docs still described the removed GitHub-URL import and the old Add Skill dialog; rewrite for the create page and file/paste import. * feat(tools): read-only tool detail, create lands on the new tool, drop dead wand prompt API - Viewers without edit rights could not open a custom tool at all, while the equivalent skill and custom-block surfaces both offer a read-only view. The detail page now takes `readOnly`: editors inert, no Save/Discard/Delete, no Generate. Creating still requires edit rights. - Creating a tool bounced back to the list while creating a skill lands on the new skill. Tools now do the same. The upsert returns the workspace's whole tool list (newest first) rather than just the new row, so the id is matched by title instead of by index — the same trap that produced the skill-create navigation bug. - Remove `openPrompt`/`closePrompt` from useWand. `closePrompt`'s last callers went away with the custom-tool-modal extraction and `openPrompt` had none before it; nothing reads `isPromptVisible` any more either. * fix(tools): read-only editors, design-system wrench, skills-matching tool identity - readOnly never reached the editors: the prop gated actions and Generate but the schema and code fields were still typable for viewers without edit rights. Wire disabled through both fields into CodeEditor. - The row icon used lucide's Wrench (strokeWidth 2) where @sim/emcn/icons ships one drawn for this system (1.55, tuned viewBox), and it inherited body text colour instead of --text-icon. Swap it. - Give the tool detail page the same identity heading as skill detail: tile, name, and description at the top left, instead of only a header title. - Extract ResourceTile so the skills and tools tiles share one definition (SkillTile now composes it), and add an opt-in `iconFilled` to SettingsResourceRow so the tools list tile matches the skills gallery. Both default to today's behaviour for every existing consumer. * fix(mentions): use the product's own glyph for every @ mention kind The `@` menu and the inserted chip mapped kinds to arbitrary lucide icons — `Sparkles` for a skill, a generic `File` for every file — while the rest of the product has a settled glyph per resource. Mirror CHAT_CONTEXT_KIND_REGISTRY, which Chat's `@` menu already renders from: - skill now uses AgentSkillsIcon, the same glyph SkillTile shows everywhere - workflow / folder / table / knowledge use the @sim/emcn/icons set the sidebar and the chat registry use - file derives its icon from the filename extension, so a .pdf and a .csv are distinguishable, matching the file list and Chat's context chips - integration keeps the block's brand icon from the registry Also drop the generic placeholder. `kind` is untrusted — the node schema defaults it to `''` and a hand-written `sim:` link can carry anything — but an unrecognized kind now yields no icon instead of a meaningless box, which is what the chat registry does. The menu already guarded a missing icon; the chip now does too, so this cannot crash on a malformed link. * chore(db): drop skill_member migration 0266 for regeneration on latest staging * feat(db): regenerate skill_member migration as 0267 after staging merge --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Waleed Latif <walif6@gmail.com> |
||
|
|
d24bc7eccb |
feat(agent-stream): thinking and tool streaming (#5671)
* feat(agent-stream): add agent-events thinking/tool streaming for chat and canvas Ship the agent-events-v1 protocol with provider tool loops, dual-gated chat thinking, DeepSeek/Groq/OpenAI reasoning wiring, and ChatGPT-like thinking chrome. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): clear stuck streaming UI and format db snapshot Biome was failing CI on migrations/meta/0261_snapshot.json. Also settle assistant streaming/tool flags when SSE ends without a terminal frame, without clobbering Stop's finalized content. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): satisfy biome format and import order Auto-format the sim package for CI lint:check, and repair the Anthropic streaming tool-loop payload after an unsafe delete-to-undefined rewrite. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): keep drained answer on abort and update migration journal test Treat AbortError from reader.cancel as a cancelled pump result so soft-complete retains answerText. Point the workspace storage migration journal assertion at 0261_chat_include_thinking. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(chat): keep Stop notice when server emits cancel error Ignore terminal SSE error frames after the user aborts so "Client cancelled request" cannot overwrite "Response stopped by user". Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(chat): ChatGPT-style thinking shimmer and stick-to-bottom scroll Add left-to-right shimmer on live thinking label/body, keep scroll working by shimmering an inner node, and follow the answer only while near the bottom. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): stop pump on client disconnect; soft-complete agents only Abort the agent stream pump when the projected HTTP body is cancelled so provider work does not continue after disconnect. Limit AbortError soft-success to Agent blocks so Function/HTTP cancels still fail in logs. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): persist includeThinking across pause snapshots Paused chat runs with Include thinking enabled were dropping the flag when serializing the pause snapshot, so resume always rebuilt streams without thinking/tool SSE frames. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): keep drained answer text when stream times out Persist pump answerText onto the streaming execution before throwing on timeout, and carry that partial content into the failed block output so logs match what the client already saw. Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(chat): auto-collapse tools chrome when tool streaming ends Match thinking UX: open while tools run, collapse when finished, and keep the panel open only if the user manually reopens it. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): settle canvas stream chrome on failure paths Clear agentStreamActive and settle running tool chips when blocks error, timeouts cancel runs, or execution ends without stream:done so the output panel does not stay on live Thinking/Using tools chrome. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lint): organize imports in terminal console store Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent-stream): mark open tools cancelled on HITL pause Pause can interrupt a tool loop without tool end events; settling those chips as success incorrectly showed unfinished tools as complete. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(db): drop branch-local 0261 migration ahead of staging merge * chore(db): regenerate include_thinking migration as 0266 post staging merge * fix(providers): resolve type errors in streaming tool loop call sites * fix(agent-stream): gate agent events opt-in and correct provider loop behavior - streamToolCalls and provider thinking requests now require run-level agentEvents opt-in (canvas on, chat dual-gated, API off) so existing runs keep pre-agent-events behavior exactly - OpenAI reasoning summaries opt-in + strip-and-retry on unverified-org 400 - streaming loops run tool postProcess again (firecrawl/exa async results) - bedrock live loop falls back to silent path for responseFormat - deepseek: reasoning_content pass-back unconditional, 'none' sends disabled - groq: x_groq.usage fallback, reasoning params gated, qwen none disables - gemini: functionCall parts echoed verbatim, local ids only for events - truncated turns (max_tokens/length) no longer execute partial tool calls - MAX_TOOL_ITERATIONS exit flushes last turn text as final answer - iterations reports actual model calls; shared loop plumbing extracted * refactor(agent-stream): consolidate protocol, dedupe client/server plumbing, hygiene - canonical ChatStreamFrame union + type guards consumed by server emitters and the chat client; stream_error restored to legacy log-only handling - strip thinking/tool args from providerTiming on public final envelopes - shared tool-chip lifecycle module for chat, canvas, and console store - shared sink-to-execution-events forwarder replaces the copy-pasted adapter in the execute route and HITL manager; LIVE_ONLY event set shared - stream:thinking payload field renamed data->text; canvas thinking batched - abort reasons carried as AbortError DOMExceptions so raw fetch consumers classify correctly; thinking cap renamed to chars and scope-documented - kimi wired for agent events like the other compat providers - deleted dead exports/step-N comments; fixtures match real wire shapes; loop tests use explicit mocks instead of importOriginal * test(agent-stream): cover the dual-gated execution path and typed abort reasons - chat route tests assert agentEvents reaches executeWorkflow only when policy and protocol header agree - execution-limits tests assert AbortError-typed reasons - executor metadata type carries agentEvents * fix(deploy-modal): align include-thinking spacing with the modal's 6.5px rhythm * docs(agent-stream): autogenerate per-model thinking/tool stream support on the Agent block page - capabilities.thinking.streamed ('full' | 'summary' | 'none') on models.ts, explicit for the Anthropic family where visibility varies per generation; getThinkingStreamVisibility exposes the derivation for docs and UI alike - scripts/sync-agent-stream-docs.ts regenerates the support tables between markers in workflows/blocks/agent.mdx from the model registry and STREAMING_TOOL_CALL_PROVIDERS; --check fails on drift or missing metadata - wired agent-stream-docs:check into CI next to the other sync gates * feat(anthropic): request summarized thinking display for omitted-default Claude models The newest Claude generations (Fable 5, Sonnet 5, Opus 4.8/4.7) default thinking.display to omitted — empty thinking blocks, no deltas. On agent-events runs Sim now opts back in with display: 'summarized', driven by the registry's streamed metadata; legacy runs keep the exact pre-agent-events request shape. Registry, generated docs, and the family capability table updated accordingly. * docs(skills): cover thinking.streamed and agent-stream docs sync in model skills * chore(deps): upgrade @anthropic-ai/sdk to 0.114.0 and adopt official types - adaptive thinking, display, and output_config are now SDK-typed; the only remaining custom payload field is output_format (beta-header structured outputs, which the SDK models as output_config.format instead) - anthropic stream events narrow on the SDK's discriminated unions instead of anonymous casts; compat deltas type content/tool_calls from the OpenAI SDK with vendor reasoning fields as an explicit optional extension - @sim/auth exposes an explicit VerifyAuth contract so its declarations no longer reference better-auth's nested zod instance (TS2883 under fresh install layouts); realtime consumer aligned - docs app zod pinned to the repo's exact 4.3.6 so ai SDK types bind the same zod instance (docs type-check was latently broken) - knowledge embedding tests made hermetic against local .env keys and hosted rotation fallback * refactor(providers): replace legacy as-any stream casts with annotated typed casts * refactor(providers): finish provider audit — remove dead byte-stream helper, annotate remaining legacy casts Audit of all 26 providers for the agent-events feature confirmed every streaming execution declares agent-events-v1 and every adapter emits AgentStreamEvent objects. Cleanup from the audit: the unconsumed legacy createOpenAICompatibleStream byte helper is deleted, and the remaining streamResponse-as-any casts (xai, nvidia, kimi, meta, zai, sakana) are annotated typed casts matching the groq/deepseek fix. * feat(streaming): stream answer text live during tool loops via turn_end protocol The live tool loops buffered all answer text per model turn (classification of intermediate vs final is only known at turn end), so gated surfaces saw thinking stream, then dead air with the thinking chrome stuck open, then the whole answer at once. Loops now emit text deltas live as `turn: 'pending'` plus a `turn_end` event per turn. The pump buffers pending text and projects it to the byte path (answerText/logs/memory/legacy clients) only on a final turn_end, so all settled semantics are unchanged. Gated surfaces render the pending text as it streams and reconcile with a reset when a turn resolves to tools: - public chat: live `chunk` frames from the sink + dual-gated `chunk_reset`; byte-path frame emission is suppressed to avoid duplicates (kept for response-format transformed streams via clientStreamTransformed) - canvas: forwarder emits live `stream:chunk` + `stream:chunk_reset`; the execute route and HITL resume readers stop re-emitting byte chunks; panel chat tracks per-block segments and replaces content on flush - chat client: per-block text segments, chunk_reset handling, and thinking chrome now settles on tool start as well as first answer chunk * fix(streaming): address validated review findings across provider gating and reset reconciliation Three-reviewer pass over the branch, findings validated against staging: - agent-handler forwards agentEvents to executeProviderRequest — the flag was computed but dropped in the field-by-field copy, so provider-side thinking requests (OpenAI summaries, Gemini includeThoughts, Anthropic summarized display) never activated on opted-in runs - openai: restore summary:'auto' alongside explicit reasoning effort — staging always paired them; gating summary purely on agentEvents changed legacy payloads - gemini: Gemini 2 + tools + responseFormat falls back to the silent path; the live loop never applied the deferred responseSchema for AUTO tools - openai-compat loop: malformed tool-argument JSON fails the call instead of executing with defaulted {} args (staging parsed inside the execution try) - openai-compat parser: a vendor id arriving after a synthesized start no longer renames the call (start/end ids stayed consistent) - stream-pump: abort closes the byte projection so a drain blocked on backpressure cannot deadlock teardown - chunk_reset removes the block from the client text order (deployed chat + panel chat) so a reset block re-registers at arrival position — fixes separator/order corruption when parallel blocks stream around a reset - resume route echoes the negotiated X-Sim-Stream-Protocol response header (parity with the chat route); docs: [DONE] wire shape + final-vs-error terminal semantics corrected * chore(deps): exempt pinned @anthropic-ai/sdk 0.114.0 from the release-age gate CI's bun install --frozen-lockfile blocks 0.114.0 (published 2026-07-23, younger than the 7-day supply-chain gate). The pin is exact and was vetted for the agent-events streaming work; following the existing bunfig pattern, the exclusion ages out on 2026-07-30 and should be dropped then. * chore(providers): fix double-cast-allowed annotation placement for the strict boundary audit The audit only recognizes the annotation on the line directly above the cast; two annotations had drifted behind intervening code lines (groq stream params, deepseek loop messages) and the OpenAI reasoning-summary widening cast was never annotated. No behavior change. * fix(chat): settle straggler tool chips as error when final reports failure A failed run can still terminate with a `final` frame carrying success: false; running chips previously settled green regardless of the outcome. * fix(canvas): wire agent stream chrome into run-from-block Run-from-block executions emit the same live stream:thinking/stream:tool events as full runs but registered none of the handlers, so the terminal never showed thinking or tool chips on that path. The per-run chrome (batched thinking writes + tool chip lifecycle + settlement on stream done, block error, and every terminal execution state) is extracted into a shared createAgentStreamChrome factory consumed by both paths. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> |
||
|
Justin Blumencranz
|
03adc8fe1d | fix(chat): rotate fork chat split icon 90 degrees (#5914) | ||
|
Waleed
|
d48722a04e |
fix(helm): correct chart docs, examples, and dead config across the board (#5907)
* fix(helm): correct chart docs, examples, and dead config across the board Audit-driven accuracy pass over the chart's entire documentation surface, verified by rendering every example against the templates: - migrations run as an init container on the app pod, not a Job — fix the README component list, troubleshooting commands, and sim-helm skill refs; drop the dead migrations-job NetworkPolicy ingress rule - referenced-but-never-created resources: document the GKE ManagedCertificate creation (values-gcp), comment out the key-file Secret mount that stuck all pods in ContainerCreating (values-gcp), enable certManager for the postgres TLS issuerRef (values-production), add the cert-manager cluster-issuer annotation nginx needs (values-azure) - values-external-db: networkPolicy.egress is a list, not a map (the map rendered an invalid manifest); fill schema-failing placeholder host/username - realtime >1 replica requires REDIS_URL (Socket.IO Redis adapter) — default examples to 1 replica with the scaling note, and warn where autoscaling HPAs override replicaCount - pod anti-affinity selectors matched nothing (simstudio vs sim name label) - kubernetes.mdx: install commands were missing required CRON_SECRET and postgresql password (failed at template time), wrong deployment name in port-forward, stale version requirements, unsupported key-remapping claim - remove unimplemented app.secrets.existingSecret.keys from values + schema; fix README PDB default, cronjob list, /metrics caveat, NOTES secret count, Azure-only StorageClass in generic examples, dead SOCKET_SERVER_URL and GOOGLE_CLOUD_* env, ESO apiVersion mismatch, and skill-reference drift - bump chart to 1.0.1 * fix(helm): review round 1 — scoped example egress, in-tab secret note, copilot Job wording - external-db example egress scopes to a placeholder database CIDR instead of to: [] (which allowed every destination on 5432, defeating the isolation the example teaches) - kubernetes.mdx cloud tabs state explicitly that they reuse the variables generated in the Installation block - Copilot migrations really do run as a Helm-hook Job — restore Job wording there (only the app migrations are an init container) * fix(helm): template-sweep fixes — telemetry validity, ESO rollout checksums, dead passwordKey knob - telemetry: memory_limiter gets the required check_interval (collector failed startup validation whenever telemetry.enabled=true); the jaeger exporter was removed from collector-contrib in v0.86 — export to Jaeger via its native OTLP endpoint instead (otlp/jaeger, default port 4317) - app/realtime rollout checksums now hash the ExternalSecret manifest too, mirroring the copilot pattern — with ESO enabled the inline Secret renders empty, so remoteRefs changes never rolled the pods - remove the unimplemented existingSecret.passwordKey knob (values, schema, README, dead helpers): nothing consumed it, and a non-default value silently produced a DATABASE_URL with an unexpandable placeholder; secrets must use the standard POSTGRES_PASSWORD / EXTERNAL_DB_PASSWORD keys - drop the orphaned sim.migrations.labels helper (its only consumer was the dead NetworkPolicy rule removed earlier) - helm test pod image resolves through sim.image so global.imageRegistry mirroring applies; NetworkPolicy realtime-ingress comment reflects actual traffic direction; smoke unittest suite loads the newly referenced external-secret template * chore(helm): bump chart to 1.1.0 with upgrade notes Removing (inert) documented values keys and changing the rollout-checksum inputs is a values-surface change — per SemVer chart conventions that is more than a patch. Adds an Upgrading section documenting the one-time pod roll, the removed no-op keys, and the Jaeger-over-OTLP change. * fix(helm): review round — external-db NP opt-in with real-CIDR-first flow, prod Jaeger OTLP endpoint - external-db example ships networkPolicy disabled so a verbatim install always reaches the database; the scoped egress rule stays as the documented opt-in (set your CIDR first, then enable) - values-production still pointed telemetry.jaeger at the legacy 14250 collector port — now Jaeger's OTLP gRPC endpoint to match the otlp/jaeger exporter * feat(helm): autoscaling.realtime.enabled toggle so examples can scale the app without unsafe realtime replicas Cursor correctly flagged that comment-level warnings didn't stop a verbatim production/external-db install from running the realtime HPA at minReplicas 2 without REDIS_URL (silent cross-pod event loss). Adds an opt-out toggle (default true — existing deployments unchanged): the realtime HPA renders only when autoscaling.realtime.enabled, and the realtime Deployment keeps spec.replicas under its control when the HPA is excluded. The three autoscaling examples set it false with the Redis rationale; README and upgrade notes document the toggle. * fix(helm): review round — whitelabeled realtime HPA opt-out, external-db isolation on with required CIDR in install flow - values-whitelabeled now actually sets autoscaling.realtime.enabled: false (the earlier batch aborted before reaching this file — Cursor caught it) - external-db keeps networkPolicy enabled (no isolation regression); the DB egress CIDR is marked REQUIRED and wired into both documented install commands via --set, so the copy-paste flow sets the real subnet in the same breath as the DB host * fix(helm): use a syntactically valid example CIDR in the external-db install commands An unreplaced <YOUR_DB_CIDR> literal fails Kubernetes CIDR validation and aborts the install; every sibling placeholder in the same command (host, username) installs fine and simply doesn't connect until replaced. The CIDR now behaves the same way: valid example value (10.20.0.0/24), explicitly marked as the operator's database subnet in both commands and the values comment. * fix(helm): remove text after continuation backslashes in external-db install commands A trailing comment after the line-continuation backslash (and equally a comment line spliced mid-command) breaks the shell command when copied — the remaining --set overrides run as separate commands and required-value validation fails. Both documented commands now reconstruct to bash-clean multi-line invocations (verified with bash -n); the CIDR guidance lives in the networkPolicy section comment. * fix(docs): cloud-tab installs are alternatives via helm upgrade --install Following Installation and then a cloud tab ran helm install twice for the same release and failed on the second. The tabs now state they replace the generic install and use helm upgrade --install, which is idempotent and also converts an existing generic install to the cloud values. * fix(docs): honest conversion caveat for cloud-tab upgrades over an existing install The cloud values rename the bundled Postgres database to simstudio, which Postgres only applies at first initialization — an in-place conversion of a generic install would point DATABASE_URL at a nonexistent database. Document the two safe paths: keep the original name via --set, or uninstall + delete PVCs and install fresh. * fix(helm): external-db header install command declares its secrets and includes CRON_SECRET The primary documented command failed the chart's required-value validation (missing app.env.CRON_SECRET with cronjobs default-on) and referenced an undeclared DB_PASSWORD. It now shows the export lines for every variable it uses and sets CRON_SECRET; both commands verified with bash -n. * chore(helm): restore values.schema.json formatting — surgical deletions only The earlier programmatic edit reformatted the whole file (~390 lines of whitespace churn hiding the 12 real deleted lines). Re-applied the removal of the dead keys/passwordKey properties as text-level deletions preserving the original style. * fix(helm+docs): explicit secret exports in external-db header; conversion must reuse original secrets - all five export lines are written out (three were only named in a trailing comment, so a verbatim copy passed empty required values) - the cloud-conversion caveat now leads with reusing the original secret values (helm get values) — a regenerated ENCRYPTION_KEY makes previously encrypted credentials undecryptable |
||
|
Waleed
|
ca2ea0066d |
improvement(library): add citations and internal links, correct stale pricing (#5913)
* improvement(library): add citations and internal links, correct stale pricing The library had zero third-party citations across 16 posts (~51k words) and averaged 1.2 internal links per post, with six posts at zero. Auditing every dollar figure against the vendor's own pricing page while adding the citations turned up several claims that had gone stale, plus one product that is being shut down. Corrections, each verified against the vendor's page on 2026-07-23: - Relay.app is winding down (signups closed 2026-07-16, free accounts end 2026-08-15, paid 2026-09-14). It was recommended as the human-in-the-loop pick in best-zapier-alternatives; that recommendation now points to a platform you can still sign up for - Make Core is $12/mo billed monthly, not $10.59; annual saves ~15%. One post also described $12 as the annual rate - Pabbly Connect lifetime starts at $349, not $249, and the Standard/Pro/ Ultimate tier structure quoted no longer exists - Workato publishes no pricing at all, so the "~$1,000/month" figure is replaced with the fact that every deal is quoted through sales - Dify is at ~149k GitHub stars, not 131k - n8n cloud Starter is EUR 20/mo billed annually for 2,500 executions Verified-correct figures were left alone and given a source link: Zapier $29.99/mo monthly for 750 tasks, Activepieces 10 free flows then $5/flow/mo, Power Automate $15/user/mo, Lindy $49.99/mo, and the Grand View Research RPA market figures. Also: 59 outbound citations and 3-6 internal links per post (zero posts left without internal links), and MDX external links now carry target=_blank plus rel=noopener noreferrer, which the landing SEO/GEO rule requires but the MDX anchor was not applying. * fix(content): treat only non-Sim hosts as external links in MDX The first pass classified any http(s) href as external, so the 33 absolute Sim URLs already in content (sim.ai, sim.ai/slack, www.sim.ai/blog/*, and 14 docs.sim.ai pages) would have opened in a new tab with rel=noopener, which is wrong for first-party links and contradicted the comment above the check. Classification now compares the hostname against the apex derived from SITE_URL, so the apex and any Sim subdomain stay same-tab. SITE_URL is used rather than getBaseDomain() so a post renders identically in dev, preview, and production instead of varying with NEXT_PUBLIC_APP_URL. The leading dot in the suffix check keeps lookalikes such as evil-sim.ai external. |
||
|
Waleed
|
1ffd4f0739 | improvement(landing): close remaining SEO keyword gaps on home and enterprise (#5912) | ||
|
Waleed
|
877dc9e719 | fix(landing): lead enterprise summary with Sim so product schema names the module (#5910) | ||
|
Waleed
|
dbbfce88c4 |
fix(landing): restore single-paragraph hero copy on home and enterprise (#5906)
* fix(landing): restore single-paragraph hero copy on home and enterprise * fix(landing): tighten enterprise hero description * revert(landing): restore original home and enterprise hero headings * improvement(landing): simplify enterprise hero description * improvement(landing): restore building-and-managing homepage headline |
||
|
Waleed
|
79c57bfaf6 |
improvement(content): surface last-verified and updated dates, codify citation rules (#5904)
* improvement(content): surface last-verified and updated dates, codify citation rules Comparison pages already computed a latest-verified date from every fact source's asOf and emitted it as JSON-LD dateModified, but never showed it. Library and blog posts had the same gap: dateModified existed only as an invisible meta tag. A freshness signal that no reader can see does nothing for the reader deciding whether to trust the page. - /comparisons/[provider] renders "Last verified <date>" from the existing getLatestVerifiedDate(), so the visible date and the JSON-LD read the same value and cannot drift - /library and /blog posts render "Updated <date>" next to the publish date, only when the modified day actually differs; otherwise the meta fallback stays. dateModified is emitted exactly once either way - collapse three duplicated toLocaleDateString blocks in content-post-page into one module-scope formatDate (same UTC pinning, identical output) - landing-seo-geo rules: add citation/internal-linking and freshness sections, and extend the rule's paths to content MDX so it attaches when authoring posts, not just TSX * fix(content): wrap post metadata row so the Updated date cannot overflow on narrow screens The published/updated/authors/share row was a non-wrapping flex. With the Updated label active and two authors it overflowed at 390px; the row also already overflowed at 320px on staging before this PR, with no Updated label at all. flex-wrap fixes both and leaves desktop identical. * improvement(comparisons): fold last-verified date into the intro sentence The paragraph already ended "Every fact below is sourced and dated" and a separate line then stated the date, which read as redundant and added a standalone metadata row to the header. Folding it into that sentence drops the extra line and keeps the date as real server-rendered text in a <time> element, so crawlers and AI answer engines still see it. A tooltip would not: Tooltip.Content renders null during SSR and while closed. |
||
|
Theodore Li
|
96c67e9123 |
feat(sandbox): add Daytona as a manual-flip failover for E2B (#5860)
* feat(sandbox): add Daytona as a manual-flip failover for E2B
E2B was a hard single point of failure: lib/execution/e2b.ts had no retry
and no fallback, so a failed Sandbox.create() killed Python function blocks,
JS-with-imports, shell, doc generation and the Pi cloud agent outright.
Extract a SandboxRunner boundary (lib/execution/remote-sandbox) with an E2B
runner and a Daytona runner, selected once per execution by the
sandbox-provider-daytona AppConfig flag. Everything above the provider
boundary — marker parsing, mount materialization, file export, corruption
handling — is unchanged.
Selection resolves before create() and never mid-execution, since user code
has side effects. Each sandbox kind fails closed when its snapshot id is unset.
Notes on the Daytona adapter:
- language binds at create(), not per call: Daytona applies it as a sandbox
label and silently runs JS through Python if passed to codeRun
- Python routes via CodeInterpreter for its {name,value,traceback} error shape,
which matches E2B's and keeps formatE2BError's line offsets correct
- timeouts convert ms to seconds
- the streaming path delivers env via the filesystem API, as
SessionExecuteRequest has no env field and secrets must not reach a command line
Drops the dead E2BExecutionResult.images field (populated, never consumed).
* improvement(sandbox): select the provider by SANDBOX_PROVIDER env var
Replaces the boolean sandbox-provider-daytona feature flag with a
SANDBOX_PROVIDER env var naming the provider ('e2b' default, or 'daytona').
A boolean doesn't scale to a third adapter; a keyed registry does.
- PROVIDERS is a Record<SandboxProviderId, SandboxProvider>, so adding an
adapter is one entry plus one id-union member — an unhandled provider is a
compile error, not a runtime surprise
- resolveProvider() reads env synchronously and throws on an unknown value
(fail fast) instead of an async feature-flag lookup
- drops the sandbox-provider-daytona flag and SANDBOX_PROVIDER_DAYTONA fallback
Verified end-to-end: a Python function block through the running app routes to
Daytona (Creating Daytona sandbox, kind: code) with SANDBOX_PROVIDER=daytona.
* fix(sandbox): gate remote execution by provider availability, not E2B
Addresses the review round on #5860.
- Availability was gated on isE2bEnabled / isE2BDocEnabled, so a Daytona-only
deployment (E2B_ENABLED unset) had its Python/shell/JS-with-imports and doc
paths rejected before the provider-neutral sandbox call could run. Replace both
with provider-aware flags (isRemoteSandboxEnabled / isDocSandboxEnabled) derived
from the selected SANDBOX_PROVIDER's own credentials + image. E2B behavior is
unchanged (the E2B branch mirrors the old definitions exactly).
- Make the function-block gate error messages provider-neutral.
- Daytona's streaming runCommand (Pi) returned empty stdout/stderr and delivered
output only via callbacks, so the Pi cloud flow — which parses markers from
stdout and formats errors from stderr — saw nothing. Accumulate the streamed
chunks and return them while still forwarding to the callbacks.
Renames the env-flag exports (and the @sim/testing mock) to match. Adds a
conformance test that the streamed Pi output lands in stdout/stderr.
* fix(sandbox): resolve SANDBOX_PROVIDER case-insensitively
Addresses the round-2 review on #5860. env-flags lowercased SANDBOX_PROVIDER
for the availability gate, but resolveProvider looked up the raw value in a
lowercase-keyed map — so 'Daytona' passed the gate then threw Unknown
SANDBOX_PROVIDER at create. resolveProvider now normalizes casing identically.
* fix(sandbox): use getErrorMessage in build/verify scripts
check:utils flagged the inline `error instanceof Error ? error.message : ...`
pattern in the two new scripts. Use getErrorMessage from @sim/utils/errors,
matching the repo convention the check enforces.
* fix(sandbox): fall back to stdout for Daytona failure text
Daytona merges both streams into stdout and returns an empty stderr, but the
shell-error, base64-export, and URL-mount error builders read only
result.stderr — so Daytona failures surfaced a generic 'Process exited with
code N' / 'base64 failed' / 'curl exited N' instead of the real command output
that the API and agents rely on. Fall back to stdout before the generic message
(provider-agnostic: E2B still populates stderr). Strengthens the shell-error
conformance test to assert the real output surfaces.
* fix(sandbox): enforce timeout on Daytona streaming; drop leftover E2B copy
- Daytona's streaming path (Pi) started the command with runAsync:true and then
awaited getSessionCommandLogs with no bound, so a hung command never timed out
the way E2B's commands.run({ timeoutMs }) does. Race the log stream against the
timeout; on expiry return exit 124 with the accumulated output, and the finally's
deleteSession terminates the still-running command.
- Two user-facing strings still named E2B after the provider-neutral rename (the
isolated-vm sandboxPath remediation and the disabled-xlsx message). Made both
provider-neutral.
Adds a streaming-timeout conformance test.
* fix(sandbox): handle orphaned stream promise + preserve error detail
Two regressions from the previous timeout fix:
- When the timeout won the race, the abandoned getSessionCommandLogs promise
would reject on deleteSession with no handler (unhandledRejection). Attach a
.catch that records the error and yields an 'error' outcome, so a late
rejection is always handled.
- The streaming catch dropped the thrown error, so failures before any chunks
(env write, executeSessionCommand, missing cmdId) surfaced as empty output.
Fall back to getErrorMessage(error) when nothing streamed.
Adds conformance tests for the stream-reject and start-throw paths.
|
||
|
Waleed
|
3796e9db4f |
improvement(access-control): edit group details, filter by status, and colocate chat deploy auth (#5902)
* improvement(access-control): editable group details, status filters, block tooltips, chat auth colocation - General tab gains editable Name and Description fields wired into the existing dirty buffer, so the header Save/Discard chips and the unsaved-changes guard cover them. Save only sends changed fields and surfaces the route's duplicate-name 409. - Blocks, Model Providers, and Platform tabs gain an All/Enabled/Disabled filter beside their search field, evaluated against the editing buffer. - Every block row carries an Info badge with the block's description. The badge sits outside the label/expand button so it never toggles the row. - The chat deploy toggle moves out of Deploy Tabs into the Chat section alongside its allowed-auth-modes dropdown, mirroring the Files section. * improvement(access-control): polish group detail filters, tooltips, and details fields Follows the cleanup review: - reconcile the post-save baseline from the server response instead of local values, matching the scope/default writes - pin the status-filter dropdown width so the search field stops resizing - restore flex-1 on the block-name button and move the row hover surface to the wrapper so Info badges align in a column - add empty states for filter-empty lists and neutralize Select All there - surface a 'Name is required' message next to the disabled Save - align hint text on the field-hint tokens and drop a redundant TSDoc * refactor(access-control): keep chat and files toggles in the platform registry The first pass pulled hideDeployChatbot out of the declarative platformFeatures array and hand-rolled a Chat section beside the existing bespoke Files one. That forfeited search, status filtering, Select All, category grouping and the Info hint, and the replacement platformSectionVisible re-implemented two of those with different semantics — searching 'deploy' or 'deployment' hid the very control named Deployment, and Select All silently skipped both toggles. Both toggles are now ordinary registry entries under their own Chat and Files categories, with an id-keyed featureExtras map supplying the nested auth-mode dropdown. Search, filtering, Select All, hints and the empty state are correct by construction, and the parallel filter pipeline is gone. Also from the review: - index the allow-lists into Sets so per-row membership checks are O(1) - split the search and status passes so the common 'all' filter returns the searched list by reference and a checkbox toggle no longer re-sorts ~180 rows - extract StatusFilterChip and AuthModeField instead of stamping out the dropdowns three and two times - derive nameChanged/descriptionChanged once instead of repeating the comparisons in the save payload - lock the config key-order invariant the dirty check depends on with a test * polish(access-control): apply the second cleanup round - indent the nested auth-mode field so it lines up under its toggle's label instead of reading as a sibling row, and label the dropdown for screen readers - order Chat right after Deploy Tabs so the three deploy targets stay adjacent - flush the trailing Select All chips on the providers and platform rows - drop the doubled margin on the name error (SettingRow already gaps it) - hoist PLATFORM_FEATURES and PLATFORM_CATEGORY_ORDER to module scope - drop the useCallback on the two save/discard handlers; nothing observes their identity and their deps changed on every keystroke anyway - fix a comment that still pointed at a 'Hide Chat' toggle that no longer exists * fix(access-control): keep the block disclosure chevron inside its toggle button Splitting the chevron out so the Info badge could sit beside the name left the chevron with no click handler — the visible expansion affordance did nothing. It goes back inside the button; Info stays outside it, since an Info trigger is itself a button and cannot nest. * chore(access-control): use structuredClone in the config key-order test check:utils bans JSON.parse(JSON.stringify(...)). The clone only needs to hand the schema a distinct object; structuredClone preserves key order the same way. * fix(access-control): trim descriptions so a padded value can't wedge the form descriptionChanged compared a trimmed draft against the raw saved description, so a group whose stored description carried padding opened dirty and could never be cleared — Discard restored the same padded string, and the unsaved-changes guard then blocked navigation until a save rewrote it. The contract now trims description on create and update, matching what name already did, and the dirty check trims both sides so existing padded rows behave too. * improvement(access-control): seed the description buffer trimmed Keeps the editing buffer and the dirty baseline normalized the same way, so a legacy row with padding no longer shows stray whitespace in the input and the buffer never round-trips padding a save would strip anyway. * fix(emcn): forward aria-label/aria-labelledby from ChipDropdown to its trigger ChipDropdown destructures only its known props, so an aria-labelledby passed by a consumer never reached the trigger button — AuthModeField's wiring to its visible label was silently dropped and the control had no accessible name. Both attributes are now explicit, typed props forwarded to the trigger. Kept as two named props rather than a rest spread so the component still owns its chrome and consumers can't smuggle arbitrary attributes onto the button. * fix(access-control): correct the nested field indent, platform row hover, and aria name - aria-labelledby REPLACES the content-derived name, so naming the auth-mode dropdown with its caption alone dropped the selected value from the accessible name — worse than no attribute. It now references caption + trigger, which needed ChipDropdown to forward id as well. - The nested field's pl-[30px] assumed a 14px checkbox; the default is 16px, so the caption sat 2px left of the label it hangs under while the dropdown (not flush, so mx-0.5) sat at 32. Both are pl-8 + flush now. - Platform feature rows kept their hover surface on the label, so the highlight stopped 20px short of the row edge while the Blocks tab ran flush. Moved to the wrapper, matching the core-blocks cell. - Split the platform search and status passes like the provider and block lists, so a toggle no longer recomputes three chained memos. - Dropped an unreachable allLabel and a comment that would go stale on merge. * fix(access-control): trim the name comparison the same way as description nameChanged compared a trimmed buffer against an untrimmed baseline — the exact asymmetry already fixed for description. A group stored before the name schema gained .trim() opens permanently dirty: Save/Discard visible and the unsaved-changes modal firing on back, until a save rewrites it. Both buffers now seed trimmed and compare trimmed on both sides. Also dims the Info badge along with the row it belongs to when the block is disallowed. |
||
|
Waleed
|
6f90d8976d |
fix(mcp): stream pinned transport under Bun (providers + self-hosted-private MCP) (#5901)
* fix(mcp): stream pinned transport via undici.request + redirect interceptor Extends the Bun undici-streaming fix to createPinnedFetchWithDispatcher (providers, A2A, self-hosted-private MCP over SSE). It now routes through undiciRequestAsResponse like the guarded builder, so streaming bodies deliver under Bun. Unlike the guarded path it has no followRedirectsGuarded wrapper (it's handed straight to provider SDKs), so redirects are followed via undici's redirect interceptor composed onto the pinned Agent — every hop still dispatches through the pinned connect.lookup (resolvedIP), so a redirect can't escape to another address, matching the old fetch guarantee. secureFetchWithPinnedIP (raw Node http, tools path) is untouched. * fix(mcp): honor redirect mode + drop cross-origin credentials on pinned fetch Replaces the always-on redirect interceptor with redirect-mode-aware handling: - redirect:'manual' returns the 3xx without following (detectMcpAuthType inspects it) - redirect:'error' throws on a 3xx - default 'follow' uses followRedirectsGuarded, which drops ALL headers on a cross-origin hop (so a redirect can't disclose a provider api-key to another origin — Greptile P1) and stamps the final response.url + redirected flag. Extracts the shared Request-lift helper used by both guarded and pinned builders. * fix(mcp): don't block private IP-literal URLs on the pinned fetch path Routing the pinned fetch through followRedirectsGuarded added an initial assertGuardedRedirectTarget check the old undici.fetch path never ran, which would block a self-hosted MCP configured with a private IP-literal URL (e.g. http://10.0.0.5:3000/mcp) — its own transport. The pinned path's callers already validate the target and the private carve-out intentionally pins to a private IP, so skip the initial-target check (validateInitialTarget: false) while still validating every redirect hop. Adds a regression test. * fix(mcp): carry redirect mode from a Request input in liftFetchArgs liftFetchArgs copied method/headers/body/signal from a Request but omitted redirect, so a Request({ redirect: 'manual' }) on the pinned path defaulted to 'follow' and was transparently followed. Copy input.redirect (explicit init still wins). Adds a Request-input redirect-mode test. * fix(mcp): permit the pinned IP as a redirect target (initial + hops), block other private IPs Consolidates the pinned-path redirect policy into one mechanism. followRedirectsGuarded took validateInitialTarget to skip the initial private-IP check, but per-hop checks still blocked a self-hosted MCP redirecting to its own pinned private IP (e.g. a trailing-slash 301 to http://10.0.0.5/mcp/). Replace it with allowRedirectToIp: the pinned fetch permits exactly its own validated IP as a target — initial URL and any hop that stays on it — while every OTHER private target (e.g. the 169.254.169.254 metadata IP) stays blocked. Tests cover the same-IP hop (followed) and the metadata-IP escape (still refused). |
||
|
Waleed
|
a5e1030e9e |
fix(mcp): stream guarded transport via undici.request so SSE responses work under Bun (#5897)
* fix(mcp): stream guarded transport via undici.request so SSE responses work under Bun undici's fetch exposes its response body as a WHATWG ReadableStream whose bridge is broken under the Bun runtime the standalone server runs on: headers arrive but response.body never yields data, hanging every MCP streamable-HTTP (text/event-stream) tools/list read to its 30s timeout. undici's lower-level request() returns a Node Readable, which Bun streams natively. createSsrfGuardedFetchWithDispatcher (the MCP transport builder) now routes through undiciRequestAsResponse: same guarded Agent + connect.lookup (SSRF unchanged), request() instead of fetch(), and a hand-rolled Node->Web body bridge (not Readable.toWeb, which throws ERR_INVALID_STATE on the redirect body.cancel()). Buffered reads, followRedirectsGuarded, maxResponseSize, and abort all preserved and verified on both Node and Bun. * fix(mcp): use a single cast for header-iterable detection to satisfy boundary audit * fix(mcp): handle URLSearchParams OAuth bodies and copy stream chunks - toUndiciRequestBody serializes a URLSearchParams body (the MCP SDK's OAuth token/refresh exchange sends one); undici.request rejects it otherwise. - Default content-type application/x-www-form-urlencoded when the caller didn't set one (fetch parity). - nodeReadableToWebStream enqueues a copy (new Uint8Array(chunk)) not a view, so undici recycling the pooled source buffer can't corrupt queued chunks. - Tests for both. * fix(mcp): decode Content-Encoding on the undici.request transport (fetch parity) undici.request returns raw bytes; fetch auto-decompresses. Restore that: pipe a gzip/deflate/br response body through the matching zlib decoder before the WHATWG bridge and strip content-encoding/content-length. maxResponseSize still caps the compressed wire bytes; errors forward into the decoder and the source is torn down when the decoded stream ends or is cancelled. Adds a gzip decode test. * fix(mcp): guard decoder errors and null-body drain against unhandled crashes - Attach the stream bridge's error listener before piping into the zlib decoder, so a synchronous zlib error (server mislabeling a non-gzip body as gzip) rejects the reader instead of crashing the process. - Attach an error listener before draining a null-body response, and wrap Response construction in try/catch that destroys the source (no socket leak) on an out-of-range status. Adds an invalid-gzip regression test. |
||
|
Vikhyath Mondreti
|
e737901b0b | chore(blocks): rename webhook block (#5900) | ||
|
Waleed
|
5c427795a2 |
fix(confluence): preserve panel/callout macro semantics through sync (#5896)
* fix(confluence): preserve panel/callout macro semantics through sync
Confluence's rendered view HTML wraps Info/Note/Warning/Tip and custom Panel
macros in divs whose class/color convey meaning that the shared
htmlToPlainText tag-stripper discards along with the tags — a red "do not
use" warning panel becomes indistinguishable from a plain paragraph once
flattened, so RAG has no signal that a bullet under it is an exclusion rule
rather than a normal one.
Adds preserveConfluenceCallouts, a Confluence-specific pre-pass that rewrites
each detected panel into a single bracketed label (e.g. "[WARNING] Do NOT use
this form for: GitLab") before the generic plain-text conversion runs, so the
callout semantic survives both the tag strip and htmlToPlainText's trailing
whitespace collapse. Bumps the connector's content-representation marker so
already-synced pages get one automatic re-hydration under the new extraction,
rather than silently keeping their stale flattened content until their next
edit.
* fix(confluence): preserve word boundaries when extracting callout body text
Greptile P1: cheerio's .text() concatenates every descendant text node with
no separator, so pulling a macro body's text in one call fused adjacent
blocks together (e.g. a paragraph ending in "for:" immediately followed by a
list item "GitLab" became "for:GitLab"), corrupting the exact word boundaries
RAG chunking and keyword matching depend on.
extractBlockJoinedText now extracts each paragraph/list-item/heading/cell/quote
individually and joins them with a single space, keeping every block's text
intact and properly separated, matching how htmlToPlainText already treats
the rest of the page.
* fix(confluence): fix nested-block duplication in callout text extraction
Greptile P1: filtering the found blocks to only top-level ones still wasn't
enough — a nested block (an outer <li> containing its own nested <ul><li>, a
<td> containing a <blockquote>) matched the selector once, but its .text()
call recurses into and flattens its own matched descendants with no
separator, reproducing the exact word-fusion bug one level deeper (and any
duplicate-selection would have double-counted the same text).
Replaces the block-selector approach with a recursive text-node walk:
extractBlockJoinedText now visits every text node individually and joins them
all with a single space, so word boundaries are preserved at every nesting
depth with no double-counting, matching the pattern html-parser.ts already
uses elsewhere in this codebase for the same class of problem.
* fix(confluence): apply the same word-boundary-safe extraction to panel headers
Greptile: panelHeader text extraction was left on the plain .text() call
while panel/macro body extraction was already fixed to use
extractBlockJoinedText, so a rich multi-node header (e.g. <b>Warning:</b>
followed by a sibling <span>) could still fuse into "Warning:Do not use"
with no space. Panel headers now go through the same recursive text-node
walk as bodies, for consistency across every text extraction in this file.
* fix(confluence): distinguish inline formatting from block boundaries in extraction
Greptile: the recursive text-node walk unconditionally inserted a space
between every text node, which fixed block-boundary fusion but broke
genuinely inline-formatted text — "un<b>believe</b>able" became
"un believe able" and "Hello<b>!</b>" became "Hello !", corrupting valid
callout content on its way into the index.
Adds an INLINE_FORMATTING_TAGS allowlist (b, strong, i, em, span, a, etc.):
text flowing through those tags accumulates with no artificial separator,
preserving exact source adjacency, while every other tag boundary (p, li,
td, headings, br, ...) still flushes to a new segment — a block always
implies a break even with no literal whitespace in the source, but an inline
tag never does. Fixed one test that had encoded the old, incorrect
expectation for two genuinely adjacent inline tags with no source whitespace
between them, and added regression tests for mid-word inline formatting,
punctuation attached to an inline tag, and a header with real source spacing.
* fix(confluence): process nested panels/macros innermost-first
Cursor: processing matches in document order (outermost first) read a
nested, not-yet-converted panel/macro as plain body text before it ever got
its own bracketed label, silently dropping the inner callout's type. Worse,
an untitled outer panel's `.find('.panelHeader')` could reach past its own
missing header into a nested panel's header and adopt it as its own title.
Replaces the two independent .each() passes with a loop that converts only
"leaf" macros (no remaining nested macro/panel inside them) and repeats
until none are left. This processes innermost-first, so a nested macro is
already its own bracketed <p> by the time its parent's body/header text is
read, and an untitled outer panel's .find() can no longer reach a header
that isn't its own, since a leaf by definition has no nested panel left to
reach into.
* test(confluence): add explicit regression for the exact reported <br> repro
Formalizes an explicit test for the exact <br>-separated string Greptile's
review cited as broken (verified manually not to reproduce, but wasn't
directly asserted in the suite before this).
|
||
|
Theodore LiandClaude Opus 4.8
|
84e7aad633 |
improvement(slack): request the approved mention/assistant/DM scopes, advertised on v2 only (#5898)
Slack app review has since approved `app_mentions:read`, `assistant:write`, and `im:history` — they are live in the prod app manifest's `oauth_config.scopes.bot` — but the repo still had them commented out behind a stale "re-add once approved" TODO. Sim was therefore requesting a narrower grant than the app is entitled to, so newly connected accounts got tokens missing exactly the scopes backing three `simSubscribed` events on the native Sim app trigger: `app_mention` (app_mentions:read), assistant threads (assistant:write), and DMs (im:history). The requested scope set now matches the manifest's 20 approved bot scopes exactly. Scopes are per-credential, not per-block (one Slack app -> one `slack` provider -> one shared token, requested server-side via getCanonicalScopesForProvider), so the grant itself cannot be scoped to v2. What is per-block is what each picker advertises and treats as missing, so: - slack_v2 + the slack_oauth trigger advertise the full set (they host the native Sim app trigger that needs it). - The legacy v1 block stays pinned to the pre-expansion 17 scopes. It has no feature needing the new three, and advertising them there would flag every existing Slack credential as missing scopes and prompt a needless reconnect. Claude-Session: https://claude.ai/code/session_018asmKsWQ5Vi7T7wD9uHofz Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Waleed
|
bc237a0dcc |
improvement(shopify): pin Admin API to supported 2025-10 via shared constant (#5895)
* improvement(shopify): pin Admin API to supported 2025-10 via shared constant Every Shopify tool and the credential validator hardcoded the retired 2024-10 Admin API version. Retired versions still work (Shopify forward-falls to the oldest supported version) but the served version drifts silently and emits deprecation signals. - Add apps/sim/tools/shopify/constants.ts exporting SHOPIFY_API_VERSION as the single source of truth; bump to the supported 2025-10. - Wire all 21 Shopify tools, the token-service-account validator, and the OAuth store route to the shared constant (no more per-file inline version). - Derive the validator test's expected URL from the constant so it can't drift. Our operations already use modern 2024-10+ shapes (ProductCreateInput / ProductUpdateInput new product model, orderCancel new signature, CustomerInput, InventoryAdjustQuantitiesInput, fulfillmentCreate), none of the removed inline ProductInput.variants pattern — so the bump is a small, explicit step forward from the version Shopify already forward-serves. * chore(shopify): remove dead ShopifySetInventoryParams type Surfaced by /validate-integration: unexported, unused (no set_inventory tool). |
||
|
Waleed
|
d39b193201 |
feat(sidebar): search the workspace switcher when a user has many workspaces (#5893)
* feat(sidebar): search the workspace switcher when a user has many workspaces Shows a search input in the workspace dropdown once the list exceeds WORKSPACE_SEARCH_THRESHOLD (3). ArrowUp/Down move through results, Enter switches, and the query resets on close. All the search machinery is gated on showSearch so users with few workspaces get no extra re-renders. The input reuses the emcn ChipInput chrome (icon prop) rather than hand-rolling the field. The highlight tracks the highlighted workspace by identity (a stored id, not a numeric position). An effect keeps that id pinned to a workspace that is actually in the current results — seeding the first result on open and re-seeding when a query filters the current one out — so even the default highlight is identity- stable. A live list change while the menu is open (shrink, grow, or reorder from a membership change or background refetch) therefore carries the highlight and Enter along with the same workspace instead of stranding them on whatever now sits at that row. `activeIndex` derives from the id and is the single source of truth for Enter, the visual highlight, and the scroll target. Search and highlight reset via an effect keyed on the menu-open state, so closing by any path (selecting a workspace, Escape, click-away) clears them — not only the Radix-driven close that routes through onOpenChange. Hover-highlight is wired to mousemove rather than mouseenter so a keyboard-driven scrollIntoView can't fire a synthetic enter that hijacks the keyboard selection, and composition keys are ignored while an IME is active so confirming a candidate can't switch workspace. * fix(settings): pad settings-sidebar scroll list so the last item's hover isn't clipped The scrollable settings-nav container had `pt-1.5` but no bottom padding, so the last item's rounded `--surface-active` hover pill was clipped against the container's overflow edge (visible on "Custom blocks", the final Enterprise item). Give it symmetric vertical padding (`py-1.5`) so the last row gets equal breathing room. Padding sits on the scroll container, not between items, so item spacing is unchanged. |
||
|
Waleed
|
c8fea6bd3b |
fix(knowledge): replace deletion-safety heuristics with a two-phase tombstone (#5884)
* fix(knowledge): replace deletion-safety heuristics with a two-phase tombstone PR #5883 merged an interim fix (sourceConfirmedEmpty bypassing two static safety heuristics) before this follow-up redesign was ready. This supersedes that approach with a properly general fix, matching how production sync systems (Entra Connect, SCIM/Entra ID deprovisioning, Cassandra/Couchbase tombstones) handle this exact problem: never let a single observation trigger an irreversible mass deletion, no matter how confident the signal looks. A document missing from a normal sync's listing is now soft-deleted (marked pending-removal) rather than hard-deleted immediately. It's only actually purged once a *later* sync confirms it's still absent. If it reappears in between, it's resurrected automatically — this self-heals a transient outage or a bad API response without needing to distinguish 'real' emptiness from 'ambiguous' emptiness at all, which is what the removed heuristics were trying (and failing) to do from a single observation. This removes shouldSkipEmptyListing, exceedsDeletionSafetyThreshold, and sourceConfirmedEmpty entirely — Google Sheets no longer needs a connector-specific bypass flag, since a genuinely trashed spreadsheet now reconciles through the exact same general path as every other connector, with no special-casing and no new misuse surface for future connectors. A forced fullSync still purges everything absent in one pass, preserving the existing 'trigger a full sync to force cleanup' escape hatch. Uses the existing (previously unused for individual documents) document.deletedAt column as the tombstone marker — no schema migration required. shouldReconcileDeletions (the isIncremental / listingCapped / listingTruncated gate) is unchanged; it still governs whether reconciliation may run at all. Resurrection runs unconditionally even when that gate is closed, since presence is trustworthy evidence regardless of whether the listing was complete. * fix(knowledge): resurrect atomically on content update, sweep tombstoned docs on connector teardown Two real gaps found by review, both stemming from the same root cause: other code paths assumed deletedAt IS NULL means 'the only real rows' and were never updated for the new tombstone semantics. - updateDocument's guard required deletedAt IS NULL, so a tombstoned document reappearing with CHANGED content failed its content update (rejected by the guard) while the separate resurrect step still cleared deletedAt regardless — the document became active again but kept serving stale pre-tombstone content. Fixed by clearing deletedAt as part of the same update statement and dropping the guard, so content and resurrection land atomically. - Both connector-teardown cleanup paths (the ConnectorDeletedException handler in sync-engine.ts, and the connector DELETE API route) only swept documents with deletedAt IS NULL, so pending-removal documents escaped cleanup entirely and were orphaned once their connector was gone. Fixed by including tombstoned docs in both sweeps — there's no future sync left to confirm or resurrect them once the connector itself is deleted. * fix(knowledge): don't resurrect a tombstoned document whose content refresh failed Critical race found by adversarial audit: resurrectIds was derived purely from 'externalId was seen in the listing', independent of whether the paired content update actually succeeded. If updateDocument threw (hydration failure, storage upload failure, or any other transient error) or the deferred-hydration fetch itself failed, the document's row was never touched — yet the separate unconditional resurrect step still cleared deletedAt for it anyway, reproducing the exact bug this PR fixes (visible again, serving stale pre-tombstone content), just triggered by a failed write instead of a gated one. Track externalIds whose refresh attempt failed (hydration rejection or write rejection) and exclude them from partitionSyncReconciliation's resurrectIds. A failed refresh leaves the document tombstoned as-is — not soft-deleted, not hard-deleted, not resurrected — so a later sync gets a clean retry instead of the row landing in an inconsistent state either way. * fix(knowledge): exclude fulfilled-but-unverified hydration outcomes from resurrection too Both bots independently found a third instance of the same bug class: when deferred hydration for an update fulfills but has no usable content (skipped as oversized, or an empty re-fetch), the code falls back to keeping the stored content as last-known-good and counts it unchanged — correct for an already-visible document, but for a tombstoned one it means content was never actually verified as current. That fallback wasn't added to failedExternalIds, so reconciliation still resurrected it with stale pre-tombstone content despite hydration never actually confirming anything. Fixed by adding both fallback branches to failedExternalIds, same as the rejected-promise cases. Verified across every connector's skippedReason call site that this only ever happens inside getDocument (the deferred hydration path already covered here) — no connector sets skippedReason directly in listDocuments, so there's no equivalent listing-time gap to fix. * fix(knowledge): force a full listing when pending-removal documents exist A subtler instance of the same bug class, this time architectural rather than a code-path gap: an incremental listing only includes documents whose content changed since the last sync. A tombstoned document that's still genuinely present at the source but unchanged would never appear in an incremental delta at all, so it could never be resurrected — and on a connector that runs incrementally from here on (its normal syncMode), it would stay tombstoned indefinitely with no self-correcting path, only a manual full resync. Added shouldRunIncrementalSync (extracted as a testable pure function, matching shouldReconcileDeletions' existing pattern) and a cheap existence check for any pending-removal document on this connector. Whenever one exists, this sync forces a full listing instead of an incremental one, guaranteeing every tombstoned document gets a real resurrect-or-confirm decision. This only affects which listing mode runs — it doesn't touch options.fullSync, so the deletion-safety grace period for other, unrelated documents in the same sync is unaffected. * fix(knowledge): bound tombstone-forced full syncs, resurrect kept docs on connector delete Two more real findings from this review round: - A document whose refresh keeps failing every sync (e.g. permanently oversized) never resurrects and never hard-deletes (it's present in the listing, just unreadable) — correct on its own, but it also never stops being counted by hasTombstonedDocs, so it would force a full listing for this connector forever, permanently disabling incremental sync on account of one stuck document. Bounded the check to the same RETRY_WINDOW_DAYS already used for the stuck-document retry sweep below: past the window, this connector stops forcing full syncs on the stuck document's account. The document itself is unaffected — it stays tombstoned either way, matching the existing 'last-known-good forever' tolerance this codebase already accepts for any document whose hydration keeps failing. - The connector-DELETE route's deleteDocuments=false path (kept docs) counted tombstoned documents but never resolved them one way or the other. With the connector gone, there's no future sync left to ever confirm or resurrect them, so they'd become permanent invisible orphans holding storage forever. Since 'kept' documents become normal standalone KB entries once detached from their connector, resurrect any pending-removal ones as part of that transition — consistent with what happens to their non-tombstoned sibling documents. * fix(knowledge): serialize reconciliation writes against a concurrent connector delete An independent adversarial audit (not just Greptile/Cursor) found the one genuinely critical gap 6 rounds of bot review missed: resurrect/ soft-delete/hard-delete writes applied raw document IDs snapshotted at the top of the sync, with no re-check immediately before the write. A connector-DELETE request choosing to keep documents detaches them (connectorId set to NULL) via the exact same FOR UPDATE lock on the connector row that this fix now also takes before applying any reconciliation write — serializing the two: whichever transaction commits first wins, and the loser's re-check sees the up-to-date connectorId and skips any document the other request already claimed. Without this, a sync racing a 'delete connector, keep documents' request could silently resurrect-then-strand or soft/hard-delete a document the user explicitly chose to keep, with a secondary effect of misclassifying it for storage-billing decrement (which keys off whether connectorId is still set). Also tightened the excludedDocs query: it previously required deletedAt IS NULL, so a document that was both userExcluded and tombstoned (reachable via excludeConnectorDocuments, which has no deletedAt filter) fell out of the exclusion set and could be silently un-excluded and re-indexed on reappearing. Dropped that requirement so userExcluded is honored regardless of tombstone state, consistent with how existingDocs/tombstonedDocs are already merged for classification. Documented (not code-changed) the remaining lower-severity finding: a document that outlives the 7-day hasTombstonedDocs bound on a persistently-incremental connector can stay unresolved indefinitely. Deliberately not hard-deleting it after the window expires — that would delete a document with no positive evidence it's actually gone, reintroducing the exact risk this whole design exists to avoid. It's already fully excluded from search/billing/listings either way, so this is an accepted, bounded, orphaned-row trade-off, not a correctness or security issue. * fix(knowledge): close remaining hard-delete race window, guard listing-time skip/drop resurrection Two more real findings, both closing gaps in the previous round's fixes: - The FOR UPDATE lock protected resurrect/soft-delete (applied inside the same transaction) but hardDeleteDocuments still ran after that transaction committed, using IDs snapshotted under the lock. A concurrent 'delete connector, keep documents' request could still detach those same documents in the gap between commit and the hardDeleteDocuments call. Added an optional expectedConnectorId parameter to hardDeleteDocuments/hardDeleteDocumentBatch — when provided, it re-verifies connectorId at the moment of the actual delete query, not just the caller's earlier snapshot. Every other caller is unaffected (parameter is optional, defaults to no filter). - Two more listing-time paths could resurrect a tombstoned document without ever verifying its content: a listing-time skippedReason short-circuits classification straight to 'unchanged' before the hash comparison ever runs, and empty non-deferred content classifies as 'drop' unconditionally regardless of hash. Both are now added to failedExternalIds when reappearing on an existing (possibly tombstoned) document, same treatment as the deferred-hydration equivalents from the prior round. * refactor(knowledge): dedupe retry-cutoff computation, extract ownership-filter helper /simplify pass: hoist the shared RETRY_WINDOW_DAYS cutoff into one computation reused by both the tombstone-retry bound and the stuck-document retry query, and pull the FOR-UPDATE-lock reconciliation's ID filtering into a pure, directly-unit-tested filterStillOwnedReconciliationIds function matching this file's existing convention for its other decision logic. * fix(knowledge): re-verify connectorId at the actual hard-delete, fix docsDeleted count Cursor findings: expectedConnectorId was only checked on the pre-transaction SELECT in hardDeleteDocumentBatch, not on the DELETE itself — the billing lookups and KB locking in between are async and can span a concurrent "delete connector, keep documents" request, so the delete (and its embedding cleanup) now re-verifies against a fresh in-transaction snapshot instead of the stale existingIds. Also fixed docsDeleted to use hardDeleteDocuments' actual returned count instead of the pre-filter candidate count, so a sync log no longer overreports deletions that expectedConnectorId skipped. /cleanup: dropped two comments that only restated the line below them. * fix(knowledge): re-verify connectorId in updateDocument's atomic write Greptile P1: updateDocument's content-update/resurrect write only checked document.id and archivedAt, never connectorId — despite connectorId being a parameter — so a document a concurrent "delete connector, keep documents" request already detached could still be matched, resurrected, and overwritten with connector-sourced content after the connector was deleted. Adds the same connectorId ownership check already used by the reconciliation transaction and hardDeleteDocuments in this PR. * fix(knowledge): close connectorId race in the stuck-document retry path Final independent audit: the stuck-document retry block selected candidate IDs filtered by connectorId, then reset their processing state and deleted their embeddings using that stale ID set with no re-check — the same SELECT-then-write race already patched in updateDocument and hardDeleteDocumentBatch. A concurrent "delete connector, keep documents" request could null out connectorId in between, so this now re-verifies ownership immediately before the embedding delete/document update/re-enqueue and only acts on documents still owned by this connector. * fix(knowledge): lock the connector row for stuck-doc retry ownership too Cursor + Greptile (same root cause, two reports): the previous round's fix re-checked connectorId via a separate SELECT before the embedding delete and processing-state reset, but a bare re-SELECT only narrows a TOCTOU window, it never closes it — a concurrent "delete connector, keep documents" request could still commit its detach in between. Wraps the ownership re-check and both writes in a transaction that takes the same knowledge_connector FOR UPDATE lock the DELETE route takes before nulling connectorId, so the two requests serialize instead of racing, matching the pattern already used by the reconciliation transaction elsewhere in this PR. |
||
|
Theodore LiandClaude Opus 4.8
|
5adf68f332 |
feat(slack): native Sim app trigger mode via the preview-gated slack_v2 block (#5892)
* feat(slack): native Sim app trigger mode behind the preview-gated slack_v2 block Restore the native Sim Slack app mode for the slack_oauth trigger using a single-credential-picker design. The trigger is only reachable through the preview-gated slack_v2 block, so the mode inherits that gate — no separate env flag. - Re-add SIM_SUBSCRIBED_EVENTS / SLACK_SIM_EVENT_OPTIONS derived exports. - Merge the trigger credential picker (credentialKind: 'any') so it lists Sim OAuth accounts and reusable custom bots together, mirroring the slack_v2 block. - Event dropdown narrows to the Sim app's subscribed events when an OAuth account is selected, all events for a custom bot (resolved client-side from the warmed credential list). - Deploy branch discriminates by the RESOLVED credential, not a UI field: a bot credential routes by credential id (custom app); otherwise validate the event against SIM_SUBSCRIBED_EVENTS, resolve the credential owner's token, derive routingKey from Slack team_id (auth.test), and route on the shared Sim app. - The ingest endpoint and team_id fan-out routing already existed on staging. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018asmKsWQ5Vi7T7wD9uHofz * fix(slack): address review — set credentialId, scope OAuth path to workspace, drop event narrowing Review round 1 (Greptile 4/5 + Cursor Bugbot): - deploy.ts: the native Sim-app (OAuth) branch now sets providerConfig.credentialId (runtime token resolution in the slack provider and credential-disconnect cleanup both key slack_app rows on it — without it, file downloads / reaction text fail and disconnect leaves the webhook active). - deploy.ts: resolve the OAuth credential through resolveTriggerCredentialId (workspace- and oauth-scoped) so a pasted foreign/other-tenant credential id can't bind to the workflow; use the resolved canonical id for token owner lookup + routing. - deploy.ts: a deleted/secretless custom bot credential (getSlackBotCredential → null but a service_account row exists) now returns the "reconnect the bot" error instead of the misleading "connected Slack account" message. - oauth.ts: drop the credential-based event-option narrowing. The shared dropdown framework doesn't revalidate a stored value when its dependency changes, so switching a custom bot → Sim account left an orphaned event that failed deploy with a 400. The event picker now offers all events; the deploy path is the authoritative gate. - tests: add broken-bot and workspace-not-resolvable cases; assert credentialId is set. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018asmKsWQ5Vi7T7wD9uHofz --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Waleed
|
efe1de315e |
fix(tables): sniff CSV delimiter and capture the real header row on import (#5888)
* fix(tables): sniff CSV delimiter and capture the real header row on import Table CSV import derived the delimiter purely from the file extension, so semicolon-delimited exports (European-locale Excel) parsed as a single column. Header derivation also read Object.keys(rows[0]), so with relax_column_count a ragged/sparse first data row dropped its trailing columns from schema inference. - Add detectCsvDelimiter: trial-parse the head against ,/;/tab/| and pick the candidate with the most columns (tie-break on row-width consistency), quote-aware so separators inside quoted cells don't skew the guess; extension is now only the fallback - Add sniffCsvDelimiterFromStream: memory-bounded (64KB) peek-then-replay for the streaming import paths so multi-GB files sniff without buffering - Capture the true header row via the csv-parse columns callback on every path, fixing dropped columns when the first row is short - Wire into the create route, append/replace route, background runner, client dialog preview, and parseFileRows (copilot) * fix(tables): rank CSV delimiter by row consistency and bound the peek buffer Addresses review findings on the delimiter sniffer: - Rank candidates by row-width consistency first (modal column count), then column count. Ranking on column count alone let a comma that appears in a semicolon file's unquoted header win over the real separator; the wide-but-ragged split now loses to the uniform one. - Move the partial-trailing-line trim into detectCsvDelimiter so every path (stream, client preview, parseFileRows) prepares the sniff sample identically and can't disagree on the delimiter. - Cap the stream peeker's detection copy at the sniff window via Buffer.concat's length arg and replay the buffered chunks by reference, so an oversized source chunk can't break the bounded-memory contract. * fix(tables): score CSV delimiter by width*consistency and dedupe headers Addresses round-2 review findings: - Score each delimiter candidate by modalWidth * consistency instead of consistency alone. Consistency-only let a separator that appears uniformly inside values (e.g. a pipe once per row) beat a real delimiter whose rows are legitimately ragged; the product rewards a split that is both wide and uniform, with width breaking ties. Genuinely symmetric files (two valid columns under either separator) fall back to the global-default candidate order. - Dedupe the captured header row to match the parser's record keys. csv-parse collapses duplicate column names onto one key (last value wins), so reporting the raw duplicates made schema inference invent a phantom empty column and could fail mapping validation. dedupeHeaders keeps first-occurrence order, case-sensitively, matching the emitted keys. * fix(tables): keep the final row when sniffing a complete CSV detectCsvDelimiter always trimmed to the last newline, so a complete file with no trailing newline lost its final data row from scoring — leaving the header alone and letting a header-widening separator beat the real one. It now takes a `complete` flag: the trim runs only for truncated prefixes, where a mid-record cut must be dropped. The stream sniffer passes it from `exhausted`, and the buffered callers (client preview, parseFileRows) pass it when the sample covers the whole file. * fix(tables): keep the double-cast annotation adjacent to its cast Hoist the csv-parse options out of the multi-line parse() call so the `// double-cast-allowed` annotation sits directly above the `as unknown as` token — the strict boundary audit only matches an annotation within three lines of the cast. Also simplify the stream sniffer's completeness check to `exhausted` (the loop only ends early on end-of-stream, so it already means the whole file fit the sniff window). * fix(tables): observe EOF at the exact sniff-window boundary The stream sniffer's read loop stopped as soon as the buffered size reached the sniff window, so a file whose size is exactly the window never triggered the extra read that observes end-of-stream — it was judged a truncated prefix and dropped its final newline-less row, disagreeing with the buffered callers (which mark that size complete). Read while size <= window so the boundary case sees EOF and `exhausted` (hence `complete`) is set correctly. Regression test added. * test(tables): use sleep helper instead of raw setTimeout promise |
||
|
Waleed
|
44749b85b3 |
improvement(credentials): actionable Shopify admin-token rejection message (#5891)
Shopify custom-app token verification faithfully surfaces a real 401 from Shopify, but the generic 'double-check it in Shopify' copy didn't tell users what to check. The #1 real cause is pasting the wrong secret (API key / API secret key) instead of the shpat_ Admin API access token, or using a token bound to a different store. - Add an optional per-provider invalidCredentialsHelp override on the token service-account descriptor; set it for Shopify to name the exact fix. - Move the error-code to message mapping out of the shared connect modal into the descriptor module (getTokenServiceAccountErrorMessage) so provider copy is inherited from the definition rather than hard-coded in the modal. - Add unit tests for the mapper (override, fallback, all codes). |
||
|
Waleed
|
78fb2c0679 |
chore(deps): bump next to 16.2.11 to clear security advisories (#5890)
* chore(deps): bump next to 16.2.11 to clear security advisories Patches SSRF, cache confusion, DoS, and middleware-bypass advisories (GHSA-89xv-2m56-2m9x et al.) affecting next < 16.2.11 across apps/sim, apps/docs, and packages/emcn. Excludes next/@next/env from the minimum-release-age gate until the 7-day window elapses on 2026-07-28. * chore(deps): drop aged-out typescript entries from release-age excludes typescript and @typescript/typescript6 passed the 7-day minimum-release-age gate (aged out 2026-07-15 and 2026-07-13), so their exclusions are no longer needed. Keeps @typescript/native-preview (permanent nightly builds) and the Pi packages (age out 2026-07-24). |
||
|
Waleed
|
3914d2ec24 |
improvement(landing): keyword-forward SEO copy on the home and enterprise pages (#5887)
* improvement(landing): keyword-forward SEO copy on the home and enterprise pages * fix(landing): sync homepage JSON-LD title and restore visible open-source claim |
||
|
Theodore Li
|
0199508706 | fix(custom-blocks): stop image icons blowing up on class-sized surfaces (#5878) | ||
|
Vikhyath MondretiandSim Pi Agent
|
4856ef396c |
feat(library): Best AI Agents for Customer Support Automation (#5879)
Co-authored-by: Sim Pi Agent <pi@sim.ai> |
||
|
Waleed
|
387fa378a5 |
fix(knowledge): purge trashed Google Sheets tabs on a normal sync (#5883)
* fix(knowledge): purge trashed Google Sheets tabs on a normal sync Trashing a spreadsheet made listDocuments return an empty listing, but the sync engine's zero-document guard skips deletion reconciliation whenever a listing comes back empty and documents already exist — it can't tell a genuinely empty source apart from a provider outage. For a single-spreadsheet connector, trashing its one source item empties the entire listing, so the guard always fired and the stale tabs never got cleaned up on a normal sync, contradicting the documented behavior. Add shouldSkipEmptyListing, mirroring shouldReconcileDeletions: a connector can now set syncContext.sourceConfirmedEmpty when it has positively confirmed the empty result against the source (not merely inferred it from an empty listing page), letting reconciliation proceed. The Google Sheets connector sets this flag when it confirms the spreadsheet is trashed via a direct Drive metadata lookup. No other connector sets it, so this doesn't change behavior anywhere else. * fix(knowledge): let sourceConfirmedEmpty also bypass the mass-deletion safety threshold The zero-document guard bypass alone wasn't enough: for a trashed spreadsheet with more than 5 tabs, reconciliation would proceed but the separate mass-deletion ratio guard (>50% deleted, >5 docs) still blocked the actual delete on a normal sync, requiring a forced full resync anyway. Extracted the ratio guard into exceedsDeletionSafetyThreshold, mirroring shouldSkipEmptyListing, so a connector's positive source confirmation bypasses both guards consistently. |
||
|
Waleed
|
874e742a47 |
fix(connectors): purge archived/deleted source items in KB connectors (#5880)
* fix(confluence): exclude archived pages from KB connector listings so reconciliation purges them
* fix(connectors): purge archived/deleted source items across seven more KB connectors
The sync engine only purges a knowledge-base document when its source item is
absent from a full-sync listing, so any connector that keeps listing
archived/trashed/canceled items never drops them. An audit of all 51 connectors
found seven with this bug:
- asana: list only non-archived projects (the API returns both when `archived`
is omitted), so tasks under archived projects stop being re-listed
- google-sheets: skip a spreadsheet Drive reports as trashed, which stays
readable by id for 30 days before the Sheets call starts 404ing
- incidentio: exclude canceled incidents by default (cancelling is incident.io's
documented stand-in for deletion), with an explicit opt-in to sync them
- outlook: exclude Deleted Items from the all-mail listing, which Graph
otherwise includes
- servicenow: drop retired knowledge articles, which the Table API returns with
no implicit state filter
- webflow: drop archived CMS items, which the staged items endpoint always
returns and offers no way to filter
- youtube: drop playlist entries whose video was deleted or made private, which
the API keeps returning as placeholder items
Every exclusion keys off an explicit non-current signal and fails open on a
missing field or a failed metadata read, since wrongly excluding a live item
would hard-delete it. Explicit user filter selections are still honoured
verbatim; the new defaults apply only when nothing is configured.
Also flag truncated listings as capped in asana, outlook, and servicenow. All
three silently cut a listing short at their configured item cap without setting
`syncContext.listingCapped`, so reconciliation read the untraversed tail as
deleted at the source and hard-deleted it.
* fix(asana): honour the pinned-project exception on the task rehydrate path
listDocuments deliberately keeps syncing a project the user pinned via the
`project` config field even once it is archived, but getDocument ignored
sourceConfig and applied the all-parents-archived exclusion unconditionally.
For a pinned archived project the listing kept emitting its tasks while every
hydration returned null, so new tasks were dropped as empty and already-indexed
ones were frozen at their last content.
isTaskUnderActiveProject now takes the pinned project gid and keeps any task
reachable through it, matching the listing exactly. The unpinned path is
unchanged and still fails open on missing/non-boolean archived values.
* fix(connectors): key removal on explicit source signals, never on absence
Follow-up to the connector purge fixes, from an independent audit.
YouTube inferred deletion from absence: a playlist entry whose id was missing
from a `videos.list` response was dropped, so a well-formed 200 that returned 49
of 50 requested ids hard-deleted the 50th. Playlist items instead carry a
documented `status.privacyStatus`, available as a free part on a call the
connector already makes, so the extra `videos.list` request is gone along with
its quota-failure and pagination-wedge risks. An item is now excluded only on an
explicit `private`; missing, empty, or unrecognized values keep it.
ServiceNow read every record through a guard requiring a string `sys_id`, but
the listing requests `sysparm_display_value=all`, under which every field —
`sys_id` included — comes back as `{display_value, value}`. The guard rejected
every record, so the retired-article filter was unreachable and the sys_id
object would have leaked into `externalId` and `title` had it not been. Records
are now read through the existing `rawValue` normalizer, which accepts both wire
shapes, and the fixtures use the shape the API actually returns.
Also: resolve the ServiceNow cap ambiguity with `X-Total-Count` so a table that
ends exactly on a page boundary is not read as truncated; stop the Google Sheets
comment claiming a purge the engine's zero-document guard prevents; and assert
the Outlook junk-mail invariant instead of comparing a constant to itself.
Document the behavior change: content archived, retired, or trashed at the
source is now removed from the knowledge base, and restoring it re-ingests it.
|
||
|
Waleed
|
26e4c30888 |
fix(ci): unblock the deploy chain and remove the promotion-dedup gate (#5881)
* fix(ci): unblock the deploy chain — explicit need results on jobs downstream of test-build test-build's needs chain now contains dedup-promotion, which is skipped on every push event. A skipped transitive ancestor fails the implicit success() on downstream jobs, so migrate, promote-images, create-ghcr-manifests, process-docs, and create-release all cascade-skipped on push runs — blocking staging and main deploys (no ECR tag push, no CodeDeploy). Each of those jobs now uses !cancelled() plus explicit needs.<job>.result == 'success' checks, preserving their original semantics while ignoring the skipped ancestor. * chore(ci): remove the promotion-dedup gate — savings don't justify deploy-graph complexity The bespoke dedup job hand-rolled what content-addressed caching solves idiomatically, saved only ~$50-90/mo, and its needs edge just caused the deploy-chain skip incident. test-build returns to its original shape; the explicit need-result conditions on the deploy chain stay as hygiene. |
||
|
Waleed
|
227cd65f41 |
fix(mcp): bound and retry OAuth start so a transient stall recovers instead of a blank popup (#5874)
* fix(mcp): bound and retry OAuth start so a transient stall recovers instead of a blank popup Empirically root-caused a blank/stuck authorize popup: the provider (planetscale) and our guarded OAuth fetch are both fast (120/120 legs clean from staging), and /oauth/start uses local AES encryption with no Redis lock in its path — so the intermittent hang is the same transient headers-then-stalled-body class we've documented for CDN-fronted MCP hosts (a per-connection stall a fresh attempt dodges), which /oauth/start had no server-side bound against. - Bound every /oauth/start step with the shared timedStep helper (extracted from the callback route, now used by both) + an entry log, so a stalled step surfaces as a labeled error instead of hanging the request (and the browser popup) to the client's 30s timeout. - Retry mcpAuthGuarded once on a bounded 12s timeout: a fresh attempt gets a fresh connection and recovers from the transient stall automatically (two 12s attempts stay under the client's 30s deadline). McpOauthRedirectRequired (the success signal) and DCR-unsupported errors are never retried. Adds OauthStepTimeoutError + makeTimedStep to the shared oauth barrel and test mocks. * fix(mcp): drop the unsafe OAuth-start retry; fail fast without error-logging success Review fixes on the bound+retry change: - Removed the mcpAuthGuarded auto-retry. timedStep can't cancel the loser, so a lingering first attempt shares this server's OAuth row and could overwrite the retry's PKCE verifier / state after the client already got the second authorize URL, breaking the callback. Recovery is now fail-fast (504) → the user re-clicks, which is a clean fresh flow (fresh connection dodges the transient stall) with no shared-state race. - Catch McpOauthRedirectRequired (the success signal) INSIDE the bounded step and return it as a value, so a successful authorize is no longer error-logged as 'OAuth step failed'. - Tighten step budgets (5s DB x3 + 12s auth = 27s) to stay under the client's 30s /oauth/start deadline. * fix(mcp): route all bounded-step timeouts to the 504 handler Move OauthStepTimeoutError handling to the outer catch so a DB-step timeout (loadServer/getOrCreateOauthRow/loadPreregisteredClient) returns the same fast 504 'try again' as the auth step, not a generic 500. Documents that the fresh retry is race-safe: the callback correlates on the state nonce, so a lingering timed-out attempt overwriting the row's state only yields a clean invalid_state on the user's fresh authorize URL — never silent corruption. * fix(mcp): bound the setOauthRowUser write too so no step escapes the budget The user-stamp write was the one DB op left unbounded on the start path; wrap it in timedStep(DB_STEP_MS) so every step stays inside the sub-30s budget and its timeout routes to the same 504. * fix(mcp): shrink OAuth-start step budgets to fit the 30s client deadline with 4 DB steps Bounding setOauthRowUser added a fourth possible DB step, so 4x5+12=32s exceeded the client's 30s /oauth/start abort. Lower DB steps to 4s and auth to 10s: 4x4+10=26s worst case, leaving margin for middleware/network. Comment corrected. |
||
|
Waleed
|
dda602a367 |
improvement(tests+ci): phase 3 — shared-mock convergence completion and CI runner-minute cuts (#5875)
* chore(ci): cut redundant runner minutes — dedup promotion-PR test runs, companion-pr-check concurrency, right-size trivial jobs
- ci.yml: new dedup-promotion gate skips the pull_request test-build on
staging/main-headed promotion PRs only when the merge tree provably equals
the head tree (empty base delta over the merge base) AND the push-event run
at the same sha passed its test jobs (polled). Fail-open on any error/
timeout/failure, job-level skip only (skipped job reports Success); verified
no required status checks are configured on main/staging rulesets.
Measured 39 duplicate PR runs / 5.15 days (~227/mo) at ~7.1 min each on
8vcpu (~57 vcpu-min), probe costs ~9 vcpu-min worst case on 2vcpu.
- companion-pr-check.yml: per-PR concurrency group with cancel-in-progress so
superseded synchronize/edit runs stop; no paths filter (check depends on PR
body + cross-repo state, not changed files).
- detect-version and check-docs-changes: 4vcpu -> 2vcpu Blacksmith runners
(pure shell / depth-2 checkout + path filter only).
* improvement(testing): complete stateful shared mocks for env, urls, redis-config, environment-utils
Shared mock infrastructure for vitest isolate:false convergence:
- packages/testing/src/mocks/env.mock.ts: stateful envMock (live env proxy, setEnv/resetEnvMock, process.env fallback)
- packages/testing/src/mocks/urls.mock.ts: complete urlsMock with real-behavior default impls + resetUrlsMock
- packages/testing/src/mocks/redis-config.mock.ts: adds getRedisConnectionDefaults + resetRedisConfigMock
- packages/testing/src/mocks/environment-utils.mock.ts: new environmentUtilsMock + fns + reset
- contract tests: env.mock.test.ts, urls.mock.test.ts, redis-config.mock.test.ts, environment-utils.mock.test.ts
- packages/testing/src/mocks/index.ts: barrel exports
- apps/sim/vitest.setup.ts: global installs for env, urls, redis, environment/utils
- real-module tests unmocked: lib/core/config/env.test.ts, lib/core/config/redis.test.ts, lib/core/utils/urls.test.ts, tools/index.test.ts (urls)
- stubEnv/process.env fallout migrated to setEnv: lib/webhooks/providers/{revenuecat,rootly,instantly}.test.ts, app/api/auth/oauth2/authorize/route.test.ts
* improvement(tests): drop redundant local mocks in executor/tools/providers and misc dirs (shared-worker readiness)
* improvement(tests): drop redundant local mocks in app routes (shared-worker readiness)
* improvement(tests): drop redundant local mocks in lib (shared-worker readiness)
* fix(ci+testing): live base-tip recheck before dedup skip; prod-aware urls mock fallbacks
- the dedup gate re-verifies merge-tree equivalence against the LIVE base
tip at decision time, closing the window where the base branch gains real
commits during the poll (frozen BASE_SHA check alone was stale)
- the urls mock's getBaseUrl protocol prefix and getBaseDomain parse
fallback now follow the shared isProd flag, mirroring the real module
* fix(ci+testing): fail-closed nojobs fallback in dedup gate; TLS-aware redis defaults mock
- the dedup gate no longer infers coverage from overall run conclusion when
no 'Test and Build /' jobs match — a renamed or skipped test job now runs
the tests instead of skipping them
- the shared getRedisConnectionDefaults mock mirrors the real TLS resolution
(rediss:// to a raw IP requires REDIS_TLS_SERVERNAME and yields
tls.servername)
* fix(ci): keep polling while nested test jobs have not appeared yet
An in-progress push run lists its reusable-workflow jobs only after the
caller starts; nojobs is now terminal (fail closed) only once the run has
completed without them.
|
||
|
Waleed
|
9d8e14ce9f |
improvement(tests): converge env-flags mocks onto a complete shared mock (#5871)
* improvement(tests): converge env-flags mocks onto a complete shared mock
* fix(tests): drop the repo's only bare vi.mock automock
A bare vi.mock('drizzle-orm') automock colliding with factory mocks of the
same module in a shared worker corrupts vitest's mock registry (upstream
vitest-dev/vitest#10290 / #10145, reproduced in isolation). The global
factory mock already covers this suite.
* chore(tests): drop defensive resets in non-mutating suites, merge sequential setEnvFlags calls
* fix(tests): run providers/utils cases sequentially over shared env-flags state
|
||
|
Waleed
|
02bc8f1828 |
feat(ci): warm Next.js builds via Turbopack persistent cache on a sticky disk (#5869)
* feat(ci): warm Next.js builds via Turbopack persistent cache on a sticky disk - enable experimental.turbopackFileSystemCacheForBuild behind NEXT_TURBOPACK_BUILD_CACHE so only the CI check build opts in; production image builds stay on the default cold path until the feature stabilizes - mount ./apps/sim/.next/cache as a Blacksmith sticky disk (cache-mount) instead of actions/cache: the turbopack cache is ~5 GB, which a sticky disk mounts in ~1s while an actions/cache round-trip would eat the win - measured locally: 105s cold compile vs 22s warm (4.8x) * chore(ci): drop the superseded actions/cache comment and restore trailing newline |
||
|
Waleed
|
2b5a92a3c8 |
feat(auth): org session policies — lifetime/idle limits, org-wide revocation (#5862)
* feat(auth): org session policies — lifetime/idle limits, org-wide revocation, cookie-cache versioning * refactor(auth): consolidate session-policy clamp semantics, shared security-policy version module, canonical bounds, docs * polish(session-policy): cleanup pass — muted field labels, spinner reset, state tracker, response-seeded baseline, comment trims * fix(session-policy): govern member sessions by membership (closes revoke cookie-cache hole), normalize createdAt, remount on org switch, sync audit mock * fix(session-policy): clamp pre-join sessions on invite acceptance, normalize expiresAt, sync unified nav test * fix(session-policy): invalidate membership cache on removal/transfer, spare impersonator sessions in revoke-all, raise idle floor to 2x cookie window * fix(session-policy): resolve governing org by membership only — activeOrganizationId goes stale across transfer/leave * fix(session-policy): atomic policy save + eager clamp, asymmetric membership TTL, admin-add cache invalidation * fix(session-policy): org-scoped cookie version string, atomic revoke delete+bump * fix(session-policy): plan-gate effective policy so downgraded orgs stop enforcing automatically * chore(session-policy): drop dead bumpSecurityPolicyVersion helper — call sites bump transactionally * fix(session-policy): unify join paths on applySessionPolicyToNewMember; final audit polish (dead exports, response bound, test name) |
||
|
WaleedandMarcus Chandra
|
8cce661a37 |
feat(api): proxyUrl for residential/custom proxy egress on the API block (#5867)
* feat(api): add proxyUrl for residential/custom proxy egress on the API block The HTTP/API block egresses from the app runtime's fixed datacenter IPs via secureFetchWithPinnedIP, so targets behind Cloudflare/WAF that block datacenter IPs (e.g. state .gov license portals) return 403/429 even when the identical request works from a browser. There was no way to route a request through a residential/custom proxy. Add an optional `proxyUrl` field (Advanced) to the API block. When set, the request routes through the given http:// proxy so it egresses from that proxy's IP. Security: - validateAndPinProxyUrl resolves the proxy host's DNS and blocks private/reserved/loopback IPs (same SSRF guard as target URLs), then pins the connection by rewriting the host to the resolved IP (creds/port preserved), closing the DNS-rebinding window. - Restricted to the http: proxy scheme (https/socks rejected) so host pinning is safe without breaking TLS-to-proxy SNI. - Target-IP pinning is intentionally bypassed when a proxy is active (the proxy resolves the target); target URL validation still runs. Threaded block field -> http tool param -> formatRequestParams -> executeToolRequest (validate + pin) -> secureFetchWithPinnedIP, which swaps its pinned Node agent for HttpsProxyAgent/HttpProxyAgent (keyed off target protocol) when proxyUrl is set. * docs(api): document the Proxy URL advanced field and steer proxy credentials to env vars * fix(api): reject loopback/private proxy hosts unconditionally, closing the self-hosted rebinding gap * chore(api): tighten proxy-path inline comments --------- Co-authored-by: Marcus Chandra <mzxchandra@gmail.com> |
||
|
Waleed
|
21ac7b1bba |
improvement(tests): db-mock migration tranche 4 — billing, webhooks/execution/logs, routes/misc (final) (#5866)
* improvement(tests): db-mock migration tranche 4 — billing, webhooks/execution/logs, routes/misc (final) * fix(tests): route agent-handler MCP server rows through queueTableRows |
||
|
Waleed
|
51307c40c0 |
improvement(tests): db-mock migration tranche 3 — lib/workflows, lib/copilot remainder, ee/core/misc (#5864)
* improvement(tests): db-mock migration tranche 3 — lib/workflows, lib/copilot remainder, ee/core/misc * improvement(tests): use the shared notLike operator in idempotency cleanup suite |
||
|
Waleed
|
52659d4a89 |
improvement(tests): db-mock migration tranche 2 — copilot, mothership, workspaces, connectors, mcp (#5863)
* improvement(tests): db-mock migration tranche 2 — copilot, mothership, workspaces, connectors, mcp * fix(testing): drain unconsumed ...Once overrides in resetDbChainMock vi.clearAllMocks clears call history only — a ...Once override queued by a previous test but never consumed survived into the next test. resetDbChainMock now mockReset()s every shared spy and stable wrapper, which restores the original implementation AND drains once-queues. |
||
|
Waleed
|
62a8ce4953 |
improvement(tests): db-mock migration tranche 1 — knowledge, billing/org, workflows/background (#5861)
* improvement(tests): migrate knowledge, billing/org, and workflows/background suites off private @sim/db factories
* improvement(tests): db-mock migration tranche 1 — knowledge, billing/org, workflows/background
- migrate 19 suites off private vi.mock('@sim/db') factories onto the shared
dbChainMock + queueTableRows API (net ~-1,260 lines of bespoke chain
plumbing); resolves the known shared-worker rival pairs (knowledge
processing-queue vs api utils; billing polluters; persistence/utils vs
schedules/deploy)
- add .for() to the mock's limit builder (drizzle .limit(1).for('update'))
with a contract test
- document the join-table queue fallback footgun on queueTableRows
|
||
|
Waleed
|
c083be9def |
improvement(auth): bump better-auth to 1.6.23 and add trusted-proxy client IP resolution (#5857)
* improvement(auth): bump better-auth to 1.6.23 and add trusted-proxy client IP resolution * chore(billing): record checkout-scope mirror re-verification against @better-auth/stripe 1.6.23 * chore(deploy): expose AUTH_TRUSTED_PROXIES in docker-compose.prod and Helm chart |
||
|
Waleed
|
49d3804bee |
fix(ci): save the Next.js build cache every run instead of freezing it at the lockfile key (#5859)
* fix(ci): save the Next.js build cache every run instead of freezing it at the lockfile key
The cache key was only runner.os + bun.lock hash, and GitHub caches are
immutable per key: the first run after a lockfile change saved the cache
once, then every later run hit the primary key and skipped the save
('Cache hit occurred on the primary key ... not saving cache'), so builds
compiled against a cache stale since the last lockfile bump. Suffix the
key with the commit SHA so each run saves its refreshed cache, and
restore via prefix match to the most recent entry.
* fix(ci): make the Next.js cache key unique per run attempt so reruns can save too
|
||
|
Waleed
|
3215a12da9 |
improvement(testing): consolidate @sim/db mocks into one table-aware chain mock (#5856)
* improvement(testing): consolidate @sim/db mocks into one table-aware chain mock - back databaseMock and dbChainMock with the SAME db instance so a module bound to either export hits identical chain fns — rival-mock divergence between the two @sim/testing db mocks is structurally impossible now - add queueTableRows(table, rows): FIFO per-table select routing keyed by schema-mock table identity, consumed at where() materialization and resolved by every downstream terminal (limit/orderBy/groupBy/for/joins) - delete createMockDb (duplicate chain implementation, no external users) - migrate the five suites that hand-rolled table routing + databaseMock delegation (billing plan/usage/usage-log, admin dashboard-organizations, workspaces/utils) onto queueTableRows; net -295 lines - add a contract test for the mock itself and a test script to @sim/testing so its tests actually run under turbo * fix(testing): harden table routing — join-table queues, direct-await from, mutation isolation - track the chain's tables as a list (from + joins) so rows queued for a join-only table route correctly; from-table queue checked first - make the from/join builder a lazy thenable so awaiting a select with no where clause resolves queued rows (dequeue at await, never double-consumed) - update/delete/set clear the routing context so a mutation's where() can never consume rows queued for a select - document the left-to-right chain-construction assumption; contract tests for all three behaviors * fix(testing): close routing over each chain's own tables for direct-await builders * refactor(testing): move all chain routing state into per-chain closures - shared dbChainMockFns entries become pure spy/override ports: their default implementation returns a sentinel that chain-local builders replace, while any mock* override on the spy wins verbatim - each select().from() captures its own immutable table list; where(), joins, terminals, and direct awaits all resolve through that closure, so partially-built chains for different tables interleave without cross-talk - no module-level routing state remains * fix(testing): lazy queue consumption at resolution and wrapper restore on reset - each chain holds one lazy rows supplier: the queued set is dequeued only when a default thenable actually resolves, so a chain answered by a per-test terminal override leaves its queued rows for the next chain - resetDbChainMock also mockReset()s the stable db entry-point wrappers so direct overrides on databaseMock.db.* cannot outlive a suite |
||
|
Theodore LiandClaude Opus 4.8
|
ae7b5eff6c |
feat(copilot): service account setup & reconnect in chat (#5786)
* feat(copilot): add service_account_get_setup_link handler
Resolves a loosely-specified integration name to the catalog slug whose
detail page mounts ConnectServiceAccountModal, and returns
`/integrations/{slug}?connect=service-account`. The agent surfaces it via
the existing <credential type="link"> tag, so the user gets a Connect
button and supplies the key material in Sim's own form — the agent never
handles the secret.
Exact matches beat fuzzy ones so a caller naming a specific service lands
on it (gmail stays Gmail rather than collapsing to Drive), and family
names resolve through an explicit canonical map rather than to whichever
member sorts first.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Phx1MLjf8Ui3M3VpwisZds
* fix(copilot): reject service account ids in oauth_get_auth_link
The fuzzy provider match falls back to substring containment, so
`slack-custom-bot` contains `slack` and resolved to the Slack OAuth
service. The tool then returned a personal-OAuth authorize URL and
reported success — a user who asked for a shared custom bot got a
Connect button that linked their own account instead. Every service
account id degraded this way (notion-, salesforce-, zoom-, linear-),
always silently.
Guard runs before the fuzzy pass and points at
service_account_get_setup_link. Keys off the id being a service-account
id, not off the integration offering one, so `slack` and `notion` still
resolve for OAuth.
Moves the narrowing predicate out of the integration catalog module so
callers that need only the predicate skip the integrations.json load and
the OAUTH_PROVIDERS walk.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Phx1MLjf8Ui3M3VpwisZds
* feat(copilot): open the service account form in-chat instead of linking out
The tool handed back a /integrations/{slug}?connect=service-account URL,
so accepting the agent's offer navigated away from the conversation that
asked for the credential. Adds a `service_account` credential tag that
mounts ConnectServiceAccountModal over the chat; setup_url stays as the
headless/MCP fallback.
The tag carries a provider and no value — the secret is typed into Sim's
own form and never enters the transcript — so the validator gets a branch
alongside secret_input/sim_key rather than falling through to the
value-required check.
Extracts useServiceAccountConnectTarget so the chat and the integrations
page share one source of truth for the connect label and the preview
gate. Custom Slack bots ride the slack_v2 flag; without the shared gate
the chat would have surfaced a setup form the integrations page hides.
Modal is lazy-loaded off the deep path (not the barrel) to keep three
provider-specific setup forms out of the chat's initial chunk.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Phx1MLjf8Ui3M3VpwisZds
* fix(copilot): gate service account tool on the same preview flag as the UI
The in-chat connect button hides itself when the provider's gating block
is preview-hidden (a custom Slack bot needs slack_v2). The tool didn't
check this, so it returned success for slack-custom-bot even when slack_v2
was preview-gated off — the agent said "here's the setup form" and the
button silently rendered nothing, leaving the user with no form at all.
Adds getServiceAccountGatingBlockType as the single source for the
provider→gating-block mapping, consumed by both the tool (server-side, via
getBlockVisibilityForCopilot) and the connect hook (client overlay). When
the gating block is hidden the tool now fails with a fall-back-to-OAuth
message instead of promising an invisible form.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Phx1MLjf8Ui3M3VpwisZds
* feat(copilot): make the tool own service-account discovery
Removes the VFS auth-metadata exposure and returns connectNoun from the
service_account_get_setup_link result instead. The VFS aggregate was a
second, viewer-independent source of truth that couldn't agree with the
per-viewer preview gate (it always hid slack-custom-bot, even for viewers
with slack_v2 revealed, while the tool accepts it for them). The tool now
resolves the provider, applies the per-viewer gate, and returns either the
in-chat button + connectNoun or a fall-back-to-oauth error — one source of
truth. connectNoun stays DRY via getServiceAccountConnectNoun, shared with
the connect-button label.
* feat(copilot): make service-account setup a direct tag, no tool
The agent now emits the service_account credential tag directly from
intent — like secret_input — instead of round-tripping through a tool.
Removes service_account_get_setup_link (handler, registration, display
title, Go tool def) and restores auth.serviceAccount as the VFS discovery
field so the agent knows which providers support a service account.
The link-vs-tag distinction was the wrong axis: only oauth needs a tool,
because its button carries a minted URL that can't be reconstructed. The
service_account tag carries just a provider name the agent already knows,
so it needs no tool — discovery lives in the VFS (auth.serviceAccount,
GA-only, so slack's preview-gated custom bot is never proactively
offered), and the per-viewer gate lives in the renderer, which renders
nothing when a provider isn't available for the viewer (no OAuth
fallback — a shared credential and a personal one are different intents).
oauth_get_auth_link's service-account-id guard now points at the tag.
* feat(copilot): support service-account reconnect from chat
Reconnect had no service-account path — it required oauth_get_auth_link
and a link tag for every repair, so rotating a workspace service account
either errored or pushed the user through OAuth.
The service_account tag now takes an optional credentialId; when present
the renderer opens the modal in reconnect mode (rotates the secret on that
credential in place, id preserved) and labels the button "Reconnect X".
credentials.json now carries each credential's type (oauth vs
service_account) so the agent can branch: service accounts reconnect via
the tag + credentialId, oauth via oauth_get_auth_link as before.
* fix(copilot): coherent service-account rejection in oauth_get_auth_link
Review round on #5786:
- The service-account-id guard threw into the generic catch, which
overwrote its recovery hint with a "connect manually" message and a
workspace oauth_url — contradictory signals. It now returns a coherent
failure directly, before the try, with no oauth_url.
- Normalize spaces/underscores before the check so a readable form
("slack custom bot", "google service account") is caught too, not
passed to the fuzzy OAuth resolver.
- Remove listServiceAccountIntegrationNames — dead after the tool was
removed (its only caller was the deleted handler's error copy).
* fix(copilot): service-account discovery must un-gate after the block GAs
Review round on #5786: describeServiceAccountForOAuthProvider used
`getBlock(...)?.preview ?? true`, which treats a GA'd gating block — one
that dropped its `preview` flag, exactly slack_v2's documented migration —
as still gated, so the custom bot would stay omitted from VFS discovery
forever after GA even though the UI shows it. Reuse the canonical
isHiddenUnder(null, block) predicate instead, so a non-preview block is
visible. Adds service-account-gate.test.ts covering preview → omit, GA →
include, and missing → fail-closed with a mocked getBlock (the block
registry is globally stubbed, so the real slack_v2 preview flag isn't
observable through serializeIntegrationSchema).
* fix(copilot): align SA resolver normalization and reject blank credentialId
Review round on #5786:
- resolveServiceAccountIntegration only lowercased/trimmed, but the
oauth_get_auth_link guard normalizes spaces/underscores to hyphens
before rejecting a service-account id and steering the agent to a
service_account tag. The chat renderer then couldn't resolve those same
readable forms ("slack custom bot", "notion_service_account") and
rendered nothing. Apply the same normalization to the id lookups (raw
query still used for display-name matches).
- service_account tag validation rejected a blank provider but allowed a
whitespace-only credentialId, which is truthy — the renderer took the
reconnect path and tried to rotate a non-existent credential. Reject a
blank/whitespace credentialId.
* refactor(credentials): route the editor SA picker through the canonical connect hook
The workflow-editor credential selector (from #5800's merged picker) resolved
its service-account setup surface inline and mounted the modal with NO preview
gate — so a `credentialKind: 'service-account'` picker would offer a custom-bot
setup even when slack_v2 is preview-gated off, the leak the integrations page
and chat already guard against.
Route it through the shared useServiceAccountConnectTarget hook (the same
resolver chat and the integrations page use): suppress the setup action when
`hidden`, and use the hook's vendor-accurate label ("Add private app token",
"Set up a custom bot") as the default connect-row copy. Existing service
accounts stay selectable; the per-block `credentialLabels.serviceAccountConnect`
override still wins. One resolver now backs all three SA connect surfaces.
* docs(add-block): document credentialKind and the service-account picker
The add-block skill had no mention of credentialKind — the mechanism (#5800)
that controls whether an oauth-input offers OAuth, service-account, or a merged
picker — and its example was a plain oauth-input mislabeled "Service Account".
Documents the three credentialKind modes, that a default oauth-input already
lets users select an existing service account (they fold in), and the
credentialLabels / allowServiceAccounts companions. Regenerates the .claude and
.cursor projections.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
WaleedandMarcus Chandra
|
6f33a9485b |
feat(workflows): IDE-style reference viewer for workflows (#5854)
* feat(workflows): add IDE-style reference viewer for workflows
Adds a "Show references" viewer so you can see how workflows connect:
which workflows call a given workflow ("Used by") and which workflows it
calls ("Uses"), rendered as recursive, clickable trees.
- Opened via Cmd/Ctrl+click on a sidebar workflow row and a "Show
references" context-menu item.
- Resolves references through both the workflow / workflow_input blocks
(reusing isWorkflowBlockType) and published custom blocks
(custom_block_* -> source workflow), scoped to the workspace.
- Builds the whole workspace reference graph once from live workflow_blocks
state; cycle-safe DFS marks A->B->A loops as (cycle) leaves.
- Contract-bound GET /api/workflows/[id]/references with workspace-level
authz; React Query hook gated to fetch only when the modal opens.
- Unit tests for the pure graph/tree logic (cycles, self-refs, dangling
drop, custom-block + workflow_input resolution) and route tests
(401/400/403/200).
* fix(workflows): correct reference resolution for active mode, cycles, cache, and graph size
Addresses review findings on the reference viewer:
- Resolve the workflow-block child via resolveActiveCanonicalValue (the
shared SOT) instead of basic-first `||`, so an advanced-mode block whose
old basic workflowId value lingers resolves to the active manual value.
- Keep self-references (A -> A) and render them as a cycle leaf instead of
dropping the edge, matching the cycle-safe viewer's purpose.
- Set the references query staleTime to 0 so reopening the always-mounted
modal refetches live editor state instead of serving a stale cached graph.
- Bound converging paths: a node already expanded elsewhere in the tree is
emitted once more as a plain leaf (edge stays visible) rather than
re-expanded, so a densely reconverging graph can't grow exponentially.
* improvement(workflows): align reference viewer auth, coverage, and UI with platform conventions
- authorize via authorizeWorkflowByWorkspacePermission and derive the
workspace server-side (404/403 semantics; drops the client-supplied
workspaceId query param from the contract, hook, and modal)
- add workflow-tool call edges: workflow_input tools inside tool-input
sub-blocks now appear in both trees; non-call selector shapes stay
deliberately excluded (documented against remap-internal-ids)
- restore native cmd/ctrl+click open-in-new-tab on sidebar workflow rows;
references stay reachable from the context menu
- mount ReferencesModal on demand per row, deleting the prevIsOpen reset,
the enabled knob, and the staleTime-0 workaround (now 30s)
- align the tree with design tokens (--text-icon, --surface-hover, px-4
text gutter) and drop the hardcoded brand hex
- escape LIKE wildcards in the custom_block_ prefix match; import
MAX_CALL_CHAIN_DEPTH instead of mirroring it; remove dead fallbacks,
the duplicate not-found scan, and the redundant custom-block row map
* improvement(workflows): final polish on the reference viewer
- drop the vestigial isOpen prop (conditional mount owns visibility)
- unify on the emcn Workflow icon in tree rows
- inline the static className and derive nodes without an annotation
- remove one restating test comment
* fix(workflows): resolve tool references by active canonical mode and keep the reference cache live
- workflow_input tools inside tool-input now resolve basic/advanced via the
index-scoped canonicalModes override, mirroring execution (Cursor finding)
- staleTime back to 0: no mutation invalidates this key, so a reopen must
background-refetch; on-demand mounting keeps the cached tree painting
instantly (Greptile P1)
- modal header uses the em-dash label-entity convention; tree items carry
aria-level instead of a static aria-selected
* fix(workflows): cover legacy workflow-typed tools and retry depth-truncated expansions
- toolInputCallees matches both workflow tool type spellings via
isWorkflowBlockType and passes the tool's own type as the legacy
canonicalModes fallback, matching providers/utils resolution
- a depth-capped expansion no longer poisons the expanded set, so a
shallower path re-expands the node in full (Cursor finding)
- the allowed-but-workspaceless auth branch now returns 403, not the
authz result's 200
- tests: legacy tool type + per-tool index-scope isolation, diamond
re-expansion with a real subtree, depth ceiling, shallow-path retry
---------
Co-authored-by: Marcus Chandra <mzxchandra@gmail.com>
|
||
|
Waleed
|
5338485ac1 |
feat(sidebar): add Slack Community link to help dropdown (#5858)
* feat(sidebar): add Slack Community link to help dropdown * chore(sidebar): use existing Slack community invite link |
||
|
Theodore Li
|
d7d42faff8 |
fix(mothership): stop-button transitions freeze in place (#5838)
* fix(mothership): freeze the transcript on user stop instead of settle-scrolling * fix(mothership): swap stopped row into the shimmer slot and floor the sizer min-height * fix(mothership): detach auto-scroll on stop and dead-band the sizer floor * improvement(mothership): net-zero settle — equal tail regions, drained floor, one growth signal * fix(mothership): follow the post-stop drain to the end instead of freezing * fix(mothership): single stopped tail region and drain cleanup * fix(mothership): clamp-aware chase interrupt so the floor drain can't park the settle follow * fix(mothership): park the chase only on real upward top moves, not growth * improvement(mothership): stack the stopped status above the actions row * improvement(mothership): compact stopped-turn tail with the original 10px rhythm * fix(mothership): single drain cadence and a settle-window gesture kill switch * fix(mothership): debt-aware drain fast-path and settle-window handle retention |