Rework the workspace around the AI-workspace model: a Mothership home, a
top-level Skills route, connected-credential and integration-detail pages,
and a polished sidebar/settings surface. Replace the notifications store
with a unified toast system (provider-level dismiss/pause, countdown ring).
Integrations & catalog:
- Add a BlockMeta layer (tags + catalog templates) scoped to catalog-visible
integrations; every catalog integration carries >=7 grounded templates.
- Rework the taxonomy: each block declares category tools|blocks|triggers.
3rd-party services are 'tools'; first-party primitives (postgres, mysql,
knowledge, file, search, stt/tts, image/video generators, thinking, etc.)
are 'blocks'. Versioned blocks follow the upgrade paradigm (old hidden,
latest in toolbar/docs).
- Generate integrations.json + tool docs canonically from block configs.
Architecture & cleanup:
- Consolidate block data extraction behind a single latest-version strategy
(getCanonicalBlocksByCategory; version-consistent getBlockMeta).
- Unify version-suffix handling in @sim/utils/string (stripVersionSuffix /
isVersionedType, with tests); registry, generate-docs, tools/utils, and
integrations all route through it.
- Repair latent broken barrels, remove dead code, fix BlockMeta-related type
errors and 5 broken docs links.
Behavior-preserving for block execution and the toolbar's tool/block listing.
* feat(integrations): add ZoomInfo, align Wiza, audit Apollo, refresh docs
- Add ZoomInfo integration: search/enrich contacts & companies, intent, news (6 tools), proxy route, block, and icon
- Validate and align Wiza tools/block/outputs against live API docs
- Audit Apollo tools: tighten params, outputs, and types
- Update tool docs (.mdx), icons, icon mappings, and integrations.json
* fix(zoominfo): use useId for ZoomInfoIcon clipPath to avoid duplicate DOM ids
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(apollo): address PR review on sequence add and bulk enrich
- sequence_add_contacts: send large contact_ids/label_names arrays in the
POST body (Rails merges query + body params) to avoid reverse-proxy URL
length limits; keep scalar settings in the query string
- organization_bulk_enrich: add back-compat shim mapping the legacy
`organizations` ({name, domain}[]) subBlock value to the new `domains`
string array so saved workflows keep running
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(integrations): unique ZoomInfo icon clip id, numeric employee range filters
- ZoomInfoIcon: derive clipPath id from useId() so multiple instances don't collide
- ZoomInfo company search: send employeeRangeMin/Max as numbers, matching revenueMin/Max
* fix(zoominfo): send employeeRange filters as strings per API schema
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(zoominfo): send contactAccuracyScoreMin as string per Contacts Search schema
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(apollo): harden people_search pagination, correct bulk-update output docs
- people_search: read pagination from both the nested `pagination` object
(legacy /mixed_people/search) and top-level fields, avoiding silent
fallback to defaults
- account_bulk_update: correct output descriptions — accounts support up to
1000 per request and async is opt-in (not auto-triggered at 100 like contacts)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(wiza): correct company enrichment credits shape in output docs
Company enrichment returns api_credits { total, company_credits }, not the email/phone/scrape breakdown used by individual reveals. Description-only fix verified against docs.wiza.co.
* fix(apollo): send sequence add contact_ids/label_names as query params per docs
Apollo documents every field for emailer_campaigns/:id/add_contact_ids as a query parameter with no request body. Append contact_ids[]/label_names[] to the query string instead of the JSON body to match the documented contract.
* feat(apollo): expose account_stage_id uniform field for bulk update accounts
Apollo documents account_stage_id as a Body Param for /accounts/bulk_update ('when using account_ids, apply this account stage to all accounts'). Adds it to the tool params, body builder, type, block subBlock, and params mapper alongside name/owner_id.
* docs(apollo): correct contact_update typed_custom_fields description
Apollo's update-a-contact endpoint documents typed_custom_fields, so drop
the inaccurate "not officially documented" caveat.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(zoominfo): default required outputFields on enrich; parse nested API error object
- ZoomInfo enrich endpoints require outputFields; send a curated default set when omitted so requests don't fail
- extractZoomInfoError now reads the GTM REST nested error object ({error:{code,message}}) instead of dropping it to a generic HTTP message
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* improvement(wiza): add wandConfig to complex prospect-search filter fields
Adds AI-assist wandConfig (json-object) with format examples to the structured filter inputs (job_title, job_company, past_company, company_industry, location, company_location) and the full filters object, completing the wandConfig checklist item for the Wiza block.
* fix(findymail): surface API .error messages and alphabetize registry
- transformResponse error branches now fall back to the response body's
`error` field before the generic status string, so Findymail's actual
messages ("Not enough credits" on 402, "Subscription is paused" on 423,
"One identifier is required..." on 422) reach the user instead of a
bare "Findymail API error: <status>". Applied to all 11 tools.
- alphabetize the findymail entries in tools/registry.ts to match the
already-alphabetical import block and the integration guideline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* improvement(billing): migrate hot path writes away from user_stats
* fix period start and end sot
* address comments
* Remove stale billing migration
Co-authored-by: Cursor <cursoragent@cursor.com>
* regen migrations
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Splits copilot chat messages out of the copilot_chats.messages JSONB
column into a dedicated copilot_messages table. JSONB stays canonical
during R+0 — every write path dual-writes to the new table best-effort
(try/catch + log warn, never throws).
Migration 0217 creates the table + indexes and inline-backfills history
from JSONB so OSS self-hosters don't need to run a separate script.
Write paths covered:
- post.ts (user message append)
- terminal-state.ts (assistant turn finalize)
- update-messages/route.ts (snapshot replace)
- inbox/executor.ts (background turn)
- fork/route.ts (chat clone)
- superuser/import-workflow/route.ts (chat import)
Each call threads chatModel + streamId where relevant; ON CONFLICT DO
UPDATE preserves existing stream_id / model via COALESCE.
For pre-R+1 reconciliation, run:
bun apps/sim/scripts/copilot-messages-reconcile.ts [--since='7 days']
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* fix(mothership): persist queued messages, edit-in-place preserves order
* fix(mothership): pause drain while head is in edit, restore handoff cleanup on edit, merge on migrate
* improvement(mothership): strip editing on persist; tighten comments to codebase style
* improvement(mothership): omit editing from partialize entirely
* fix(mothership): honor user removal during dispatch in failure-restore path
* chore(auth): upgrade better-auth 1.3.12 → 1.6.11
* chore(auth): address Greptile review — broaden change-email type + migration newline
* fix(auth): correct oneTimeToken expiresIn unit (minutes, not seconds)
Better-auth's oneTimeToken expiresIn is in minutes (multiplied by 60_000ms
internally). Sim's existing 24*60*60 evaluated to ~60 days of token
lifetime instead of the intended 24 hours. Tokens are one-time-use and
typically consumed within seconds of generation (Socket.IO handshake),
so this tightens an unused security window without affecting UX.
* improvement(cron): fire-and-forget for cron-invoked endpoints
* fix(cron): add staleness takeover to single-flight guard
* improvement(cron): drop single-flight guard, rely on DB row claiming
* fix(tables): coerce row values to column types on write instead of failing
* fix(tables): persist coerced values in upsert match + bulk update, normalize Date to ISO
* fix(tables): guard date coercion against out-of-range values
* improvement(schedu
les): retries, concurrency limits
* improvement(schedules): remove stale generated migration
Drop the pre-merge generated 0213 migration so it can be regenerated after syncing with staging.
Co-authored-by: Cursor <cursoragent@cursor.com>
* address comments
* fix pinned version for lru cache
* retryable errs cleanup
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(db): disable statement_timeout for migrations
* fix(ci): route migration workflow through guarded migrate.ts
* feat(tables): workflow-column run fixes + bounded "run N rows"
- Pass group.autoRun as the add-group dispatch flag so an autoRun=false
column no longer opens a no-op dispatch that flashes the run-count badge.
- Scope the context-menu re-run to the right-clicked workflow cell's group
(cascading to dependents) instead of every group on the row.
- Add an extensible per-dispatch row cap (DispatchLimit { type:'rows', max })
surfaced as "Run 10 / 1,000 empty rows" in the group header; dispatcher
stops after N eligible rows. New limit/processed_count columns on
table_run_dispatches.
- Fix stranded "Queued" cells: the cascade owner now treats a queued marker
(orphan pre-stamp) as a manual run so autoRun=false requested groups are
picked up, and drains late markers before releasing the row lock.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(db): regenerate dispatch limit migration on staging chain (0214)
Re-numbers the table_run_dispatches limit/processed_count columns from the
collided 0212 to 0214 after merging staging (which added its own 0212/0213).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(tables): lint formatting
* fix(tables): address PR review on dispatch cap + cascade drain
- Don't consume the row cap when batchEnqueueAndWait fails; a transient
failure no longer completes a capped dispatch with zero rows started.
- Outer cascade-drain loop only re-drives a genuine queued marker, not any
eligible group, so an empty-output group can't re-run forever.
- completeDispatch forwards limit on the terminal SSE event.
- Extract shared LIMITED_RUN_PRESETS for the Run-N-rows menu items.
* chore(lint): format generated tool-schemas-v1
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(tables): native enrichments sidebar + workflow input mapping
Add a Clay-style enrichments catalog to the table view and wire per-row
input mapping into workflow-backed columns.
- New "Enrichments" entry in the New-column dropdown opens a sliding panel
listing curated enrichment templates; picking one swaps to the workflow
config in-place (no cross-slide) with a back button.
- Type the workflow sidebar as manual | enrichment; enrichment hides the
launch + add-column-inputs affordances.
- Add a "Workflow inputs" advanced panel mapping Start-block input fields to
table columns (left-of-workflow columns only), with name-match auto-fill
and collapsible input-mapping-style rows.
- Persist type + inputMappings on the workflow group (types, contract, route,
service, hook) — jsonb, no migration.
- Consume inputMappings at run time: when present, feed Start-block fields
from the mapped columns; otherwise fall back to name-match spread.
- Clean up inputMappings on column rename/delete (stripGroupDeps + renameColumn).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(emcn): extract CollapsibleCard and reuse for input mapping
Pull the collapsible field-card markup (surface-4 header + surface-2 body,
click/keyboard toggle, truncated title + optional badge) into a shared
`CollapsibleCard` emcn component, and use it in the workflow-builder input
mapping rows and the table sidebar's input-mapping panel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(tables): code-defined enrichment registry run directly per row
Enrichments are now TS configs in apps/sim/enrichments/ (registry, like
connectors) that run directly per table row via the existing run/dispatch/
cell-write rails — no workflow execution.
- enrichments/{types,registry} + work-email (heuristic) and phone-number (stub).
- WorkflowGroup gains enrichmentId; WorkflowGroupOutput gains outputId
(workflowId/blockId/path kept required, '' for enrichment groups).
- Executor branches on group.type === 'enrichment' → maps inputMappings →
enrich() → outputs by outputId → cell-write. Missing required inputs skip
(blank cell) instead of erroring.
- Sidebar lists the registry; enrichment-config panel maps inputs to columns
and creates the enrichment group (no workflow UI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(enrichments): provider fallback cascade + hosted-key usage source
Replace each enrichment's single enrich() with an ordered providers[]
fallback cascade. Providers are plain data ({ id, label, toolId,
buildParams, mapOutput }) so the catalog stays client-safe; the
server-only runner (run.ts) calls executeTool per provider, first
non-empty result wins, misses/errors fall through, all-miss = blank cell.
Wire four enrichments on the hosted-safe providers (Hunter, PDL):
- Work Email (fullName, companyDomain): Hunter -> PDL
- Phone Number (fullName, companyDomain): PDL
- Company Domain (companyName): PDL
- Company Info (domain): PDL -> Hunter
Person enrichments take a single canonical fullName (Clay-style); Hunter
gets first/last via splitName(), PDL takes name directly.
Add 'enrichment' to usage_log_source enum (+ migration) so hosted-key
tool cost from these per-row calls can be billed to the table owner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(enrichments): bill hosted-key cost; surface provider errors; abort safety
- runEnrichment now returns { result, cost, error }: accumulates hosted-key
cost across the cascade, and sets `error` only when every provider that ran
errored (auth/rate-limit/outage) vs a clean miss.
- Executor records the cost to the table owner (createdBy) via recordUsage
(source 'enrichment'); billing failures are logged, never error the cell.
- F1: all-providers-errored now writes status 'error' instead of a blank
'completed' cell that looked like "no data found".
- F2: re-check the abort signal after the cascade so a cancel mid-tool-call
isn't recorded as a completed empty cell.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(tables): present enrichment columns as first-class in the grid
- Meta-header shows the enrichment's name + icon (Mail/Phone/Globe/Building2)
instead of "Workflow" + a color chip.
- Per-column header icon uses the enrichment's icon (via columnSourceInfo)
instead of the generic play icon.
- Hide "View execution" for enrichment cells in both the row context menu and
the action bar (no workflow execution exists to open); also hide the
meta-menu "View workflow" item for enrichment groups.
- Clicking an enrichment column header now opens the enrichments sidebar in
edit mode (pre-filled input mappings, Update via useUpdateWorkflowGroup)
instead of the workflow "Configure workflow" sidebar.
- Enrichment config lets the user name each output column (editable per-output,
deduped defaults) since enrichments can produce multiple columns.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tables): enrichment columns use type icon; output names editable
- Drop the per-column enrichment icon (it duplicated the meta-header icon).
Enrichment output columns now render the standard column-type icon (Text,
etc.) — the enrichment's icon stays only on the group meta-header.
- Make output column names editable in the enrichment config edit mode too;
changed names rename their columns via useUpdateColumn (the rename cascades
into the group's output refs server-side). Validation excludes the output's
own current name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tables): wrap enrichment catalog descriptions instead of truncating
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tables): edit enrichment output columns via the plain column editor
Edit column on an enrichment output now opens the normal column-config sidebar
(rename / type / unique) instead of the workflow 'Configure output column'
panel, which showed workflow-only fields and blocked a simple rename.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(copilot): list_enrichments + add_enrichment table tools
Let the copilot enumerate the code-defined enrichment registry and add an
enrichment column to a table (validating required input mappings against the
table's columns), backed by the same workflow-group machinery the UI uses.
* fix(enrichments): address PR review feedback
- Guard the enrichment cell path on `enrichmentId` so a group typed
'enrichment' without a registry id falls through to the workflow path
instead of erroring.
- Clear stale output values when skipping a row for missing required inputs,
so the auto cascade re-enriches once inputs return (was left completed+filled).
- Write a terminal state on abort in the enrichment path (matches the workflow
path) so a cancel between run and terminal-write can't leave the cell running.
- Edit mode: apply the group update (mappings/deps/auto-run) before column
renames so the primary edit lands even if a rename fails.
- Disable Save once validation has surfaced a missing required input.
- Use the workflowGroupById map instead of O(n) find in the context-menu and
action-bar hot paths.
* chore(commands): add /add-enrichment command
Guides adding a code-defined table enrichment to the registry, with a required
step to verify each provider tool has hosted-key support and chain to
/add-hosted-key when it doesn't.
* fix(enrichments): address second-pass PR review
- updateWorkflowGroup output diff now keys on outputId (falling back to
blockId::path) so enrichment outputs — which share empty blockId/path —
no longer collapse to one key and drop sibling columns.
- Enrichment terminal write now clears output columns absent from the result,
so a partial/empty re-run doesn't leave stale values.
- Editing a group whose enrichment was removed from the registry shows an
explanatory panel instead of silently falling through to the new-enrichment
catalog.
* feat(tables): show "Not found" badge for empty completed enrichment cells
An enrichment that runs to completion but matches nothing now renders a gray
"Not found" badge (like the Queued/Waiting cell states) instead of a blank
cell, so a real miss is distinguishable from an unrun cell. Scoped to
enrichment output columns; an empty string no longer counts as a value.
* fix(enrichments): don't re-run completed no-match enrichments on auto cascade
A completed enrichment with empty outputs is a real no-match result, not an
unfinished run. Eligibility now treats an enrichment's completed status as
terminal (regardless of output fill), so the auto cascade stops re-invoking
billable provider calls on every no-match row each dispatch. Input changes
still clear the exec entry, so genuine re-runs are unaffected; manual Run all
still re-runs.
* fix(enrichments): treat provider 404 as no-match, not a cell error
Providers like People Data Labs signal 'no record found' with HTTP 404, which
executeTool surfaces as a failed ToolResponse (status on output.status). The
cascade now treats a 404 as a clean miss — falls through to the next provider
and lets the cell render 'Not found' — instead of marking the cell errored.
Auth/rate-limit/5xx still propagate as real errors.
* fix(tools): surface HTTP status on error ToolResponse output
executeTool's catch handled Error instances in its first branch and only
extracted status/statusText/data for non-Error object throws — so HTTP errors
(thrown as Error instances carrying .status) lost their status on the returned
output. Surface it for Error instances too, so callers can branch on the
status (e.g. the enrichment cascade treating a provider 404 as a no-match).
* fix lint
* Revert ff
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cron schedules all fire on the same boundary (e.g. every :00), stampeding
the Postgres connection pool at the top of each minute/hour. Spread each due
schedule's start across a [0, 30s) window via trigger.dev's delay option
(no compute billed during the delay). Wires the previously-unused
EnqueueOptions.delayMs through the trigger.dev backend.
* Add queueing for hosted keys
* feat(rate-limiter): FIFO queue for hosted-key per-workspace fairness
Replace the per-call distributed lock with a Redis-backed FIFO queue so
callers within a workspace get strict ordering instead of racing the
bucket. Adds heartbeat-based crash recovery and dead-head reaping in a
single Lua script. Bumps Exa search hosted RPM from 5 to 60.
* fix(rate-limiter): bound hosted-key queue wait to execution budget; fix heartbeat + telemetry
Tie the per-workspace hosted-key queue wait to the surrounding execution
budget instead of a flat 5-minute cap. acquireKey now accepts the execution
AbortSignal (threaded from ExecutionContext): when present, the wait is
bounded by the run's actual plan timeout / cancellation, with the enterprise
async ceiling as a backstop; when absent it falls back to MAX_QUEUE_WAIT_MS.
This lets long-running async (Trigger.dev) runs use their full budget while
no longer letting a single queued call burn a short sync run's entire budget.
Also addresses Greptile review:
- P1: share one lastHeartbeatAt across all wait phases and cap every sleep to
HEARTBEAT_REFRESH_INTERVAL_MS so a long low-RPM retryAfterMs can no longer
let the head's heartbeat lapse mid-wait and break FIFO ordering.
- P2: derive hostedKeyQueueWaited telemetry reason from the actual bottleneck
(queue_position / dimension / actor_requests) instead of hardcoding it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(rate-limiter): make hosted-key queue waits abort-interruptible
Replace the plain capped sleeps in the queue-head and bucket-capacity wait
loops with an interruptibleSleep that resolves early when the execution
AbortSignal fires (timeout or cancellation), cleaning up its own timer and
listener. Previously a cancelled/timed-out run could overshoot by up to the
heartbeat cap (~10s) before the loop re-checked its budget; now it wakes
within a tick. The cap remains for heartbeat renewal.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(db): disable statement_timeout for migrations
* fix(ci): route migration workflow through guarded migrate.ts
* feat(hosted-keys): add Hunter.io and People Data Labs hosted key support
* improvement(api): use HttpError base class for typed-error status mapping
* chore(api): drop unreachable runtime check after HttpError instanceof guard
* feat: add LiteLLM as AI gateway provider
* fix: add litellm to attachments, provider store, utils, and block guards
* fix: add frontend model discovery pipeline for litellm provider
Add API route, contract, query hook case, and ProviderModelsLoader
entry so litellm models are fetched and synced to the store on
workspace load, matching the vllm/ollama/openrouter/fireworks pattern.
Also fixes defaultModel to empty string and adds litellm/ prefix
early-return in blocks/utils.ts (reviewer feedback).
* fix: remove azureEndpoint fallback from LiteLLM provider
Copy-paste artifact from vLLM provider. LiteLLM should only use
LITELLM_BASE_URL, not fall back to azureEndpoint which could cause
requests to be routed to the wrong server.
* fix(litellm): close audit gaps from PR #4644
- byok.ts: add litellm branch to getApiKeyWithBYOK so workflow
block execution can resolve the proxy key instead of throwing
"API key is required for litellm ..."
- check-api-validation-contracts.ts: bump route baseline 755 -> 756
to account for the new /api/providers/litellm/models route
- .env.example: document LITELLM_BASE_URL / LITELLM_API_KEY
- copilot edit-workflow validation: include LiteLLM in the list of
user-configured prefixed providers shown to the model
- providers/utils.ts: drop stray optional-chain on providers.litellm
to match the vllm pattern
- lint: apply biome formatting fixes (multi-line if, SVG path,
multi-line DYNAMIC_MODEL_PROVIDERS)
* fix(litellm): final parity gaps from second audit
- blocks/utils.ts getModelOptions(): include litellm models in the
combined model dropdown — was previously dropping any
proxy-discovered models from the agent block model picker.
- get-blocks-metadata-tool.ts mockProvidersState: add litellm bucket
so the server-side copilot block-metadata fallback can render
model options when the providers store is not initialized.
- blocks/utils.test.ts: add litellm to mock providers state (initial
+ beforeEach reset) and add a parallel store-bucket guard test
mirroring the vLLM case.
- providers/utils.test.ts: add parallel getApiKey test for litellm.
* feat(litellm): use official LiteLLM brand icon and color
- icons.tsx: replace the placeholder letterform with the official
LiteLLM brand mark embedded as a PNG data URI in an SVG image.
- models.ts: set color: #040229 on the litellm provider definition
to match the brand background.
* chore(litellm): validate /v1/models response with shared schema in initialize()
Match the API route handler — both code paths now run the same
vllmUpstreamResponseSchema.parse() over the upstream /v1/models
JSON instead of a raw type-cast, so malformed upstream payloads
surface a descriptive ZodError instead of a downstream TypeError.
Addresses Greptile review feedback on PR #4739.
---------
Co-authored-by: RheagalFire <arishalam121@gmail.com>
* feat(zoom): add KB connector for cloud recording transcripts, fix refresh token rotation
* fix(zoom): trim maxRecordings within page, relax VTT cue-id parsing
* fix(zoom): widen incremental sync overlap to 30 days for late transcripts
* fix(files): never dedup external URL fetches by path filename
External URL fetches in the file parse route were keyed on the path
filename, so two distinct URLs whose paths ended in `image.png` (e.g.
every Slack clipboard paste) collided in the workspace cache and
returned stale bytes from a prior fetch.
Extracts the fetch + save flow into `fetchExternalUrlToWorkspace` so
the broken dedup pattern can't be reintroduced. The helper always
downloads; `uploadWorkspaceFile` handles name collisions on save by
suffixing (`image.png` -> `image (1).png` -> ...).
* fix(files): distinguish non-member from read-only in workspace-save skip log
Splits the workspace-save skip branch so non-members (permission === null)
log 'user is not a workspace member' instead of the misleading 'lacks
write permission'. Adds a test covering the null case so the relaxed
behavior (vs. the prior route's hard 'File not found') is explicit.
* improvement(kb-connectors): align connector UI surfaces and strip redundant description suffix
- Fix icon colors to use --text-icon token across connector cards and modal
- Switch Reconnect/Update access buttons from active to primary variant
- Lift search box surface from --surface-2 to --surface-3 so it's visible against modal bg
- Replace raw <button> elements with emcn Button in base.tsx
- Shrink Connected Sources modal from lg to md
- Strip " to/into/from your knowledge base" from all 27 connector descriptions to prevent overflow
* fix(resource): replace raw pagination button with emcn Button
* fix(resource): prevent permission-gated breadcrumb items from flashing on load
* fix(resource): self-remove keydown listener on Escape and include session loading in isLoading guard
* fix(large-refs): cleanup based on table read
* address comments
* address comments
* bubble up storage ref errors
* cleanup code
* do not attempt blob deletion for infra outage
* cleanup dup helper
* fix(tools): pin resolved IP in DB connectors to prevent DNS-rebinding SSRF
`validateDatabaseHost` resolved an IP that was then discarded — drivers re-resolved
the hostname at connect time, enabling DNS-rebinding TOCTOU.
- mongodb: pass resolved IP via MongoClient `lookup` option
- mysql: pin TCP socket via `stream` factory; keep hostname for TLS servername
- postgresql: connect to resolved IP; pass `ssl` object with `servername` for SNI
- redis: parse URL explicitly and pass options-only (URL+options breaks override
due to ioredis's lodash.defaults); pin host and set `tls.servername` for rediss
- neo4j: pin IP for plain `bolt://`; leave `bolt+s`/`neo4j+s` unchanged to keep
Aura cert validation working (driver hardcodes servername with no override)
* chore(tools): remove explainer comments from DB connector SSRF fix
* fix(tools): add explicit TCP timeout to mysql stream factory
* fix(tools): unify postgres ssl handling to send SNI in preferred mode
* fix(tools): preserve postgres 'preferred' fallback behavior for backward compat
* fix(tools): reject non-numeric Redis URL db segment instead of silently using db 0
* improvement(mcp): post-merge hardening — protocol negotiation + distributed OAuth lock + typed error dispatch
- Inbound MCP server now negotiates protocolVersion per MCP 2025-06-18: echoes
the client's requested version when supported, falls back to our latest.
Previously hardcoded to the oldest spec version (2024-11-05).
- withMcpOauthRefreshLock now takes a Postgres advisory transaction lock in
addition to the in-process Promise chain, so concurrent processes (multi-task
ECS) serialize on a per-OAuth-row basis. Previously a refresh race across
processes could rotate a token under another process's feet and force re-auth.
- categorizeError dispatches on McpOauthAuthorizationRequiredError /
UnauthorizedError / McpConnectionError first, only falling back to substring
matching for SDK / third-party errors. Adds 502 for connection failures and
503 for cooldown. Tests cover all four typed cases.
- discoverTools no longer pretends to handle deferred-side-effect rejections
via a dead allSettled().catch() — each side-effect already self-logs; we just
swallow per-promise to silence unhandled-rejection warnings.
* fix(mcp): bugbot review on hardening PR
- Replace `as never` cast in negotiateProtocolVersion with `as readonly string[]`
on the array — preserves TypeScript narrowing on the comparison while
satisfying the readonly-tuple `.includes()` constraint properly.
- Document the pg_advisory_xact_lock tradeoff: session-level locks
(`pg_advisory_lock`) would release the connection earlier, but PgBouncer
transaction-pooling mode breaks them. xact_lock is the correct choice for
Sim's deployment; if pool pressure becomes real, the comment notes the
Redlock escape hatch.
* chore(mcp): trim verbose comments + reuse SDK Tool type in McpTool
- McpTool now extends `Pick<Tool, 'name' | 'description'>` from
@modelcontextprotocol/sdk so name/description fields stay in sync with
the SDK; serverId/serverName remain Sim-specific additions.
- Drop file-header restatements ("MCP Types - for connecting to external
MCP servers"), one-line wrapper docstrings ("Get connection status"),
and narrative comment blocks that just restate visible code.
- Keep only comments that document non-obvious "why" — OAuth refresh-lock
tradeoff, in-flight dedup key composition, SDK Tool.inputSchema typing,
preregistered-client semantics, postMessage handshake contract.
* improvement(mcp): swap PG advisory lock for Redis mutex on OAuth refresh
withMcpOauthRefreshLock now uses `coalesceLocally` + Redis acquireLock/
releaseLock with polling — the same primitives backing regular OAuth
refresh (`app/api/auth/oauth/utils.ts`). No more pinning a Postgres
connection for the duration of the SDK's OAuth HTTP refresh.
- In-process dedup: shared promise via `coalesceLocally`.
- Cross-process: Redis SET NX EX mutex; followers poll until the leader
releases (30s max wait, 100ms poll), then acquire and run fn().
- Each MCP caller still constructs its own client (semantics preserved).
- Falls open when Redis is unavailable — same behavior as the regular
OAuth refresh code path.
* improvement(mcp): use SDK protocol versions + pool pinned undici agents + cover OAuth lock
- McpClient.SUPPORTED_VERSIONS removed; getVersionInfo() and the inbound
serve route both import LATEST_PROTOCOL_VERSION / SUPPORTED_PROTOCOL_VERSIONS
directly from @modelcontextprotocol/sdk/types.js. New protocol revisions
ship automatically with SDK upgrades.
- pinned-fetch now caches undici Agents in a module-level LRU keyed by
resolvedIP (max 64). Back-to-back MCP calls to the same server reuse the
keep-alive connection pool instead of opening fresh TCP + TLS each time.
- New integration tests for withMcpOauthRefreshLock covering: in-process
dedup via coalesceLocally, cross-process serialization via Redis mutex,
fall-open on Redis unavailable, lock release on throw, release-failure
swallow, per-row key isolation.
* fix(mcp): serialize OAuth refresh callers; do not share McpClient
withMcpOauthRefreshLock previously wrapped fn() in coalesceLocally, which
returns the SAME promise (and the same resolved value) to all in-process
callers. fn() returns a stateful McpClient — sharing it meant whichever
caller finished first would disconnect the client while another was still
mid-call, leaving in-flight RPC on a closed connection.
Swap coalesceLocally for a per-row Promise chain: each caller waits for
the previous to settle, then runs its OWN fn() (gets its own client).
Cross-process Redis mutex semantics unchanged.
The "shareable scalar" assumption that makes coalesceLocally correct for
regular OAuth refresh (returns an access token string) does not hold for
MCP, where each caller needs an independent connection.
* fix(mcp): bugbot — TTL watchdog on OAuth lock + don't close evicted pinned agents
- Redis refresh lock now uses a 15s TTL with a watchdog that extends every
5s while fn() runs. Long-running OAuth refreshes no longer lose the lock
mid-flight and let another process race the same refresh.
- Pinned-agent LRU eviction no longer calls `agent.close()`. Existing
`createMcpPinnedFetch` closures hold the dispatcher reference and were
using a closed Agent after eviction. We drop from the cache and let GC
release the dispatcher once the last closure dies; undici closes idle
keep-alive connections via its own internal timeout.
- New tests: watchdog extends while fn() runs and stops once it settles;
evicted agents are not closed and captured closures still work.
* fix(mcp): throw instead of falling open when refresh lock wait exceeds deadline
When the Redis refresh lock can't be acquired within REFRESH_MAX_WAIT_MS
the previous code ran fn() uncoordinated — but another process can still
be holding the lock (watchdog-extended) and refreshing the same OAuth
row, recreating the exact race the lock prevents.
Throw on deadline. The caller can retry; the Redis-down branch remains
the only path that runs fn() uncoordinated (no coordination is possible
there).
* docs(mcp): restore TSDoc that documented intent on exported types/methods
Earlier comment-trim pass went too far on a few exports — restored
the TSDoc that explained non-obvious "why" decisions:
- SimMcpOauthProviderInit.preregistered: when set, the SDK skips DCR.
- McpServerConfig.userId: required for OAuth; selects which user's
stored tokens to use.
- McpOauthAuthorizationRequiredError: benign pending state vs failure.
- McpToolsChangedCallback / McpClientOptions: notification semantics,
DNS-rebinding pinning rationale, OAuth provider contract.
- StoredMcpToolReference / StoredMcpTool: minimal vs extended use.
- McpClient.connect: documents listChanged handler registration.
- McpService.executeTool: documents session-error retry behavior.
Pure-restatement comments ("Disconnect from MCP server") stay trimmed.
* fix(hubspot): fall back to objectType default in selector fetchOptions
useSubBlockStore.getValue returns null for default-valued dropdowns
until the user interacts with them. The properties, pipelines, stages,
and ownerId selectors were treating that as "no selection" and
short-circuiting, so the dropdowns appeared empty even though the
trigger uses 'contact' as the visible default.
Adds resolveSelectedObjectType to mirror the rendered default, so the
selectors fire on first paint with a valid objectType.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(hubspot): validate credentialId in selector routes
Mirrors the Gmail/Webflow/Jira selector route security pattern by
rejecting non-alphanumeric credentialId values before authorization
or token refresh.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(hubspot): use resolveSelectedObjectType in pipelineId/stageId fetchOptions
Both selectors used inline `?? 'contact'` fallbacks while properties and
targetPropertyName already routed through the resolver. Switch to the
shared helper so custom-object handling stays consistent across every
cascading selector.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* fix(combobox): show selected values in multi-select trigger label
The collapsed trigger was reading only `selectedOption` (the single-value
path) and falling back to the placeholder when nothing matched, so a
multi-select dropdown with 1+ checked items still rendered "Select one or
more channels" instead of the actual selections.
Added `multiSelectLabel` derived from `multiSelectValues`:
- 1 value → that label
- 2 values → "A, B"
- 3+ → "A, B +N"
Trigger now prefers `multiSelectLabel` when present and falls back to the
single-select label / placeholder otherwise. Muted-text color also flips
off when multi has any selection.
* chore(kb-connectors): strip redundant field-level descriptions
Removed 41 inline `description:` lines from configFields across 16 connectors
(Slack, MS Teams, GCal, Gmail, Notion, Linear-adjacent, Discord, Dropbox,
Evernote, Fireflies, Google Sheets, Intercom, Obsidian, Outlook, Reddit,
ServiceNow, WordPress, Zendesk). They mostly restated the field title (e.g.
"Channels to sync messages from" under a "Channels" label) and cluttered
the add/edit modal. Field titles + placeholders already communicate intent.
Connector-level `description` (used in the connector picker grid) is
unchanged.
* test(leader-lock): use fake timers to deterministically test follower polling
The "follower does a final read after timeout" test (and the
"follower returns null after timeout" test) relied on real-clock
`setTimeout` and `Date.now()` with very tight bounds (pollIntervalMs=5,
maxWaitMs=9). Any CI scheduler jitter of >4ms would cause the second
in-loop poll to be skipped, the polls counter to end at 2 instead of 3,
and the assertion `expect(result).toBe('late-leader')` to fail.
Switched both tests to `vi.useFakeTimers()` so the schedule is driven by
mocked time advanced via `vi.advanceTimersByTimeAsync`. The intent is
unchanged — verify that the in-loop deadline triggers exactly one
post-deadline last-chance call to `onFollower` — but the assertions no
longer depend on wall-clock timing.
Verified across 5 sequential runs with zero flakes.
* improvement(kb-connectors): restore field descriptions as info-icon tooltips
Restores the 41 field-level `description` lines stripped in fc644210d, but
instead of rendering them as inline muted-text paragraphs they're shown via a
small Info icon next to each field title. Hovering or focusing the icon
reveals the description in the existing emcn Tooltip. Keeps the modal layout
tight while preserving the per-field guidance.
Used <button type="button"> as the tooltip trigger (Radix asChild) rather
than <span tabIndex={0}> to satisfy a11y/noNoninteractiveTabindex.
* improvement(mcp): per-server tool queries + negative cache so one slow server can't block the workspace
Move MCP tool discovery off the workspace-aggregated `Promise.all` fan-out and
onto per-server React Query keys, matching how Cursor and Claude Code render
remote MCP. `useMcpToolsQuery` is now a `useQueries` combiner: each server has
its own cache entry, its own loading state, and a slow neighbor never gates the
others. Public shape stays compatible with existing consumers.
Add a short-TTL negative cache: when `listTools` fails (timeout, connection
error, etc.) we mark the server unhealthy for 30s so subsequent discovery calls
short-circuit instead of re-paying the timeout. OAuth-required errors are
exempt so re-auth retries immediately. Drop `LIST_TOOLS_TIMEOUT_MS` from 30s
to 10s to bound the worst-case first failure.
Invalidations are per-server where the action is per-server (OAuth popup,
per-server SSE event, refresh, update, delete). Bulk operations stay
workspace-broad. Adds tests for the negative-cache behavior and the OAuth
exemption.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* improvement(mcp): in-flight dedup + 2min negative TTL + no refetch on window focus
Three small follow-ups on top of per-server tool queries:
- Coalesce concurrent `discoverServerTools(userId, serverId, workspaceId)`
calls into a single upstream `tools/list`. Races between OAuth-callback
cache priming, post-OAuth UI refetch, and multi-tab loads no longer
double-fetch the same server.
- Bump negative-cache TTL from 30s to 2 minutes. Cleared on listChanged,
OAuth completion, manual refresh, and the next successful discovery, so
this floor only matters for genuinely dead servers — drops their floor
traffic by 4x.
- Disable `refetchOnWindowFocus` on per-server tool queries. listChanged
SSE + mutation invalidations already cover real schema changes; alt-tab
no longer triggers N parallel `tools/list` calls.
* fix(mcp): address bugbot/greptile review on per-server tool discovery
- Workspace-scoped `mcpKeys.serverToolsWorkspace(workspaceId)` prefix for bulk
invalidations (create-server, refresh-all, SSE workspace fallback, OAuth
fallback). The previous `mcpKeys.serverTools()` prefix was global and
invalidated every workspace's tools cache.
- `useMcpToolsQuery` folds `useMcpServers().isLoading` into the aggregate
`isLoading` so mounting no longer flashes an "empty tools" state during the
servers-list fetch. Aggregate `error` is suppressed when any per-server
query already returned data so one slow server can't blank out the others.
- `useForceRefreshMcpTools` invalidates the per-server query keys of servers
whose force refresh failed, so stale tools don't linger.
- `DiscoveryOutcome` error variant carries the original error, restoring the
OAuth-exemption check that `getErrorMessage(...)` previously erased.
- `discoverServerTools(userId, serverId, workspaceId, forceRefresh = false)`
now consults the positive + negative cache by default. Per-server React
Query refetches hit the cache instead of re-paying the listTools timeout;
callers that explicitly bypass cache (refresh route, OAuth callback, bulk
POST refresh) pass `forceRefresh: true`. Negative-cache hits throw a typed
`McpConnectionError` so the route layer can surface a fast 503.
* update icons
* chore(mcp): remove dead query keys and trim verbose comments
- Drop unused `mcpKeys.tools()` / `mcpKeys.toolsList()` — replaced by
per-server keys, no remaining callers.
- Trim narrative comments to keep only the non-obvious "why" notes.
* fix(mcp): map negative-cache cooldown error to HTTP 503
`McpConnectionError` thrown when a server is in cooldown previously
fell through `categorizeError` to a generic 500. Cooldown is a
transient-unavailability condition, so route it to 503.
* test(mcp): cover cooldown error → 503 categorization
* fix(mcp): address second-round bugbot review
- useMcpToolsQuery serverIds: filter on enabled + workspaceId match.
Disabled rows no longer trigger discover calls that get negative-cached,
and keepPreviousData on useMcpServers no longer races a workspace switch
into cross-workspace discover requests.
- Aggregate skips per-server data when that server's latest refetch errored,
so a broken server's last-known tools no longer linger in the workspace
view while its card shows an error.
- discoverServerTools failure path drops the positive cache alongside writing
the negative-cache marker. A cache-respecting follow-up now fails fast via
cooldown instead of returning stale tools from a now-broken server.
- useMcpTools.refreshTools drops the dead forceRefresh param — the per-server
queryFn always sends refresh=false, so the flag was never effective. Callers
wanting cache-bypass should use useForceRefreshMcpTools.
* chore(mcp): trim verbose comments
* fix(mcp): third-round bugbot review
- discoverTools failure path now drops the per-server positive cache alongside
writing the negative-cache marker, matching discoverServerTools' behavior so
a workspace-aggregate failure doesn't leave stale tools cached.
- useForceRefreshMcpTools filters disabled and out-of-workspace rows before
fan-out so disabled servers don't 404 → negative-cache themselves.
- Remove unused useMcpServerTools export — the aggregate goes through
useQueries directly, no external consumer exists.
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* improvement(kb-connectors): multi-select fields + Slack bot/app message extraction
Adds multi-value support to KB connector configuration fields and applies it
across 8 connectors: Jira (projects), Confluence (spaces), Slack (channels),
Microsoft Teams (channels), Google Calendar (calendars), Gmail (labels),
Notion (databases), and Linear (teams + projects). Each connector emits
byte-identical externalId for legacy single-value configs so existing rows
reconcile in place via the sync engine's externalId-keyed matching.
Framework changes:
- ConnectorConfigField gains `multi?: boolean`
- New `parseMultiValue` helper in @/connectors/utils
- useConnectorConfigFields state model upgraded to string|string[]
- ConnectorSelectorField renders Combobox in multi-select mode when `field.multi`
- Add/edit connector modals handle array values end-to-end
Per-connector specifics:
- Jira: JQL `project in (...)` for 2+ keys, `project = X` for one
- Confluence: routes through CQL `space in (...)` when multi; v2 fast path stays
for single+no-label; also fixes selector returning space.id instead of space.key
- Slack: loops per channel emitting one document each; extracts text from
attachments and Block Kit blocks (incl. nested attachment.blocks where GitHub
embeds PR bodies); contentHash bumped to slack-v2: to force one-time re-embed
- Microsoft Teams: loops per channel within a single team
- Google Calendar: compound cursor across calendars; single-calendar keeps
legacy externalId/contentHash for zero-churn
- Gmail: (label:A OR label:B) with quoted-form for labels with spaces
- Notion: sequential walk via JSON compound cursor; single-DB keeps bare cursor
- Linear: GraphQL IdComparator.in for multi, eq for single
* fix(kb-connectors): valuesEqual treats legacy scalar as equal to multi-array
Existing connectors created before multi-select store sourceConfig values as
scalars (e.g. projectKey: "ENG"). With the field now declared multi: true,
resolveSourceConfig returns an array (["ENG"]), and the original valuesEqual
fell through to a strict reference comparison — falsely flagging unsaved
changes on open and triggering an unnecessary string→array shape rewrite on
save.
valuesEqual now normalizes both sides to string[] via CSV-split when either
is an array, so persisted scalar and in-memory array of the same content
compare equal. Single-value (non-multi) fields keep strict string equality.
* fix(kb-connectors): GCal externalId on config downgrade, Slack silent skip, valuesEqual order
- google-calendar getDocument: derive isMultiCalendar from the externalId's
`:` separator instead of the current config count. Prevents duplicates when
a user downgrades from multi to single calendar — previously the returned
doc lost its `calendarId:` prefix and was treated as a new row by the sync
engine, orphaning the original.
- slack listDocuments: throw on unresolvable channel instead of silently
skipping. Matches MS Teams behaviour. Silent skip would let the sync
engine orphan-delete the previously indexed channel content if a bot was
removed or a channel was archived/renamed mid-life.
- edit-connector-modal valuesEqual: order-insensitive comparison for multi-
select arrays via Set membership. Multi-select UI doesn't guarantee
insertion order matches the server-returned order, so `["A","B"]` vs
`["B","A"]` would otherwise flag false unsaved changes.
* chore(kb-connectors): use emptyValue() fallback in isFieldPopulated for consistency
Behavior unchanged — isValuePopulated('') and isValuePopulated([]) both return
false — but reading the field-typed fallback inline matches the convention
used elsewhere in the hook (coerceForField, handleFieldChange, resolveSourceConfig).
* fix(kb-connectors): Linear projects selector loads across all selected teams
When the team selector is in multi-select mode, the basic-mode projects
dropdown was passing only the first team ID into the linear.projects
selector context (via readFirst in resolveDepValue), so projects from other
selected teams were invisible.
resolveDepValue now joins multi-value parents into a CSV string so dependent
selectors receive every selected parent ID. The /api/tools/linear/projects
route splits the CSV teamId, fetches projects from each team in parallel,
and dedupes by project ID. Single-team configs pass through unchanged
(`split(",")` on a bare ID yields a one-element array).
The AND-of-filters semantics in buildIssuesQuery is intentional and matches
standard GraphQL filter behavior — a user filtering on teams [A,B] and
projects [X,Y] gets issues in (A or B) AND (X or Y). With this fix the
project dropdown now shows every project under any selected team so the
user can compose the right project set.
* fix(gmail-connector): always wrap OR-containing custom query, not just unwrapped ones
The previous check `!/^\(.*\)$/.test(trimmedCustom)` was supposed to avoid
double-wrapping an already-parenthesized expression, but it false-positives
on inputs like `(from:alice) OR (from:bob)` where the parens don't bracket
the whole string. Those would skip wrapping and the top-level OR would bind
across the preceding label / category / date filters instead of the custom
clause.
Always wrap when an OR is present — double-parens are a no-op in Gmail
search syntax, so `((from:a OR from:b))` parses the same as `(from:a OR
from:b)`. Simpler than walking parens depth and provably safe.
* feat(mailer): add AWS SES and SMTP providers with auto-detect fallback
* fix(mailer): cast SES options to bridge duplicate @aws-sdk type identities
* fix(mailer): dedupe aws-sdk-sesv2, address review feedback
- Force a single @aws-sdk/client-sesv2 install via root package.json overrides; @types/nodemailer pulled in a nested copy whose nominal class brand made the two SDK type identities incompatible, breaking the CI build. With one install the cast disappears.
- Batch result message now reports successCount instead of sendable.length when entries are skipped, so "5 emails sent" no longer overstates delivery on partial failures.
- SMTP provider now warns when SMTP_HOST is set without SMTP_PORT, and when only one of SMTP_USER/SMTP_PASS is set — both previously silent misconfigurations.
- SMTP_SECURE schema is z.boolean() to match every other boolean in env.ts; runtime parsing is still handled by envBoolean.
- Strip the verbose TSDoc comments I had added.
* fix(mailer): exact sent counts in batch results, restore SES type cast
- mergeBatchResults: data.count and the message now report only emails that were actually delivered, not skipped-unsubscribed ones (they returned success: true and inflated the count). Empty-sendable branch distinguishes "all unsubscribed" from "mixed skip/failure" so the message stops lying when some entries fail validation.
- ses.ts: revert the package.json override approach (bun honors it locally but CI still installs a nested @types/nodemailer copy). Reinstate the `as unknown as` cast with a single-line WHY comment.
* fix(mailer): annotate double-cast in ses provider for strict api-validation
* fix(mailer): batch degrades isUnsubscribed errors to per-entry failures
A transient DB error in isUnsubscribed used to abort the whole batch
because the call sat outside the per-email try/catch in prepareBatch.
Wrap the unsubscribe check inside the same catch so a rejection becomes
a per-recipient failure, matching sendEmail's behavior. Lock it in
with a regression test.
Emir Karabeg