opensource/sim
3
0
Fork 0
mirror of https://github.com/simstudioai/sim.git synced 2026-11-03 23:31:47 +08:00
Code Issues Packages Projects Releases Wiki Activity

improvement(knowledge): polish tag filter dropdowns (#4816)

Browse Source 
* improvement(logs): object storage backed tracespans (#4787)

* improvement(logs): obj storage backed tracespans

* fix storage write context

* fix tests

* address comments

* address comments

* chore(db): remove migration 0219 to regenerate after staging merge

Drops the 0219_robust_shard SQL, its snapshot, and the journal entry so the
trace-spans/cost schema migration can be regenerated on top of the latest
staging migration chain (avoids a number collision with staging's migrations).

Co-authored-by: Cursor <cursoragent@cursor.com>

* improvement(billing): accurate per-member usage via shared ledger helper

Per-member/per-user usage in the org-member routes now adds the usage_log
ledger to the currentPeriodCost baseline (which is no longer incremented),
via a shared getOrgMemberLedgerByUser helper to avoid repeating the
subscription→period→ledger lookup across the admin and member-facing routes.

Co-authored-by: Cursor <cursoragent@cursor.com>

* regen migrations

* update migration

* address comments

* more code cleanup

* incorrect type cast

---------

Co-authored-by: Cursor <cursoragent@cursor.com>

* improvement(providers): harden OpenAI-compatible providers + add tests (#4796)

* improvement(providers): harden OpenAI-compatible providers + add tests

* fix(vllm): let tool-loop errors propagate instead of returning silent partial success

* fix(litellm): force tool_choice 'none' on final structured-output call

The deferred final call used tool_choice 'auto', so the model could emit
another tool_calls round instead of the structured answer, leaving content
stale. Use 'none' (matching vLLM/Fireworks) on both the streaming and
non-streaming final calls so the model must return the structured response.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(providers/ollama): drop tools from post-tool streaming call

Ollama ignores tool_choice (not in its supported fields), so vLLM/Fireworks'
tool_choice:'none' guard is a no-op here. Omit tools from the final streaming
payload instead so the summarization turn can't emit dropped tool calls.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(litellm): spread payload into deferred final call so reasoning_effort carries over

The non-streaming deferred finalPayload hand-picked fields and dropped
reasoning_effort (and any future payload field), diverging from the streaming
path which spreads ...payload. Spread payload here too for consistency.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(providers/ollama): restore enrichment TSDoc block

Keeps parity with sibling Chat Completions providers (cerebras/mistral/xai).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(fireworks): restore TSDoc on utils helpers

Restore the TSDoc blocks on supportsNativeStructuredOutputs,
createReadableStreamFromOpenAIStream, and checkForForcedToolUsage —
TSDoc is the codebase documentation standard and should not have been
stripped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(litellm): remove inline rationale comments (codebase uses TSDoc)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(providers/ollama): drop orphaned enrichment TSDoc

The block documented a function that now lives in trace-enrichment.ts, so it
documents nothing in this file.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* chore(copilot): deprecate mcp server (#4797)

* chore(copilot): deprecate mcp

* update error codes

* deprecate copilot api v1 route

* feat(integrations): hosted API keys for Findymail, Prospeo, and Wiza (#4777)

* feat(integrations): hosted API keys for Findymail, Prospeo, and Wiza

Add hosted-key support across all credit-consuming Findymail, Prospeo, and Wiza operations so Sim provides the key when a workspace has not brought its own. Register the three BYOK providers, consolidate Wiza's two-step reveal into a single polling wiza_individual_reveal op, and hide the API key field on hosted Sim for hosted operations.

* fix(integrations): harden Wiza reveal polling, soften enrichment getCost guards

Address Greptile + Cursor Bugbot review on #4777: return explicit failures from the Wiza individual_reveal poller instead of throwing (thrown errors were swallowed into a false queued success), short-circuit when the initial reveal is already terminal, tolerate transient 5xx/429 during polling, and return 0 (not throw) from Findymail getCost when the contacts/employees array is absent.

* chore(integrations): biome formatting after wiza merge resolution

* fix(wiza): type isTerminalReveal param structurally for next build typecheck

* feat(enrichments): add Findymail, Prospeo, Wiza to work-email waterfall

* feat(enrichments): add Wiza + Prospeo phone reveal to phone-number waterfall

* feat(enrichments): opportunistic identifiers + LinkedIn URL input across work-email & phone cascades

* fix(tables): reduce column header chevron size and fix sidebar shadow bleed (#4800)

* feat(slack): add install + privacy section to integration landing page (#4799)

* feat(slack): add install + privacy section to integration landing page

Adds a hand-authored, slug-keyed landing-content module (separate from the generated integrations.json so it survives regeneration) and renders an install walkthrough + privacy-policy link on integration pages when present. Also refreshes generated docs (data-enrichment entry, icon mappings, tool mdx).

* fix(landing): render privacy section independently, align CTA analytics label

* docs(landing): clarify the Slack install button is behind sign-in

* refactor(landing): bake integration landing content into generated json via docs-gen

Moves landing content (install walkthrough + privacy) out of a render-time augment and into the generation pipeline: generate-docs reads the pure-data content map and writes landingContent into integrations.json, so the page reads a single source (integration.landingContent). Canonical types live in integrations/data/types.ts.

* improvement(enrichments): align enrichments sidebar with design system (#4801)

* improvement(enrichments): align enrichments sidebar with design system

* fix(enrichments): consistent close button pattern and fix url link hover

* fix(misc): upgrade path change for new better-auth version, billing issue for workflow block agent usage (#4803)

* fix(misc): upgrade path change for new better-auth version, double-billing for workflow block agent usage

* fail loudly if stripe sub id missing

* fix(copilot): seq migration (#4804)

* chore(db): drop redundant idx_webhook_on_workflow_id_block_id index (#4809)

Removed because (workflow_id, block_id) is a left-prefix of idx_webhook_on_workflow_id_block_id_updated_at_desc, which fully covers it. The dropped index was non-unique and enforced no constraint.

* perf(copilot): read chat transcripts from copilot_messages (R+1 cutover) (#4808)

* perf(copilot): read chat transcripts from copilot_messages, not JSONB

Flip user-facing chat reads from the legacy copilot_chats.messages JSONB
array (5.7GB, 99% TOAST) to the normalized copilot_messages table via a
new loadCopilotChatMessages helper ordered by seq NULLS LAST, created_at,
id — the verified canonical order. Both chat-detail getters
(getAccessibleCopilotChat, getAccessibleCopilotChatWithMessages) now drop
the messages column from their metadata select (no more whole-array
detoast on every load) and assemble the transcript from the table after
authorization. This cascades to the copilot + mothership GET endpoints
and to resolveOrCreateChat's conversationHistory (the LLM payload).

The normalize/effective-transcript pipeline is source-agnostic
(copilot_messages.content == a JSONB array element), so transcripts are
byte-identical. Dual-write and the JSONB column stay in place as the
internal-logic source and fallback; removing JSONB writes is a later step.

Prod integrity verified before cutover: 0 messages missing, 0 NULL-seq,
0 dup keys/seq, 0 orphans, order-parity vs JSONB = 0 mismatches.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(copilot): cover auth-deny on a found row skips the messages query

Address PR review: exercise the `if (!authorized) return null` contract —
when the chat row exists but authorization fails, the getter returns null
and never issues the copilot_messages read.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* fix(tables): right-align run/stop in embedded toolbar; workflow cells format like normal cells (#4806)

* fix(tables): right-align run/stop in the embedded table toolbar

Add a right-aligned `trailing` slot to ResourceOptionsBar and move the embedded
mothership table's run/stop control into it, so Filter + Sort stay left-aligned
and run/stop sits opposite on the right. No-op for the search-bearing consumers
(logs, resource list), which don't pass `trailing`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(tables): workflow-output cells format values like normal cells

Workflow-output columns short-circuited in resolveCellRender and rendered their
value as plain text, so a sim-resource URL / external URL / JSON / date produced
by a workflow never got the chip, favicon link, or typed formatting a normal
cell gets. Factor value formatting into a shared `resolveValueKind` helper used
by both the workflow-value branch and the plain-cell branch; the workflow branch
keeps the typewriter reveal for plain streaming text via a `typewriter` flag.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(tables): detect resource/URL links on workflow output regardless of column type

Workflow output columns default to `json` (columnTypeForLeaf), so routing their
values through the type-based formatter (a) gated chip/URL promotion behind
`column.type === 'string'` — a URL produced by a json-typed output never became
a chip — and (b) JSON.stringify'd plain string values, adding quotes and losing
the typewriter reveal. Detect links (sim-resource chip / favicon URL) on the
value string directly for workflow outputs, falling back to the plain `value`
kind; plain cells keep the type-based formatting. Addresses Greptile P2 on #4806.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(icons): repair broken integration icon rendering (#4810)

* fix(icons): repair broken integration icon rendering

Two distinct bugs left integration icons broken on the /integrations page
(visible at 32-40px, hidden at the toolbar's 16px):

1. Corrupted SVG paths (Notion, Greptile, Granola, Calendly, Grafana, Bedrock):
   over-minified data dropped elliptical-arc flag digits (e.g. `A1 1 0 5.9 7`
   instead of `A1 1 0 0 0 5.9 7`); Granola's cubic stream was truncated. Browsers
   abort path parsing at the first invalid arc flag, so each rendered as a fragment
   or blank. Replaced with correct path data from canonical sources, preserving each
   icon's existing fill/gradient and bgColor.

2. Invisible glyph (Bright Data): its icon uses fill='currentColor' but bgColor was
   '#FFFFFF', and every surface forces text-white on the glyph - white-on-white.
   Changed bgColor to Bright Data's brand blue (#3d7ffc) so the white glyph reads,
   matching the white-glyph-on-brand-chip convention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(icons): restore Calendly dual-tone brand colors

Addresses review feedback: the previous fix replaced the broken Calendly icon
with a monochrome #006BFF path, dropping the cyan #0ae8f0 accent from the
original dual-tone mark. Restored the two-tone logo (blue + cyan) using clean,
valid path data, cropped to a tight square viewBox so it fills the chip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* improvement(icons): enlarge icons, fix Zoom contrast and Quiver chip

- Zoom: glyph was blue-on-blue (#0B5CFF on #2D8CFF chip); switched to
  currentColor so it renders as a white glyph on the blue chip.
- Quiver: chip bgColor #000000 -> #FFFFFF to match the icon's near-white box,
  and enlarged the mark slightly (viewBox crop).
- Enlarged (tightened viewBox, verified no clipping): RevenueCat, Prospeo,
  Granola, Firecrawl, Enrich.so, and the AWS icons (RDS, DynamoDB, SQS,
  CloudFormation, Athena, CloudWatch, SES, Bedrock, S3).
- ZoomInfo left unchanged: it is a full red rounded-square logo that already
  fills its frame, so a crop would clip it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(icons): use Bright Data wordmark on white chip; repair Circleback

- Bright Data: replaced the flame glyph with the official two-tone 'bright data'
  wordmark (provided asset), centered in a symmetric viewBox. Reverted the chip
  bgColor from #3d7ffc to #FFFFFF since the blue wordmark is invisible on a blue
  chip (the wordmark is designed for a light background).
- Circleback: a minifier had rounded the pattern's image scale to scale(0),
  collapsing the embedded logo to zero size (invisible). Restored the correct
  scale (1/280 = 0.00357142857) so the C. mark renders.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(docs): sync Quiver block color card to white chip

Reflects the Quiver bgColor change (#000000 -> #FFFFFF) in the docs block info card.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* improvement(icons): enlarge AWS/Cloudflare/Dagster icons, fully white Zoom

- Enlarged (tighter viewBox, render-verified, no clipping): Cloudflare, Dagster,
  and the red AWS icons AWS IAM, Identity Center, Secrets Manager, SES, STS.
  Identity Center was anomalously small (filled ~32% of its frame); the group is
  now sized consistently (~80% fill).
- Zoom: the camera lens triangle was still #0B5CFF (blue-on-blue); switched it to
  currentColor so the whole camera renders white on the blue chip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(wiza): consolidate individual reveal into a single operation

Merges the separate Start/Get Individual Reveal operations into one Individual
Reveal operation in the Wiza docs and integrations data (operationCount 5 -> 4).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* improvement(icons): size remaining AWS icons to match the set (~80% fill)

Bring RDS, DynamoDB, SQS, CloudFormation, Athena, CloudWatch and S3 up to the
same ~80% fill as the AWS IAM/Identity Center/Secrets Manager/SES/STS group, so
all AWS icons are visually consistent. Bedrock left as-is (already ~92% fill).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(icons): use Bright Data flame mark, enlarge ZoomInfo

- Bright Data: the full 'bright data' wordmark was illegible at chip size.
  Replaced with just the flame-'i' brand mark (blue #4280f6 on the white chip),
  centered.
- ZoomInfo: cropped the viewBox toward the white 'Zi' so it's larger; the red
  rounded-square background still fills the chip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* improvement(icons): enlarge CrowdStrike icon

The falcon mark sat small in its chip because the icon used a wide 768x500
viewBox (letterboxed in the square chip). Switched to a square viewBox centered
on the mark so it fills ~80%, consistent with the other icons.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* fix(tables): serialize schema mutations to prevent parallel column clobber (#4812)

* Make workflow description nullable

* fix(tables): serialize schema mutations to prevent parallel column clobber

* fix(tables): load workflow outside schema lock; use DbOrTx for getTableById

* fix(tables): scale idle timeout in updateColumnType to avoid aborting large type changes

* fix(tables): skip stale remap types when workflowId changes concurrently

* fix(tables): scale idle timeout in updateColumnConstraints for large tables

* fix(wait): resume live/draft async waits and preserve cell context on chained waits (#4814)

* Make workflow description nullable

* fix(wait): resume live/draft async waits and preserve cell context on chained waits

* improvement(knowledge): polish tag filter dropdowns

* improvement(knowledge): soften filter section labels

* improvement(knowledge): soften list filter labels

* fix(security): harden SSO domain registration, webhook path isolation, and CSV export (#4813)

* fix(security): harden KB file access, SSO domain registration, webhook path isolation, env secrets, and CSV export

* fix(sso): scope domain conflict query with indexed lower(domain) filter

Address PR review: avoid a full-table scan on every SSO provider
registration by filtering candidate rows in SQL with
lower(domain) = <normalized>, keeping the in-memory ownership check.
Also tighten the normalizeSSODomain TSDoc.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: condense env route security comments

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* icons update

* chore(security): tighten inline comments in CSV export and KB file authorization

Condense verbose comment blocks to concise TSDoc/single-line form; no behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): validate internal serve origin in KB file authorization

Replace the bypassable isInternalFileUrl substring check in resolveInternalKbKey
with an origin allow-list (base URL, internal API base URL, TRUSTED_ORIGINS).
A crafted external host whose path is /api/files/serve/<victim-key> no longer
resolves to the victim key. Relative same-origin URLs are unaffected.

* style(sso): use idiomatic sql lower() comparison for domain conflict query

Match the repo's prevailing `sql`lower(col) = value`` idiom for the
case-insensitive SSO domain conflict lookup.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): align workspace env admin gate with hasWorkspaceAdminAccess

Use the same admin check the secrets UI uses (owner, admin permission, or
org-admin) so owners and org-admins are not wrongly denied their own decrypted
workspace secrets, while read-only members remain restricted to names only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(sso): rely on lower(domain) match for conflict detection, drop dead in-memory recheck

Address PR review: the SQL `lower(domain) = <normalized>` predicate already
excludes rows that the in-memory `normalizeSSODomain(...) === domain` recheck
claimed to catch, making that recheck dead/misleading code. Match on the
canonical lower-cased domain and filter purely by ownership. Malformed legacy
values (wildcards, schemes, ports) never match an email domain at sign-in, so
excluding them is not a gap. Test DB mock now applies the lower() predicate so
the casing-variant case is genuinely exercised.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): scope webhook deploy path conflict to active webhooks

findConflictingWebhookPathOwner omitted the isActive filter that the
runtime dispatcher (findAllWebhooksForPath) applies, so an inactive but
non-archived webhook from another workflow (e.g. after undeploy or
failure auto-disable) would permanently block any new deployment on that
path even though it never receives deliveries. Align the guard with the
runtime isActive + archivedAt filter; the earliest-owner runtime check
remains the authoritative cross-tenant protection. Also trims verbose
TSDoc on the webhook path-isolation helpers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): exclude archived workflows from webhook deploy path conflict

findConflictingWebhookPathOwner now joins workflow and filters
isNull(workflow.archivedAt), matching the runtime dispatcher
(findAllWebhooksForPath). A webhook on an archived workflow can never
receive deliveries at runtime, so it must not block legitimate path reuse
with a 409.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): anchor KB file ownership to earliest document in any state

A KB file's owner is now the earliest document referencing its key regardless of
state (active/archived/deleted/excluded); access is granted only when that owning
document is still active. Closes the residual where an attacker could plant an
active document to claim a file whose original document was archived or deleted.

* updated greptile icon

* revert(security): drop KB file authorization changes

Reverts the knowledge-base file-access work (origin-pinning / owner-pinning /
origin allow-list in verifyKBFileAccess) and its test. The other hardening fixes
(SSO domain registration, webhook path isolation, workspace env secrets, CSV
export) are unchanged. apps/sim/app/api/files/authorization.ts is restored to its
origin/staging baseline.

* fix(sso): treat caller's own user-scoped provider as owned during conflict check

Self-hosters often register SSO user-scoped via the CLI script (no
SSO_ORGANIZATION_ID). If they later enable organizations and reconfigure the
same domain org-scoped through the UI, the conflict check previously treated
their own user-scoped row as another tenant's and returned a misleading 409.
Recognize the caller's own user-scoped provider as owned so that migration is
allowed, while still blocking another user's or another org's domain.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* revert(security): remove workspace-env admin gate

Defer to a credential-based access model (separate change). Restores
GET /api/workspaces/[id]/environment to main behavior and removes the test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(security): consolidate webhook path-collision check into one helper

Extract findConflictingWebhookPathOwner to lib/webhooks/utils.server.ts as
the single source of truth for cross-tenant path-collision detection, used by
both webhook creation paths (deploy sync and the manual /api/webhooks route).

This also repairs two latent issues in the manual route's previous inline
check, which queried with limit(1) and only webhook.archivedAt:
- limit(1) inspected one arbitrary row, so a same-workflow row could mask a
  foreign collision (false negative). The shared helper scans all matching
  rows.
- It omitted isActive/workflow.archivedAt, so inactive or archived-workflow
  webhooks (which never receive deliveries) permanently blocked path reuse.
  The helper mirrors the runtime dispatcher's filter.

Same-workflow webhook reuse for upsert is now a separate, explicit lookup.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): block private/reserved IPs for hosted 1Password Connect SSRF (#4818)

* fix(security): block private/reserved IPs for hosted 1Password Connect SSRF

* test(security): use real isPrivateOrReservedIP and cover IPv6 edge cases

* improvement(integrations): validate and expand devin, cursor, and greptile (#4820)

* improvement(integrations): validate and expand devin, cursor, and greptile

- devin: fix missing org_id path segment on all session endpoints, add 7 session sub-resource tools (list messages/attachments, get/append/replace tags, archive, terminate), pagination, and is_archived output
- cursor: add get_api_key_info, list_models, list_repositories tools
- greptile: align block and docs
- normalize array outputs to default [] and tighten types

* refactor(cursor): simplify list_repositories v2 array normalization

Collapse the redundant `?? []` + `Array.isArray` double-guard into a
single Array.isArray check, per PR review feedback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(devin): scope session-tag mapping to tag ops and normalize array tag inputs

- Only map sessionTags into the tools tags param for append/replace operations,
  preventing stale sessionTags state from clobbering create_session tags
- Fall back to a wired tags value when sessionTags is empty for tag operations
- Normalize tag inputs (string or wired string[]) via normalizeTags so array
  values from other blocks no longer throw on .split

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(cursor): restore base64 file data in legacy download_artifact metadata

The legacy CursorBlock exposes only content + metadata (no v2 file
output), so metadata.data was the only way legacy-block workflows could
access downloaded artifact bytes. Restore the base64 data field and
document it in the outputs/type instead of dropping it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(devin): coerce terminateArchive to archive flag for boolean-wired input

* docs(integrations): regenerate tool docs for new devin and cursor operations

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* fix(search-replace): don't auto-navigate when content edits invalidate the active match (#4819)

* fix(search-replace): don't auto-navigate when content edits invalidate the active match

* fix(search-replace): clear afterReplaceIndexRef on apply failure and zero matches

* fix(search-replace): remove duplicate setActiveSearchTarget(null) on close

* fix(search-replace): move afterReplaceIndexRef write inside handleApply past the guard

* fix(search-replace): auto-navigate when hydration resolves with no prior active match

* chore(search-replace): remove inline comments

* fix(search-replace): revert !activeMatchId guard that caused immediate re-navigation after deselect

* improvement(enrichments): limit company-info to fields both providers return (#4817)

Hunter's company dataset returns null industry/foundedYear for many large companies (verified against the live API for Microsoft, Amazon, Google), so under the first-non-empty-wins cascade those columns appeared inconsistently across rows. Limit company-info outputs to employee count and description — the fields Hunter and PDL both reliably return — so every row is consistent. employeeCount is a string so Hunter's range bucket and PDL's exact count share the column.

* fix(files): don't reject external URLs containing '..' in file parse validation (#4821)

* fix(files): don't reject external URLs containing '..' in file parse validation

The file block's file_fetch operation rejected any external URL whose path
contained '..' (e.g. Slack files-pri slugs with a literal '...') with
'Access denied: path traversal detected'. Traversal checks only apply to
local paths — external http(s) URLs are fetched with SSRF protection
downstream and are never resolved against the filesystem, so they now
short-circuit as valid. Internal /api/files/serve/ URLs keep full traversal
protection.

* test(files): fix external-URL assertion to handle undefined error

* test(files): assert success explicitly in external-URL traversal test

* fix(files): keep traversal protection for https URLs matching internal serve paths

* feat(google-sheets): add row filtering to read with numeric operators (#4822)

* feat(google-sheets): add row filtering to read with numeric operators

Adds client-side row filtering to the Google Sheets read (v2) operation.
Filter the returned rows by a header column using text operators
(contains, not_contains, exact, not_equals, starts_with, ends_with) and
numeric/ordering operators (gt, gte, lt, lte). Filtering lives in a pure,
unit-tested helper (filterSheetRows) and runs over the fetched read range;
an optional `filter` output reports whether the column was found and how
many rows matched.

Also hardens the surrounding tools:
- trim spreadsheetId in write/update/append URL builders (matches read)
- URL-encode the v1 read default range
- expose valueInputOption for the update operation in the block

Backwards compatible: with no filter requested, read output is byte-
identical and the `filter` field is omitted. The filterMatchType union is
widened additively (4 -> 10 values).

* fix(google-sheets): correct filter metadata for missing column and header-only sheets

- matchedRows is now 0 (not totalRows) when the filter column is not found,
  so it no longer contradicts applied=false / columnFound=false
- columnFound now reflects an actual header lookup for empty/header-only
  sheets instead of being hardcoded true
- add tests covering header-only and empty sheets with present/absent columns

* fix(selectors): fetch all pages for paginated dropdown list routes (#4823)

* fix(selectors): fetch all pages for paginated dropdown list routes

Dropdown selectors fetched only the first page of paginated provider
APIs, silently hiding results past page one. Add bounded server-side
draining to the list routes across Microsoft Graph, Google, Notion,
Atlassian, Linear, AWS CloudWatch, and offset/token REST APIs, plus a
shared client-side drain cap in the selector hook. Response shapes,
stored values, and tool execution are unchanged; CloudWatch list tools
still honor a caller-supplied limit. Also fixes the Word file picker
that was searching for .xlsx files.

* fix(selectors): harden JSM and Monday pagination draining

- JSM service-desk/request-type drains advance `start` by the actual row
  count returned (not the fixed page size) and stop on an empty page, so a
  short non-final page can't skip items.
- Monday boards drain now checks `response.ok` per page, surfacing a
  mid-drain HTTP failure instead of treating it as an empty final page and
  returning a partial 200.

* docs(selectors): clarify JSM drain advances start by actual row count

The offset-advancement fix (advance `start` by the rows returned, not the
fixed page size) landed in 7b19788a8; update the TSDoc to match so it no
longer reads as advancing by `limit`.

* fix(selectors): drain fetchPage in direct fetchList callers

Making `fetchList` optional left three direct callers (outside the
useSelectorOptions hook) calling it unguarded, which broke the build's
type check. Route them through a shared `loadAllSelectorOptions` helper
that uses `fetchList` when present and otherwise drains `fetchPage`.
This also prevents a regression: `confluence.spaces` / `knowledge.documents`
now paginate via `fetchPage` only, and these callers (search/replace,
value resolution) would otherwise have silently returned no options.

* chore(selectors): rename MAX_PAGE_PAGES to MAX_NOTION_PAGES for readability

* fix(sso): re-check domain conflict before write and reject IP-address domains (#4825)

* improvement(copilot): make copilot_messages the sole transcript store, remove JSONB dual-write (#4826)

Stop writing/reading the legacy copilot_chats.messages JSONB column now that
reads are cut over to copilot_messages. Make appendCopilotChatMessages the
primary write (throws on failure instead of swallowing), repoint peripheral
readers (workspace VFS, chat cleanup, data drains, fork, superuser import) to
copilot_messages, and persist the assistant turn inside finalizeAssistantTurn's
transaction so it commits atomically with the stream-marker clear. The column
itself is dropped in a follow-up migration after this bakes.

* feat(tables): expand filter operators (not-contains, starts/ends-with, not-in, empty) (#4827)

Add does-not-contain ($ncontains), starts-with ($startsWith), ends-with
($endsWith), not-in-array ($nin, previously executed server-side but unexposed
in the UI), and is-empty/is-not-empty ($empty) filter operators end-to-end —
SQL builder, condition types, query-builder converters/constants, the filter
UI, the Table tools/block descriptions, and docs.

Also fix correctness bugs in the filter builder surfaced by the wider operator
set:
- Same-column AND rules (e.g. age > 18 AND age < 65, or name startsWith 'A'
  AND name endsWith 'Z') silently overwrote each other because the AND group
  was keyed by column name. They now merge into one operator object, which
  also makes Filter -> rules -> Filter round-trip losslessly for multi-operator
  columns.
- $nin values were not split into an array like $in, and textual-match values
  like "123" were numeric-coerced (breaking the ILIKE path).
- A non-boolean $empty operand from the raw API silently inverted the check; it
  now coerces 'true'/'false' strings and otherwise returns a 400.

* improvement(copilot): stop persisting tool-call result outputs in transcripts (#4829)

Opening a Mothership task could take many seconds because a single persisted
assistant message in copilot_messages.content can reach hundreds of MB, almost
entirely inside contentBlocks[].toolCall.result.output (e.g. a get_workflow_logs
or run_workflow result). The DB query is ~2ms; the cost is detoasting that
payload, shipping it to the browser, and parsing it.

These outputs are dead weight on the Sim side: they are never rendered (the
thread shows only tool name/title/status) and never replayed to the model (the
upstream copilot service owns conversation memory). So drop result.output before
it is persisted, keeping result.success/error plus the tool metadata.

- add stripToolResultOutput() in persisted-message.ts
- apply it in messages-store toRow (covers every write path) and in
  loadCopilotChatMessages (existing rows render fast on read)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* feat(providers): add Together AI, Baseten, and Ollama Cloud model providers (#4830)

* feat(providers): add Together AI, Baseten, and Ollama Cloud model providers

* fix(providers): guard Ollama streaming fast-path with hasActiveTools

Match Together/Baseten/Fireworks: when tools are supplied but all are
filtered out (usageControl 'none'), take the single streaming call instead
of an extra non-streaming round-trip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(providers): filter non-chat model types from Together model list

* refactor(providers): dedupe Ollama Cloud upstream schema

ollamaCloudUpstreamResponseSchema was byte-for-byte identical to
ollamaUpstreamResponseSchema (both /api/tags endpoints return the same
{ models: [{ name }] } shape). Drop the duplicate and reuse the shared schema.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* fix(knowledge): calendar view sync, deduplicate popover animation classes, type-safe filter cast

* cleanup(knowledge): remove TRIGGER_BORDER_CLASS duplication, inline displayLabel, drop enabledFilterParam alias

---------

Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Waleed <walif6@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Theodore Li <theo@sim.ai>
Co-authored-by: andresdjasso <andresdjasso@users.noreply.github.com>
This commit is contained in:
andres
2026-06-01 12:49:05 -07:00
committed by GitHub
co-authored by Claude Opus 4.8 Vikhyath Mondreti Cursor Waleed Theodore Li andresdjasso
parent 6a7e7344fa
commit 6c0594899a
324 changed files with 56043 additions and 7259 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/docs/components/icons.tsx
+329 -135
View File
File diff suppressed because one or more lines are too long
apps/docs/content/docs/de/copilot/index.mdx
-95
View File
@@ -246,98 +246,3 @@ Die Copilot-Nutzung wird pro Token des zugrunde liegenden LLM abgerechnet. Wenn
<Callout type="info">
Siehe die [Seite zur Kostenberechnung](/execution/costs) für Abrechnungsdetails.
</Callout>
## Copilot MCP
Sie können Copilot als MCP-Server in Ihrem bevorzugten Editor oder AI-Client verwenden. Damit können Sie Sim-Workflows direkt aus Tools wie Cursor, Claude Code, Claude Desktop und VS Code erstellen, testen, bereitstellen und verwalten.
### Generieren eines Copilot-API-Schlüssels
Um sich mit dem Copilot-MCP-Server zu verbinden, benötigen Sie einen **Copilot-API-Schlüssel**:
1. Gehen Sie zu [sim.ai](https://sim.ai) und melden Sie sich an
2. Navigieren Sie zu **Einstellungen** → **Copilot**
3. Klicken Sie auf **API-Schlüssel generieren**
4. Kopieren Sie den Schlüssel – er wird nur einmal angezeigt
Der Schlüssel sieht aus wie `sk-sim-copilot-...`. Sie werden ihn in der folgenden Konfiguration verwenden.
### Cursor
Fügen Sie Folgendes zu Ihrer `.cursor/mcp.json` (Projektebene) oder den globalen Cursor-MCP-Einstellungen hinzu:
```json
{
"mcpServers": {
"sim-copilot": {
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
```
Ersetzen Sie `YOUR_COPILOT_API_KEY` durch den oben generierten Schlüssel.
### Claude Code
Führen Sie den folgenden Befehl aus, um den Copilot MCP-Server hinzuzufügen:
```bash
claude mcp add sim-copilot \
--transport http \
https://www.sim.ai/api/mcp/copilot \
--header "X-API-Key: YOUR_COPILOT_API_KEY"
```
Ersetzen Sie `YOUR_COPILOT_API_KEY` durch Ihren Schlüssel.
### Claude Desktop
Claude Desktop benötigt [`mcp-remote`](https://www.npmjs.com/package/mcp-remote), um sich mit HTTP-basierten MCP-Servern zu verbinden. Fügen Sie Folgendes zu Ihrer Claude Desktop-Konfigurationsdatei hinzu (`~/Library/Application Support/Claude/claude_desktop_config.json` unter macOS):
```json
{
"mcpServers": {
"sim-copilot": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://www.sim.ai/api/mcp/copilot",
"--header",
"X-API-Key: YOUR_COPILOT_API_KEY"
]
}
}
}
```
Ersetzen Sie `YOUR_COPILOT_API_KEY` durch Ihren Schlüssel.
### VS Code
Fügen Sie Folgendes zu Ihrer VS Code `settings.json` oder Workspace `.vscode/settings.json` hinzu:
```json
{
"mcp": {
"servers": {
"sim-copilot": {
"type": "http",
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
}
```
Ersetzen Sie `YOUR_COPILOT_API_KEY` durch Ihren Schlüssel.
<Callout type="info">
Für selbst gehostete Deployments ersetzen Sie `https://www.sim.ai` durch Ihre selbst gehostete Sim-URL.
</Callout>
apps/docs/content/docs/en/blocks/table.mdx
+5 -1
View File
@@ -275,7 +275,11 @@ Filters use MongoDB-style operators for flexible querying:
| `$lte` | Less than or equal | `{"quantity": {"$lte": 10}}` |
| `$in` | In array | `{"status": {"$in": ["active", "pending"]}}` |
| `$nin` | Not in array | `{"type": {"$nin": ["spam", "blocked"]}}` |
| `$contains` | String contains | `{"email": {"$contains": "@gmail.com"}}` |
| `$contains` | String contains (case-insensitive) | `{"email": {"$contains": "@gmail.com"}}` |
| `$ncontains` | Does not contain (case-insensitive; matches empty cells) | `{"email": {"$ncontains": "@spam.com"}}` |
| `$startsWith` | Starts with (case-insensitive) | `{"name": {"$startsWith": "Dr."}}` |
| `$endsWith` | Ends with (case-insensitive) | `{"file": {"$endsWith": ".pdf"}}` |
| `$empty` | Cell is empty (`true`) or non-empty (`false`) | `{"phone": {"$empty": true}}` |
### Combining Filters
apps/docs/content/docs/en/copilot/index.mdx
-84
View File
@@ -50,90 +50,6 @@ For complex requests, Copilot may show its reasoning in an expandable thinking b
Copilot usage is billed per token and counts toward your plan's credit usage. If you reach your limit, enable on-demand billing from Settings → Subscription.
## Copilot MCP
You can use Copilot as an MCP server to build, test, and manage Sim workflows from external editors — Cursor, Claude Code, Claude Desktop, and VS Code.
### Generating a Copilot API Key
1. Go to [sim.ai](https://sim.ai) and sign in
2. Navigate to **Settings** → **Copilot**
3. Click **Generate API Key**
4. Copy the key — it is only shown once
The key will look like `sk-sim-copilot-...`.
### Cursor
Add to `.cursor/mcp.json`:
```json
{
"mcpServers": {
"sim-copilot": {
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
```
### Claude Code
```bash
claude mcp add sim-copilot \
--transport http \
https://www.sim.ai/api/mcp/copilot \
--header "X-API-Key: YOUR_COPILOT_API_KEY"
```
### Claude Desktop
Claude Desktop requires [`mcp-remote`](https://www.npmjs.com/package/mcp-remote). Add to `~/Library/Application Support/Claude/claude_desktop_config.json`:
```json
{
"mcpServers": {
"sim-copilot": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://www.sim.ai/api/mcp/copilot",
"--header",
"X-API-Key: YOUR_COPILOT_API_KEY"
]
}
}
}
```
### VS Code
Add to `settings.json` or `.vscode/settings.json`:
```json
{
"mcp": {
"servers": {
"sim-copilot": {
"type": "http",
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
}
```
<Callout type="info">
For self-hosted deployments, replace `https://www.sim.ai` with your self-hosted Sim URL.
</Callout>
<FAQ items={[
{ question: "How is Copilot different from Mothership?", answer: "Copilot is scoped to the workflow you have open — it reads and edits that workflow's blocks and connections. Mothership has access to your entire workspace and can build workflows, manage tables, run research, schedule jobs, and take actions across integrations." },
{ question: "Can Copilot access other workflows or workspace data?", answer: "Copilot is scoped to the current workflow. For tasks that span multiple workflows or require workspace-level context, use Mothership." },
apps/docs/content/docs/en/tools/cursor.mdx
+53
View File
@@ -211,4 +211,57 @@ Download a generated artifact file from a cloud agent. Returns the file for exec
| --------- | ---- | ----------- |
| `file` | file | Downloaded artifact file stored in execution files |
### `cursor_list_models`
List the models available for launching cloud agents. Returns API-aligned fields only.
#### Input
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Cursor API key |
#### Output
| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `models` | array | Array of available model names |
### `cursor_list_repositories`
List the GitHub repositories accessible to the authenticated user. Returns API-aligned fields only.
#### Input
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Cursor API key |
#### Output
| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `repositories` | array | Array of accessible repositories |
| ↳ `owner` | string | Repository owner |
| ↳ `name` | string | Repository name |
| ↳ `repository` | string | Repository URL |
### `cursor_get_api_key_info`
Retrieve details about the API key currently in use. Returns API-aligned fields only.
#### Input
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Cursor API key |
#### Output
| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `apiKeyName` | string | Name of the API key |
| `createdAt` | string | API key creation timestamp |
| `userEmail` | string | Email of the key owner |
apps/docs/content/docs/en/tools/devin.mdx
+189 -7
View File
@@ -50,6 +50,7 @@ Create a new Devin session with a prompt. Devin will autonomously work on the ta
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Devin API key \(service user credential starting with cog_\) |
| `orgId` | string | Yes | Devin organization ID \(prefixed with org-\) |
| `prompt` | string | Yes | The task prompt for Devin to work on |
| `playbookId` | string | No | Optional playbook ID to guide the session |
| `maxAcuLimit` | number | No | Maximum ACU limit for the session |
@@ -67,10 +68,11 @@ Create a new Devin session with a prompt. Devin will autonomously work on the ta
| `createdAt` | number | Unix timestamp when the session was created |
| `updatedAt` | number | Unix timestamp when the session was last updated |
| `acusConsumed` | number | ACUs consumed by the session |
| `tags` | json | Tags associated with the session |
| `pullRequests` | json | Pull requests created during the session |
| `tags` | json | Tags associated with the session \(array of strings\) |
| `pullRequests` | json | Pull requests created during the session \(\[\{pr_url, pr_state\}\]\) |
| `structuredOutput` | json | Structured output from the session |
| `playbookId` | string | Associated playbook ID |
| `isArchived` | boolean | Whether the session is archived |
### `devin_get_session`
@@ -81,6 +83,7 @@ Retrieve details of an existing Devin session including status, tags, pull reque
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Devin API key \(service user credential starting with cog_\) |
| `orgId` | string | Yes | Devin organization ID \(prefixed with org-\) |
| `sessionId` | string | Yes | The session ID to retrieve |
#### Output
@@ -95,10 +98,11 @@ Retrieve details of an existing Devin session including status, tags, pull reque
| `createdAt` | number | Unix timestamp when the session was created |
| `updatedAt` | number | Unix timestamp when the session was last updated |
| `acusConsumed` | number | ACUs consumed by the session |
| `tags` | json | Tags associated with the session |
| `pullRequests` | json | Pull requests created during the session |
| `tags` | json | Tags associated with the session \(array of strings\) |
| `pullRequests` | json | Pull requests created during the session \(\[\{pr_url, pr_state\}\]\) |
| `structuredOutput` | json | Structured output from the session |
| `playbookId` | string | Associated playbook ID |
| `isArchived` | boolean | Whether the session is archived |
### `devin_list_sessions`
@@ -109,7 +113,9 @@ List Devin sessions in the organization. Returns up to 100 sessions by default.
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Devin API key \(service user credential starting with cog_\) |
| `orgId` | string | Yes | Devin organization ID \(prefixed with org-\) |
| `limit` | number | No | Maximum number of sessions to return \(1-200, default: 100\) |
| `after` | string | No | Pagination cursor \(endCursor from a previous response\) to fetch the next page |
#### Output
@@ -123,7 +129,14 @@ List Devin sessions in the organization. Returns up to 100 sessions by default.
| ↳ `title` | string | Session title |
| ↳ `createdAt` | number | Creation timestamp \(Unix\) |
| ↳ `updatedAt` | number | Last updated timestamp \(Unix\) |
| ↳ `tags` | json | Session tags |
| ↳ `tags` | json | Session tags \(array of strings\) |
| ↳ `acusConsumed` | number | ACUs consumed by the session |
| ↳ `pullRequests` | json | Pull requests created during the session \(\[\{pr_url, pr_state\}\]\) |
| ↳ `playbookId` | string | Associated playbook ID |
| ↳ `isArchived` | boolean | Whether the session is archived |
| `endCursor` | string | Pagination cursor for the next page, or null if last page |
| `hasNextPage` | boolean | Whether more sessions are available |
| `total` | number | Total number of sessions, if provided |
### `devin_send_message`
@@ -134,6 +147,7 @@ Send a message to a Devin session. If the session is suspended, it will be autom
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Devin API key \(service user credential starting with cog_\) |
| `orgId` | string | Yes | Devin organization ID \(prefixed with org-\) |
| `sessionId` | string | Yes | The session ID to send the message to |
| `message` | string | Yes | The message to send to Devin |
@@ -149,9 +163,177 @@ Send a message to a Devin session. If the session is suspended, it will be autom
| `createdAt` | number | Unix timestamp when the session was created |
| `updatedAt` | number | Unix timestamp when the session was last updated |
| `acusConsumed` | number | ACUs consumed by the session |
| `tags` | json | Tags associated with the session |
| `pullRequests` | json | Pull requests created during the session |
| `tags` | json | Tags associated with the session \(array of strings\) |
| `pullRequests` | json | Pull requests created during the session \(\[\{pr_url, pr_state\}\]\) |
| `structuredOutput` | json | Structured output from the session |
| `playbookId` | string | Associated playbook ID |
| `isArchived` | boolean | Whether the session is archived |
### `devin_list_session_messages`
List the messages exchanged in a Devin session, including messages from both the user and Devin.
#### Input
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Devin API key \(service user credential starting with cog_\) |
| `orgId` | string | Yes | Devin organization ID \(prefixed with org-\) |
| `sessionId` | string | Yes | The session ID to list messages for |
| `limit` | number | No | Maximum number of messages to return \(1-200, default: 100\) |
| `after` | string | No | Pagination cursor \(endCursor from a previous response\) to fetch the next page |
#### Output
| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `messages` | array | Messages exchanged in the session |
| ↳ `eventId` | string | Unique identifier for the message event |
| ↳ `source` | string | Origin of the message \(devin or user\) |
| ↳ `message` | string | The message content |
| ↳ `createdAt` | number | Unix timestamp when the message was created |
| `endCursor` | string | Pagination cursor for the next page, or null if last page |
| `hasNextPage` | boolean | Whether more messages are available |
| `total` | number | Total number of messages, if provided |
### `devin_list_session_attachments`
List the files uploaded to or produced by a Devin session.
#### Input
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Devin API key \(service user credential starting with cog_\) |
| `orgId` | string | Yes | Devin organization ID \(prefixed with org-\) |
| `sessionId` | string | Yes | The session ID to list attachments for |
#### Output
| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `attachments` | array | Attachments associated with the session |
| ↳ `attachmentId` | string | Unique identifier for the attachment |
| ↳ `name` | string | Attachment file name |
| ↳ `url` | string | URL to download the attachment |
| ↳ `source` | string | Origin of the attachment \(devin or user\) |
| ↳ `contentType` | string | MIME type of the attachment |
### `devin_get_session_tags`
Retrieve the tags currently applied to a Devin session.
#### Input
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Devin API key \(service user credential starting with cog_\) |
| `orgId` | string | Yes | Devin organization ID \(prefixed with org-\) |
| `sessionId` | string | Yes | The session ID to retrieve tags for |
#### Output
| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `tags` | json | Tags applied to the session \(array of strings\) |
### `devin_append_session_tags`
Add tags to a Devin session without removing existing tags (max 50 tags total).
#### Input
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Devin API key \(service user credential starting with cog_\) |
| `orgId` | string | Yes | Devin organization ID \(prefixed with org-\) |
| `sessionId` | string | Yes | The session ID to add tags to |
| `tags` | string | Yes | Tags to append to the session \(comma-separated string or array of strings\) |
#### Output
| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `tags` | json | Updated list of tags on the session \(array of strings\) |
### `devin_replace_session_tags`
Replace all tags on a Devin session with a new set of tags (max 50 tags).
#### Input
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Devin API key \(service user credential starting with cog_\) |
| `orgId` | string | Yes | Devin organization ID \(prefixed with org-\) |
| `sessionId` | string | Yes | The session ID to replace tags on |
| `tags` | string | Yes | Tags that will overwrite the existing tags \(comma-separated string or array of strings\) |
#### Output
| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `tags` | json | Updated list of tags on the session \(array of strings\) |
### `devin_archive_session`
Archive a Devin session. Archived sessions can still be viewed but cannot be modified or resumed.
#### Input
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Devin API key \(service user credential starting with cog_\) |
| `orgId` | string | Yes | Devin organization ID \(prefixed with org-\) |
| `sessionId` | string | Yes | The session ID to archive |
#### Output
| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `sessionId` | string | Unique identifier for the session |
| `url` | string | URL to view the session in the Devin UI |
| `status` | string | Session status \(new, claimed, running, exit, error, suspended, resuming\) |
| `statusDetail` | string | Detailed status \(working, waiting_for_user, waiting_for_approval, finished, inactivity, etc.\) |
| `title` | string | Session title |
| `createdAt` | number | Unix timestamp when the session was created |
| `updatedAt` | number | Unix timestamp when the session was last updated |
| `acusConsumed` | number | ACUs consumed by the session |
| `tags` | json | Tags associated with the session \(array of strings\) |
| `pullRequests` | json | Pull requests created during the session \(\[\{pr_url, pr_state\}\]\) |
| `structuredOutput` | json | Structured output from the session |
| `playbookId` | string | Associated playbook ID |
| `isArchived` | boolean | Whether the session is archived |
### `devin_terminate_session`
Terminate a Devin session. Optionally archive the session instead of permanently terminating it.
#### Input
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Devin API key \(service user credential starting with cog_\) |
| `orgId` | string | Yes | Devin organization ID \(prefixed with org-\) |
| `sessionId` | string | Yes | The session ID to terminate |
| `archive` | boolean | No | Archive the session instead of permanently terminating it \(default: false\) |
#### Output
| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `sessionId` | string | Unique identifier for the session |
| `url` | string | URL to view the session in the Devin UI |
| `status` | string | Session status \(new, claimed, running, exit, error, suspended, resuming\) |
| `statusDetail` | string | Detailed status \(working, waiting_for_user, waiting_for_approval, finished, inactivity, etc.\) |
| `title` | string | Session title |
| `createdAt` | number | Unix timestamp when the session was created |
| `updatedAt` | number | Unix timestamp when the session was last updated |
| `acusConsumed` | number | ACUs consumed by the session |
| `tags` | json | Tags associated with the session \(array of strings\) |
| `pullRequests` | json | Pull requests created during the session \(\[\{pr_url, pr_state\}\]\) |
| `structuredOutput` | json | Structured output from the session |
| `playbookId` | string | Associated playbook ID |
| `isArchived` | boolean | Whether the session is archived |
apps/docs/content/docs/en/tools/greptile.mdx
+1 -1
View File
@@ -7,7 +7,7 @@ import { BlockInfoCard } from "@/components/ui/block-info-card"
<BlockInfoCard
type="greptile"
color="#FFFFFF"
color="#5DE195"
/>
{/* MANUAL-CONTENT-START:intro */}
apps/docs/content/docs/en/tools/quiver.mdx
+1 -1
View File
@@ -7,7 +7,7 @@ import { BlockInfoCard } from "@/components/ui/block-info-card"
<BlockInfoCard
type="quiver"
color="#000000"
color="#FFFFFF"
/>
{/* MANUAL-CONTENT-START:intro */}
apps/docs/content/docs/en/tools/wiza.mdx
+3 -23
View File
@@ -151,9 +151,9 @@ Enrich a company by name, domain, LinkedIn ID, or LinkedIn slug with detailed fi
| `company_country` | string | Country |
| `credits` | json | Credits deducted for this enrichment \(api_credits: \{ total, company_credits \}\) |
### `wiza_start_individual_reveal`
### `wiza_individual_reveal`
Start an individual reveal to enrich a contact via LinkedIn URL, name+company, or email
Reveal a contact via LinkedIn URL, name + company/domain, or email. Starts the reveal and polls until it resolves. Uses 2 credits per valid email and 5 credits per phone, charged only on success.
#### Input
@@ -168,26 +168,6 @@ Start an individual reveal to enrich a contact via LinkedIn URL, name+company, o
| `email` | string | No | Email address \(use alone or with other identifiers\) |
| `accept_work` | boolean | No | Whether to accept work emails \(email_options\) |
| `accept_personal` | boolean | No | Whether to accept personal emails \(email_options\) |
| `callback_url` | string | No | Optional URL to receive a callback with the reveal update |
#### Output
| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `id` | number | Individual reveal ID \(use with Get Individual Reveal\) |
| `status` | string | Reveal status: queued, resolving, finished, or failed |
| `is_complete` | boolean | Whether the reveal has completed |
### `wiza_get_individual_reveal`
Retrieve the status and enriched data for an individual reveal by ID
#### Input
| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `apiKey` | string | Yes | Wiza API key |
| `id` | string | Yes | Individual reveal ID returned from Start Individual Reveal |
#### Output
@@ -227,7 +207,7 @@ Retrieve the status and enriched data for an individual reveal by ID
| `company_linkedin` | string | Company LinkedIn URL |
| `company_location` | string | Full company location |
| `company_description` | string | Company description |
| `credits` | json | Credits deducted for this reveal \(api_credits: \{ total, email_credits, phone_credits, scrape_credits \}\) |
| `credits` | json | Credits consumed by the reveal |
### `wiza_get_credits`
apps/docs/content/docs/es/copilot/index.mdx
-95
View File
@@ -246,98 +246,3 @@ El uso de Copilot se factura por token del LLM subyacente. Si alcanzas tu límit
<Callout type="info">
Consulta la [página de cálculo de costos](/execution/costs) para detalles de facturación.
</Callout>
## Copilot MCP
Puedes usar Copilot como servidor MCP en tu editor o cliente de IA favorito. Esto te permite construir, probar, desplegar y gestionar flujos de trabajo de Sim directamente desde herramientas como Cursor, Claude Code, Claude Desktop y VS Code.
### Generar una clave API de Copilot
Para conectarte al servidor MCP de Copilot, necesitas una **clave API de Copilot**:
1. Ve a [sim.ai](https://sim.ai) e inicia sesión
2. Navega a **Configuración** → **Copilot**
3. Haz clic en **Generar clave API**
4. Copia la clave — solo se muestra una vez
La clave se verá como `sk-sim-copilot-...`. Usarás esto en la configuración a continuación.
### Cursor
Agrega lo siguiente a tu `.cursor/mcp.json` (nivel de proyecto) o configuración global de MCP de Cursor:
```json
{
"mcpServers": {
"sim-copilot": {
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
```
Reemplaza `YOUR_COPILOT_API_KEY` con la clave que generaste anteriormente.
### Claude Code
Ejecuta el siguiente comando para añadir el servidor MCP de Copilot:
```bash
claude mcp add sim-copilot \
--transport http \
https://www.sim.ai/api/mcp/copilot \
--header "X-API-Key: YOUR_COPILOT_API_KEY"
```
Reemplaza `YOUR_COPILOT_API_KEY` con tu clave.
### Claude Desktop
Claude Desktop requiere [`mcp-remote`](https://www.npmjs.com/package/mcp-remote) para conectarse a servidores MCP basados en HTTP. Añade lo siguiente a tu archivo de configuración de Claude Desktop (`~/Library/Application Support/Claude/claude_desktop_config.json` en macOS):
```json
{
"mcpServers": {
"sim-copilot": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://www.sim.ai/api/mcp/copilot",
"--header",
"X-API-Key: YOUR_COPILOT_API_KEY"
]
}
}
}
```
Reemplaza `YOUR_COPILOT_API_KEY` con tu clave.
### VS Code
Añade lo siguiente a tu `settings.json` de VS Code o al `.vscode/settings.json` del espacio de trabajo:
```json
{
"mcp": {
"servers": {
"sim-copilot": {
"type": "http",
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
}
```
Reemplaza `YOUR_COPILOT_API_KEY` con tu clave.
<Callout type="info">
Para implementaciones auto-alojadas, reemplaza `https://www.sim.ai` con tu URL de Sim auto-alojada.
</Callout>
apps/docs/content/docs/fr/copilot/index.mdx
-95
View File
@@ -246,98 +246,3 @@ L'utilisation de Copilot est facturée par jeton du LLM sous-jacent. Si vous att
<Callout type="info">
Consultez la [page de calcul des coûts](/execution/costs) pour les détails de facturation.
</Callout>
## Copilot MCP
Vous pouvez utiliser Copilot comme serveur MCP dans votre éditeur ou client IA préféré. Cela vous permet de créer, tester, déployer et gérer des workflows Sim directement depuis des outils comme Cursor, Claude Code, Claude Desktop et VS Code.
### Générer une clé API Copilot
Pour vous connecter au serveur MCP Copilot, vous avez besoin d'une **clé API Copilot** :
1. Rendez-vous sur [sim.ai](https://sim.ai) et connectez-vous
2. Accédez à **Paramètres** → **Copilot**
3. Cliquez sur **Générer une clé API**
4. Copiez la clé — elle n'est affichée qu'une seule fois
La clé ressemblera à `sk-sim-copilot-...`. Vous l'utiliserez dans la configuration ci-dessous.
### Cursor
Ajoutez ce qui suit à votre `.cursor/mcp.json` (niveau projet) ou aux paramètres MCP globaux de Cursor :
```json
{
"mcpServers": {
"sim-copilot": {
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
```
Remplacez `YOUR_COPILOT_API_KEY` par la clé que vous avez générée ci-dessus.
### Claude Code
Exécutez la commande suivante pour ajouter le serveur MCP Copilot :
```bash
claude mcp add sim-copilot \
--transport http \
https://www.sim.ai/api/mcp/copilot \
--header "X-API-Key: YOUR_COPILOT_API_KEY"
```
Remplacez `YOUR_COPILOT_API_KEY` par votre clé.
### Claude Desktop
Claude Desktop nécessite [`mcp-remote`](https://www.npmjs.com/package/mcp-remote) pour se connecter aux serveurs MCP basés sur HTTP. Ajoutez ce qui suit à votre fichier de configuration Claude Desktop (`~/Library/Application Support/Claude/claude_desktop_config.json` sur macOS) :
```json
{
"mcpServers": {
"sim-copilot": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://www.sim.ai/api/mcp/copilot",
"--header",
"X-API-Key: YOUR_COPILOT_API_KEY"
]
}
}
}
```
Remplacez `YOUR_COPILOT_API_KEY` par votre clé.
### VS Code
Ajoutez ce qui suit à votre `settings.json` VS Code ou à votre `.vscode/settings.json` d'espace de travail :
```json
{
"mcp": {
"servers": {
"sim-copilot": {
"type": "http",
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
}
```
Remplacez `YOUR_COPILOT_API_KEY` par votre clé.
<Callout type="info">
Pour les déploiements auto-hébergés, remplacez `https://www.sim.ai` par votre URL Sim auto-hébergée.
</Callout>
apps/docs/content/docs/ja/copilot/index.mdx
-95
View File
@@ -246,98 +246,3 @@ Copilotの使用量は、基盤となるLLMからのトークン単位で課金
<Callout type="info">
課金の詳細については、[コスト計算ページ](/execution/costs)を参照してください。
</Callout>
## Copilot MCP
お気に入りのエディタやAIクライアントで、CopilotをMCPサーバーとして使用できます。これにより、Cursor、Claude Code、Claude Desktop、VS Codeなどのツールから直接、Simワークフローの構築、テスト、デプロイ、管理が可能になります。
### Copilot APIキーの生成
Copilot MCPサーバーに接続するには、**Copilot APIキー**が必要です。
1. [sim.ai](https://sim.ai)にアクセスしてサインイン
2. **設定** → **Copilot**に移動
3. **APIキーを生成**をクリック
4. キーをコピー(一度のみ表示されます)
キーは`sk-sim-copilot-...`のような形式です。以下の設定で使用します。
### Cursor
`.cursor/mcp.json`(プロジェクトレベル)またはグローバルなCursor MCP設定に以下を追加してください。
```json
{
"mcpServers": {
"sim-copilot": {
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
```
`YOUR_COPILOT_API_KEY`を上記で生成したキーに置き換えてください。
### Claude Code
次のコマンドを実行してCopilot MCPサーバーを追加します:
```bash
claude mcp add sim-copilot \
--transport http \
https://www.sim.ai/api/mcp/copilot \
--header "X-API-Key: YOUR_COPILOT_API_KEY"
```
`YOUR_COPILOT_API_KEY`をあなたのキーに置き換えてください。
### Claude Desktop
Claude DesktopはHTTPベースのMCPサーバーに接続するために[`mcp-remote`](https://www.npmjs.com/package/mcp-remote)が必要です。Claude Desktopの設定ファイル(macOSでは`~/Library/Application Support/Claude/claude_desktop_config.json`)に以下を追加してください:
```json
{
"mcpServers": {
"sim-copilot": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://www.sim.ai/api/mcp/copilot",
"--header",
"X-API-Key: YOUR_COPILOT_API_KEY"
]
}
}
}
```
`YOUR_COPILOT_API_KEY`をあなたのキーに置き換えてください。
### VS Code
VS Codeの`settings.json`またはワークスペースの`.vscode/settings.json`に以下を追加してください:
```json
{
"mcp": {
"servers": {
"sim-copilot": {
"type": "http",
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
}
```
`YOUR_COPILOT_API_KEY`をあなたのキーに置き換えてください。
<Callout type="info">
セルフホスト型デプロイメントの場合は、`https://www.sim.ai`をあなたのセルフホスト型SimのURLに置き換えてください。
</Callout>
apps/docs/content/docs/zh/copilot/index.mdx
-95
View File
@@ -246,98 +246,3 @@ Copilot 的使用按底层 LLM 的 token 计费。如果达到使用上限，Cop
<Callout type="info">
计费详情请参见 [成本计算页面](/execution/costs)。
</Callout>
## Copilot MCP
你可以在常用编辑器或 AI 客户端中将 Copilot 作为 MCP 服务器使用。这样可以直接在 Cursor、Claude Code、Claude Desktop 和 VS Code 等工具中构建、测试、部署和管理 Sim 工作流。
### 生成 Copilot API 密钥
要连接 Copilot MCP 服务器，你需要一个 **Copilot API 密钥**：
1. 访问 [sim.ai](https://sim.ai) 并登录
2. 进入 **设置** → **Copilot**
3. 点击 **生成 API 密钥**
4. 复制密钥 — 该密钥只显示一次
密钥类似于 `sk-sim-copilot-...`。你将在下方配置中用到它。
### Cursor
将以下内容添加到你的 `.cursor/mcp.json`（项目级）或全局 Cursor MCP 设置中：
```json
{
"mcpServers": {
"sim-copilot": {
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
```
将 `YOUR_COPILOT_API_KEY` 替换为你上面生成的密钥。
### Claude Code
运行以下命令以添加 Copilot MCP 服务器：
```bash
claude mcp add sim-copilot \
--transport http \
https://www.sim.ai/api/mcp/copilot \
--header "X-API-Key: YOUR_COPILOT_API_KEY"
```
将 `YOUR_COPILOT_API_KEY` 替换为你的密钥。
### Claude Desktop
Claude Desktop 需要 [`mcp-remote`](https://www.npmjs.com/package/mcp-remote) 以连接基于 HTTP 的 MCP 服务器。请将以下内容添加到你的 Claude Desktop 配置文件（macOS 上为 `~/Library/Application Support/Claude/claude_desktop_config.json`）：
```json
{
"mcpServers": {
"sim-copilot": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://www.sim.ai/api/mcp/copilot",
"--header",
"X-API-Key: YOUR_COPILOT_API_KEY"
]
}
}
}
```
将 `YOUR_COPILOT_API_KEY` 替换为你的密钥。
### VS Code
请将以下内容添加到你的 VS Code `settings.json` 或工作区 `.vscode/settings.json`：
```json
{
"mcp": {
"servers": {
"sim-copilot": {
"type": "http",
"url": "https://www.sim.ai/api/mcp/copilot",
"headers": {
"X-API-Key": "YOUR_COPILOT_API_KEY"
}
}
}
}
}
```
将 `YOUR_COPILOT_API_KEY` 替换为你的密钥。
<Callout type="info">
对于自托管部署，请将 `https://www.sim.ai` 替换为你的自托管 Sim URL。
</Callout>
apps/sim/app/(landing)/integrations/(shell)/[slug]/page.tsx
+73
View File
@@ -221,6 +221,8 @@ export default async function IntegrationPage({ params }: { params: Promise<{ sl
const { name, description, longDescription, bgColor, docsUrl, operations, triggers, authType } =
integration
const landingContent = integration.landingContent
const IconComponent = blockTypeToIconMap[integration.type]
const faqs = buildFAQs(integration)
const relatedSlugs = getRelatedSlugs(slug, operations, authType)
@@ -443,6 +445,77 @@ export default async function IntegrationPage({ params }: { params: Promise<{ sl
</>
)}
{/* Install / Add to workspace (integration-specific) */}
{landingContent?.install && (
<>
<section aria-labelledby='install-heading' className='px-6 py-10'>
<h2
id='install-heading'
className='mb-4 text-[20px] text-white leading-[100%] tracking-[-0.02em]'
>
{landingContent.install.heading}
</h2>
<p className='mb-6 max-w-[700px] text-[15px] text-[var(--landing-text-body)] leading-[150%] tracking-[0.02em]'>
{landingContent.install.intro}
</p>
<ol className='space-y-4' aria-label={`Steps to add ${name}`}>
{landingContent.install.steps.map((item, index) => (
<li key={item.title} className='flex gap-4'>
<span
className='mt-0.5 flex size-7 shrink-0 items-center justify-center rounded-full border border-[var(--landing-border-strong)] font-martian-mono text-[11px] text-[var(--landing-text-subtle)]'
aria-hidden='true'
>
{String(index + 1).padStart(2, '0')}
</span>
<div>
<h3 className='mb-1 text-[15px] text-white tracking-[-0.02em]'>
{item.title}
</h3>
<p className='text-[14px] text-[var(--landing-text-body)] leading-[150%] tracking-[0.02em]'>
{item.body}
</p>
</div>
</li>
))}
</ol>
<div className='mt-8 flex flex-wrap gap-2'>
<IntegrationCtaButton
label={`Add to ${name}`}
className='inline-flex h-[32px] items-center gap-2 rounded-[5px] border border-white bg-white px-2.5 font-season text-black text-sm transition-colors hover:border-[#E0E0E0] hover:bg-[#E0E0E0]'
>
Add to {name}
</IntegrationCtaButton>
</div>
</section>
<div className='h-px w-full bg-[var(--landing-bg-elevated)]' />
</>
)}
{/* Privacy & data (integration-specific) */}
{landingContent?.privacy && (
<>
<section aria-labelledby='privacy-heading' className='px-6 py-10'>
<h2
id='privacy-heading'
className='mb-4 text-[20px] text-white leading-[100%] tracking-[-0.02em]'
>
Privacy & data
</h2>
<p className='max-w-[700px] text-[15px] text-[var(--landing-text-body)] leading-[150%] tracking-[0.02em]'>
{landingContent.privacy.body}{' '}
<Link
href={landingContent.privacy.href}
className='text-[var(--landing-text)] underline underline-offset-2 hover:text-white'
>
Privacy Policy
</Link>
.
</p>
</section>
<div className='h-px w-full bg-[var(--landing-bg-elevated)]' />
</>
)}
{/* How to automate */}
<section aria-labelledby='how-it-works-heading' className='px-6 py-10'>
<h2
apps/sim/app/(landing)/integrations/data/types.ts
+63
View File
@@ -0,0 +1,63 @@
// Shared types for the integrations section of the landing site.
// Mirrors the shape written by scripts/generate-docs.ts → writeIntegrationsJson().
export type AuthType = 'oauth' | 'api-key' | 'none'
interface TriggerInfo {
id: string
name: string
description: string
}
interface OperationInfo {
name: string
description: string
}
export interface FAQItem {
question: string
answer: string
}
export interface IntegrationInstallStep {
title: string
body: string
}
export interface IntegrationLandingContent {
/**
* Install walkthrough for OAuth apps whose connection lives behind sign-in.
* Provides the "Add to {app}" instructions that app marketplaces require
* when the install button sits behind a login.
*/
install?: {
heading: string
intro: string
steps: IntegrationInstallStep[]
}
/** Short data-handling summary shown next to a privacy-policy link. */
privacy?: {
body: string
href: string
}
}
export interface Integration {
type: string
slug: string
name: string
description: string
longDescription: string
bgColor: string
iconName: string
docsUrl: string
operations: OperationInfo[]
operationCount: number
triggers: TriggerInfo[]
triggerCount: number
authType: AuthType
category: string
integrationTypes?: string[]
tags?: string[]
landingContent?: IntegrationLandingContent
}
apps/sim/app/(landing)/models/utils.ts
+3
View File
@@ -8,6 +8,9 @@ const PROVIDER_PREFIXES: Record<string, string[]> = {
bedrock: ['bedrock/'],
cerebras: ['cerebras/'],
fireworks: ['fireworks/'],
together: ['together/'],
baseten: ['baseten/'],
'ollama-cloud': ['ollama-cloud/'],
groq: ['groq/'],
openrouter: ['openrouter/'],
vllm: ['vllm/'],
apps/sim/app/.well-known/oauth-authorization-server/api/mcp/copilot/route.ts
+2 -2
View File
@@ -1,6 +1,6 @@
import type { NextResponse } from 'next/server'
import { createMcpAuthorizationServerMetadataResponse } from '@/lib/mcp/oauth-discovery'
import { copilotMcpDeprecatedResponse } from '@/lib/mcp/copilot-deprecated'
export async function GET(): Promise<NextResponse> {
return createMcpAuthorizationServerMetadataResponse()
return copilotMcpDeprecatedResponse()
}
apps/sim/app/.well-known/oauth-protected-resource/api/mcp/copilot/route.ts
+2 -2
View File
@@ -1,6 +1,6 @@
import type { NextResponse } from 'next/server'
import { createMcpProtectedResourceMetadataResponse } from '@/lib/mcp/oauth-discovery'
import { copilotMcpDeprecatedResponse } from '@/lib/mcp/copilot-deprecated'
export async function GET(): Promise<NextResponse> {
return createMcpProtectedResourceMetadataResponse()
return copilotMcpDeprecatedResponse()
}
apps/sim/app/api/auth/oauth/microsoft/files/route.ts
+94 -36
View File
@@ -8,13 +8,57 @@ import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { getCredential, refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { GRAPH_ID_PATTERN } from '@/tools/microsoft_excel/utils'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('MicrosoftFilesAPI')
/**
* Get Excel files from Microsoft OneDrive
* Microsoft Graph paginates `search()` results via the `@odata.nextLink`
* absolute URL in the response body. Request the largest page (`$top` caps at
* 999) and drain following nextLink, bounded by a page cap.
* See https://learn.microsoft.com/en-us/graph/paging
*/
const MICROSOFT_FILES_PAGE_SIZE = 999
const MAX_MICROSOFT_FILES_PAGES = 20
interface MicrosoftGraphFile {
id: string
name?: string
mimeType?: string
webUrl?: string
size?: number
createdDateTime?: string
lastModifiedDateTime?: string
thumbnails?: Array<{ small?: { url?: string }; medium?: { url?: string } }>
createdBy?: { user?: { displayName?: string; email?: string } }
}
/**
* The shared `/api/auth/oauth/microsoft/files` route serves both the
* `microsoft.excel` and `microsoft.word` selectors. The two are distinguished
* by the `fileType` query parameter the selector forwards (defaulting to
* `excel` for backward compatibility), which drives both the search-query
* extension hint and the server-side result filter.
*/
const FILE_TYPE_CONFIG = {
excel: {
extension: '.xlsx',
mimeType: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
},
word: {
extension: '.docx',
mimeType: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
},
} as const
type MicrosoftFileType = keyof typeof FILE_TYPE_CONFIG
/**
* Get Excel or Word files from Microsoft OneDrive / SharePoint. The
* `fileType` query parameter selects which Office document type to return
* (defaults to `excel`).
*/
export const GET = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
@@ -27,6 +71,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
query: searchParams.get('query') ?? undefined,
driveId: searchParams.get('driveId') ?? undefined,
workflowId: searchParams.get('workflowId') ?? undefined,
fileType: searchParams.get('fileType') ?? undefined,
})
if (!parsedQuery.success) {
@@ -40,6 +85,9 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
const { credentialId, driveId, workflowId } = parsedQuery.data
const query = parsedQuery.data.query ?? ''
const fileType: MicrosoftFileType = parsedQuery.data.fileType ?? 'excel'
const { extension, mimeType: targetMimeType } = FILE_TYPE_CONFIG[fileType]
const authz = await authorizeCredentialUse(request, {
credentialId,
workflowId,
@@ -72,11 +120,8 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
return NextResponse.json({ error: 'Failed to obtain valid access token' }, { status: 401 })
}
// Build search query for Excel files
let searchQuery = '.xlsx'
if (query) {
searchQuery = `${query} .xlsx`
}
// Build search query for the requested Office document type
const searchQuery = query ? `${query} ${extension}` : extension
// Build the query parameters for Microsoft Graph API
const searchParams_new = new URLSearchParams()
@@ -84,7 +129,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
'$select',
'id,name,mimeType,webUrl,thumbnails,createdDateTime,lastModifiedDateTime,size,createdBy'
)
searchParams_new.append('$top', '50')
searchParams_new.append('$top', String(MICROSOFT_FILES_PAGE_SIZE))
// When driveId is provided (SharePoint), search within that specific drive.
// Otherwise, search the user's personal OneDrive.
@@ -99,44 +144,57 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
}
const drivePath = driveId ? `drives/${driveId}` : 'me/drive'
const response = await fetch(
`https://graph.microsoft.com/v1.0/${drivePath}/root/search(q='${encodeURIComponent(searchQuery)}')?${searchParams_new.toString()}`,
{
const rawFiles: MicrosoftGraphFile[] = []
let nextUrl: string | undefined =
`https://graph.microsoft.com/v1.0/${drivePath}/root/search(q='${encodeURIComponent(searchQuery)}')?${searchParams_new.toString()}`
for (let page = 0; page < MAX_MICROSOFT_FILES_PAGES && nextUrl; page++) {
const response = await fetch(nextUrl, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
}
)
if (!response.ok) {
const errorData = await response.json().catch(() => ({ error: { message: 'Unknown error' } }))
logger.error(`[${requestId}] Microsoft Graph API error`, {
status: response.status,
error: errorData.error?.message || 'Failed to fetch Excel files from Microsoft OneDrive',
})
return NextResponse.json(
{
error: errorData.error?.message || 'Failed to fetch Excel files from Microsoft OneDrive',
},
{ status: response.status }
)
if (!response.ok) {
const errorData = await response
.json()
.catch(() => ({ error: { message: 'Unknown error' } }))
logger.error(`[${requestId}] Microsoft Graph API error`, {
status: response.status,
error: errorData.error?.message || 'Failed to fetch files from Microsoft OneDrive',
})
return NextResponse.json(
{
error: errorData.error?.message || 'Failed to fetch files from Microsoft OneDrive',
},
{ status: response.status }
)
}
const data = await response.json()
rawFiles.push(...((data.value as MicrosoftGraphFile[]) || []))
const nextLink = getGraphNextPageUrl(data)
nextUrl = nextLink ? assertGraphNextPageUrl(nextLink) : undefined
if (nextUrl && page === MAX_MICROSOFT_FILES_PAGES - 1) {
logger.warn(
`[${requestId}] Microsoft files search hit pagination cap; list may be incomplete`,
{ fileType, pages: MAX_MICROSOFT_FILES_PAGES, collected: rawFiles.length }
)
}
}
const data = await response.json()
let files = data.value || []
// Transform Microsoft Graph response to match expected format and filter for Excel files
files = files
// Transform Microsoft Graph response and filter to the requested file type
const files = rawFiles
.filter(
(file: any) =>
file.name?.toLowerCase().endsWith('.xlsx') ||
file.mimeType === 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
(file: MicrosoftGraphFile) =>
file.name?.toLowerCase().endsWith(extension) || file.mimeType === targetMimeType
)
.map((file: any) => ({
.map((file: MicrosoftGraphFile) => ({
id: file.id,
name: file.name,
mimeType:
file.mimeType || 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
mimeType: file.mimeType || targetMimeType,
iconLink: file.thumbnails?.[0]?.small?.url,
webViewLink: file.webUrl,
thumbnailLink: file.thumbnails?.[0]?.medium?.url,
@@ -155,7 +213,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
return NextResponse.json({ files }, { status: 200 })
} catch (error) {
logger.error(`[${requestId}] Error fetching Excel files from Microsoft OneDrive`, error)
logger.error(`[${requestId}] Error fetching files from Microsoft OneDrive`, error)
return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
}
})
apps/sim/app/api/auth/sso/register/route.test.ts
+196
View File
@@ -0,0 +1,196 @@
/**
* @vitest-environment node
*/
import { createEnvMock, createMockRequest } from '@sim/testing'
import { beforeEach, describe, expect, it, vi } from 'vitest'
const {
mockGetSession,
mockRegisterSSOProvider,
mockHasSSOAccess,
mockValidateUrlWithDNS,
dbState,
memberTable,
ssoProviderTable,
} = vi.hoisted(() => ({
mockGetSession: vi.fn(),
mockRegisterSSOProvider: vi.fn(),
mockHasSSOAccess: vi.fn(),
mockValidateUrlWithDNS: vi.fn(),
dbState: { members: [] as any[], providers: [] as any[] },
memberTable: {
userId: 'member.userId',
organizationId: 'member.organizationId',
role: 'member.role',
},
ssoProviderTable: {
id: 'sso.id',
providerId: 'sso.providerId',
domain: 'sso.domain',
issuer: 'sso.issuer',
userId: 'sso.userId',
organizationId: 'sso.organizationId',
oidcConfig: 'sso.oidcConfig',
samlConfig: 'sso.samlConfig',
},
}))
function makeBuilder(rows: any[]): any {
const thenable: any = Promise.resolve(rows)
thenable.where = (condition: any) => {
const values = condition?.values
if (Array.isArray(values) && values.length > 0) {
const target = String(values[values.length - 1]).toLowerCase()
return makeBuilder(rows.filter((r) => String(r.domain ?? '').toLowerCase() === target))
}
return makeBuilder(rows)
}
thenable.limit = () => Promise.resolve(rows)
thenable.orderBy = () => Promise.resolve(rows)
return thenable
}
vi.mock('@sim/db', () => ({
db: {
select: () => ({
from: (table: unknown) =>
makeBuilder(table === memberTable ? dbState.members : dbState.providers),
}),
},
member: memberTable,
ssoProvider: ssoProviderTable,
}))
vi.mock('@/lib/auth', () => ({
getSession: mockGetSession,
auth: { api: { registerSSOProvider: mockRegisterSSOProvider } },
}))
vi.mock('@/lib/billing', () => ({
hasSSOAccess: mockHasSSOAccess,
}))
vi.mock('@/lib/auth/sso/domain', () => ({
normalizeSSODomain: (input: unknown): string | null => {
if (typeof input !== 'string') return null
const value = input.trim().toLowerCase()
return /^[a-z0-9-]+(\.[a-z0-9-]+)+$/.test(value) ? value : null
},
}))
vi.mock('@/lib/core/security/input-validation.server', () => ({
validateUrlWithDNS: mockValidateUrlWithDNS,
secureFetchWithPinnedIP: vi.fn(),
}))
vi.mock('@/lib/core/config/env', () => createEnvMock({ SSO_ENABLED: 'true' }))
import { POST } from '@/app/api/auth/sso/register/route'
const OIDC_BODY = {
providerType: 'oidc' as const,
providerId: 'acme-oidc',
issuer: 'https://idp.acme.com',
domain: 'acme.com',
clientId: 'client-id',
clientSecret: 'client-secret',
authorizationEndpoint: 'https://idp.acme.com/authorize',
tokenEndpoint: 'https://idp.acme.com/token',
userInfoEndpoint: 'https://idp.acme.com/userinfo',
jwksEndpoint: 'https://idp.acme.com/jwks',
}
function request(body: Record<string, unknown>) {
return createMockRequest('POST', body)
}
describe('POST /api/auth/sso/register', () => {
beforeEach(() => {
vi.clearAllMocks()
dbState.members = []
dbState.providers = []
mockGetSession.mockResolvedValue({ user: { id: 'u1' } })
mockHasSSOAccess.mockResolvedValue(true)
mockValidateUrlWithDNS.mockResolvedValue({ isValid: true, resolvedIP: '1.2.3.4' })
mockRegisterSSOProvider.mockResolvedValue({ providerId: 'acme-oidc' })
})
it('rejects callers without an Enterprise plan', async () => {
mockHasSSOAccess.mockResolvedValue(false)
const res = await POST(request({ ...OIDC_BODY, orgId: 'org1' }))
expect(res.status).toBe(403)
expect(mockRegisterSSOProvider).not.toHaveBeenCalled()
})
it('rejects callers who are not an admin/owner of the target org', async () => {
dbState.members = [{ organizationId: 'org1', role: 'member' }]
const res = await POST(request({ ...OIDC_BODY, orgId: 'org1' }))
expect(res.status).toBe(403)
expect(mockRegisterSSOProvider).not.toHaveBeenCalled()
})
it('rejects an invalid domain', async () => {
dbState.members = [{ organizationId: 'org1', role: 'owner' }]
const res = await POST(request({ ...OIDC_BODY, domain: 'not-a-domain', orgId: 'org1' }))
expect(res.status).toBe(400)
expect(mockRegisterSSOProvider).not.toHaveBeenCalled()
})
it('rejects a domain already registered by another organization', async () => {
dbState.members = [{ organizationId: 'org-attacker', role: 'owner' }]
dbState.providers = [{ domain: 'acme.com', userId: 'u-victim', organizationId: 'org-victim' }]
const res = await POST(request({ ...OIDC_BODY, orgId: 'org-attacker' }))
const json = await res.json()
expect(res.status).toBe(409)
expect(json.code).toBe('SSO_DOMAIN_ALREADY_REGISTERED')
expect(mockRegisterSSOProvider).not.toHaveBeenCalled()
})
it('matches conflicts across casing variants', async () => {
dbState.members = [{ organizationId: 'org-attacker', role: 'owner' }]
dbState.providers = [{ domain: 'ACME.com', userId: 'u-victim', organizationId: 'org-victim' }]
const res = await POST(request({ ...OIDC_BODY, orgId: 'org-attacker' }))
expect(res.status).toBe(409)
expect(mockRegisterSSOProvider).not.toHaveBeenCalled()
})
it('registers when the domain is unclaimed', async () => {
dbState.members = [{ organizationId: 'org1', role: 'owner' }]
const res = await POST(request({ ...OIDC_BODY, orgId: 'org1' }))
expect(res.status).toBe(200)
expect(mockRegisterSSOProvider).toHaveBeenCalledTimes(1)
})
it('allows the owning tenant to update its own provider for the same domain', async () => {
dbState.members = [{ organizationId: 'org1', role: 'owner' }]
dbState.providers = [{ domain: 'acme.com', userId: 'u1', organizationId: 'org1' }]
const res = await POST(request({ ...OIDC_BODY, orgId: 'org1' }))
expect(res.status).toBe(200)
expect(mockRegisterSSOProvider).toHaveBeenCalledTimes(1)
})
it('lets an org admin adopt their own user-scoped provider for the same domain', async () => {
dbState.members = [{ organizationId: 'org1', role: 'owner' }]
dbState.providers = [{ domain: 'acme.com', userId: 'u1', organizationId: null }]
const res = await POST(request({ ...OIDC_BODY, orgId: 'org1' }))
expect(res.status).toBe(200)
expect(mockRegisterSSOProvider).toHaveBeenCalledTimes(1)
})
it("still blocks an org admin from claiming another user's user-scoped domain", async () => {
dbState.members = [{ organizationId: 'org1', role: 'owner' }]
dbState.providers = [{ domain: 'acme.com', userId: 'someone-else', organizationId: null }]
const res = await POST(request({ ...OIDC_BODY, orgId: 'org1' }))
expect(res.status).toBe(409)
expect(mockRegisterSSOProvider).not.toHaveBeenCalled()
})
it('normalizes the domain before persisting it', async () => {
dbState.members = [{ organizationId: 'org1', role: 'owner' }]
const res = await POST(request({ ...OIDC_BODY, domain: 'ACME.com', orgId: 'org1' }))
expect(res.status).toBe(200)
expect(mockRegisterSSOProvider).toHaveBeenCalledTimes(1)
const config = mockRegisterSSOProvider.mock.calls[0][0].body
expect(config.domain).toBe('acme.com')
})
})
apps/sim/app/api/auth/sso/register/route.ts
+54 -2
View File
@@ -1,11 +1,12 @@
import { db, member, ssoProvider } from '@sim/db'
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { and, eq } from 'drizzle-orm'
import { and, eq, sql } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server'
import { ssoRegistrationContract } from '@/lib/api/contracts/auth'
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { auth, getSession } from '@/lib/auth'
import { normalizeSSODomain } from '@/lib/auth/sso/domain'
import { hasSSOAccess } from '@/lib/billing'
import { env } from '@/lib/core/config/env'
import {
@@ -51,7 +52,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
if (!parsed.success) return parsed.response
const body = parsed.data.body
const { providerId, issuer, domain, providerType, mapping, orgId } = body
const { providerId, issuer, providerType, mapping, orgId } = body
if (orgId) {
const [membership] = await db
@@ -67,6 +68,48 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
}
}
const domain = normalizeSSODomain(body.domain)
if (!domain) {
return NextResponse.json({ error: 'Enter a valid domain like company.com' }, { status: 400 })
}
const isOwnedByCaller = (provider: {
userId: string | null
organizationId: string | null
}): boolean => {
if (provider.userId === session.user.id && !provider.organizationId) return true
return orgId ? provider.organizationId === orgId : false
}
const findDomainConflict = async () =>
(
await db
.select({
userId: ssoProvider.userId,
organizationId: ssoProvider.organizationId,
})
.from(ssoProvider)
.where(sql`lower(${ssoProvider.domain}) = ${domain}`)
).find((provider) => !isOwnedByCaller(provider))
const domainConflictResponse = () =>
NextResponse.json(
{
error: 'This domain is already registered for SSO by another organization.',
code: 'SSO_DOMAIN_ALREADY_REGISTERED',
},
{ status: 409 }
)
if (await findDomainConflict()) {
logger.warn('Rejected SSO registration for domain owned by another tenant', {
domain,
orgId,
userId: session.user.id,
})
return domainConflictResponse()
}
const headers: Record<string, string> = {}
request.headers.forEach((value, key) => {
headers[key] = value
@@ -408,6 +451,15 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
),
})
if (await findDomainConflict()) {
logger.warn('Rejected SSO registration: domain was claimed during registration', {
domain,
orgId,
userId: session.user.id,
})
return domainConflictResponse()
}
const registration = await auth.api.registerSSOProvider({
body: providerConfig,
headers,
apps/sim/app/api/billing/route.ts
-1
View File
@@ -140,7 +140,6 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
members: rawBillingData.members.map((m) => ({
...m,
joinedAt: m.joinedAt.toISOString(),
lastActive: m.lastActive?.toISOString() || null,
})),
}
apps/sim/app/api/copilot/chat/stop/route.test.ts
+53 -143
View File
@@ -1,79 +1,19 @@
/**
* @vitest-environment node
*/
import { authMockFns } from '@sim/testing'
import { authMockFns, dbChainMock, dbChainMockFns, resetDbChainMock } from '@sim/testing'
import { NextRequest } from 'next/server'
import { beforeEach, describe, expect, it, vi } from 'vitest'
const {
mockSelect,
mockFrom,
mockWhereSelect,
mockLimit,
mockForUpdate,
mockUpdate,
mockSet,
mockWhereUpdate,
mockReturning,
mockPublishStatusChanged,
mockSql,
mockTransaction,
} = vi.hoisted(() => {
const mockSelect = vi.fn()
const mockFrom = vi.fn()
const mockWhereSelect = vi.fn()
const mockLimit = vi.fn()
const mockForUpdate = vi.fn()
const mockUpdate = vi.fn()
const mockSet = vi.fn()
const mockWhereUpdate = vi.fn()
const mockReturning = vi.fn()
const mockPublishStatusChanged = vi.fn()
const mockSql = vi.fn((strings: TemplateStringsArray, ...values: unknown[]) => ({
strings,
values,
}))
const mockTransaction = vi.fn(
(callback: (tx: { select: typeof mockSelect; update: typeof mockUpdate }) => unknown) =>
callback({ select: mockSelect, update: mockUpdate })
)
vi.mock('@sim/db', () => dbChainMock)
return {
mockSelect,
mockFrom,
mockWhereSelect,
mockLimit,
mockForUpdate,
mockUpdate,
mockSet,
mockWhereUpdate,
mockReturning,
mockPublishStatusChanged,
mockSql,
mockTransaction,
}
})
vi.mock('@sim/db/schema', () => ({
copilotChats: {
id: 'copilotChats.id',
userId: 'copilotChats.userId',
workspaceId: 'copilotChats.workspaceId',
messages: 'copilotChats.messages',
conversationId: 'copilotChats.conversationId',
},
const { mockAppendCopilotChatMessages, mockPublishStatusChanged } = vi.hoisted(() => ({
mockAppendCopilotChatMessages: vi.fn(),
mockPublishStatusChanged: vi.fn(),
}))
vi.mock('@sim/db', () => ({
db: {
transaction: mockTransaction,
},
}))
vi.mock('drizzle-orm', () => ({
and: vi.fn((...conditions: unknown[]) => ({ conditions, type: 'and' })),
eq: vi.fn((field: unknown, value: unknown) => ({ field, value, type: 'eq' })),
sql: mockSql,
vi.mock('@/lib/copilot/chat/messages-store', () => ({
appendCopilotChatMessages: mockAppendCopilotChatMessages,
}))
vi.mock('@/lib/copilot/tasks', () => ({
@@ -92,39 +32,33 @@ function createRequest(body: Record<string, unknown>) {
})
}
/**
* Sequence the two in-tx reads `finalizeAssistantTurn` issues: the chat row
* (`FOR UPDATE ... LIMIT 1`) and the last-message lookup that drives dedup
* (both terminate on `.limit(1)`).
*/
function mockReads(opts: {
chat: Record<string, unknown> | null
last?: { messageId: string; role: string }
}) {
dbChainMockFns.limit.mockResolvedValueOnce(opts.chat ? [opts.chat] : [])
dbChainMockFns.limit.mockResolvedValueOnce(opts.last ? [opts.last] : [])
}
describe('copilot chat stop route', () => {
beforeEach(() => {
vi.clearAllMocks()
// Drain the once-queue (clearAllMocks/resetDbChainMock don't), then restore defaults.
dbChainMockFns.limit.mockReset()
resetDbChainMock()
authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
mockLimit.mockResolvedValue([
{
workspaceId: 'ws-1',
messages: [{ id: 'stream-1', role: 'user', content: 'hello' }],
conversationId: 'stream-1',
},
])
mockForUpdate.mockReturnValue({ limit: mockLimit })
mockWhereSelect.mockReturnValue({ for: mockForUpdate })
mockFrom.mockReturnValue({ where: mockWhereSelect })
mockSelect.mockReturnValue({ from: mockFrom })
mockReturning.mockResolvedValue([{ workspaceId: 'ws-1' }])
mockWhereUpdate.mockReturnValue({ returning: mockReturning })
mockSet.mockReturnValue({ where: mockWhereUpdate })
mockUpdate.mockReturnValue({ set: mockSet })
})
it('returns 401 when unauthenticated', async () => {
authMockFns.mockGetSession.mockResolvedValueOnce(null)
const response = await POST(
createRequest({
chatId: 'chat-1',
streamId: 'stream-1',
content: '',
})
createRequest({ chatId: 'chat-1', streamId: 'stream-1', content: '' })
)
expect(response.status).toBe(401)
@@ -132,41 +66,37 @@ describe('copilot chat stop route', () => {
})
it('is a no-op when the chat is missing', async () => {
mockLimit.mockResolvedValueOnce([])
mockReads({ chat: null })
const response = await POST(
createRequest({
chatId: 'missing-chat',
streamId: 'stream-1',
content: '',
})
createRequest({ chatId: 'missing-chat', streamId: 'stream-1', content: '' })
)
expect(response.status).toBe(200)
expect(await response.json()).toEqual({ success: true })
expect(mockUpdate).not.toHaveBeenCalled()
expect(mockAppendCopilotChatMessages).not.toHaveBeenCalled()
})
it('appends a stopped assistant message even with no content', async () => {
mockReads({
chat: { workspaceId: 'ws-1', conversationId: 'stream-1', model: null },
last: { messageId: 'stream-1', role: 'user' },
})
const response = await POST(
createRequest({
chatId: 'chat-1',
streamId: 'stream-1',
content: '',
})
createRequest({ chatId: 'chat-1', streamId: 'stream-1', content: '' })
)
expect(response.status).toBe(200)
expect(await response.json()).toEqual({ success: true })
const setArg = mockSet.mock.calls[0]?.[0]
expect(setArg).toBeTruthy()
const setArg = dbChainMockFns.set.mock.calls[0]?.[0] as Record<string, unknown>
expect(setArg.conversationId).toBeNull()
expect(setArg.messages).toBeTruthy()
expect(Object.hasOwn(setArg, 'messages')).toBe(false)
const appendedPayload = JSON.parse(setArg.messages.values[1] as string)
expect(appendedPayload).toHaveLength(1)
expect(appendedPayload[0]).toMatchObject({
expect(mockAppendCopilotChatMessages).toHaveBeenCalledTimes(1)
const [, appended] = mockAppendCopilotChatMessages.mock.calls[0]
expect(appended[0]).toMatchObject({
role: 'assistant',
content: '',
contentBlocks: [{ type: 'complete', status: 'cancelled' }],
@@ -181,32 +111,21 @@ describe('copilot chat stop route', () => {
})
it('appends a stopped assistant message if the stream marker was already cleared', async () => {
mockLimit.mockResolvedValueOnce([
{
workspaceId: 'ws-1',
messages: [{ id: 'stream-1', role: 'user', content: 'hello' }],
conversationId: null,
},
])
mockReads({
chat: { workspaceId: 'ws-1', conversationId: null, model: null },
last: { messageId: 'stream-1', role: 'user' },
})
const response = await POST(
createRequest({
chatId: 'chat-1',
streamId: 'stream-1',
content: 'partial',
})
createRequest({ chatId: 'chat-1', streamId: 'stream-1', content: 'partial' })
)
expect(response.status).toBe(200)
expect(await response.json()).toEqual({ success: true })
const setArg = mockSet.mock.calls[0]?.[0]
expect(setArg.messages).toBeTruthy()
const appendedPayload = JSON.parse(setArg.messages.values[1] as string)
expect(appendedPayload[0]).toMatchObject({
role: 'assistant',
content: 'partial',
})
expect(mockAppendCopilotChatMessages).toHaveBeenCalledTimes(1)
const [, appended] = mockAppendCopilotChatMessages.mock.calls[0]
expect(appended[0]).toMatchObject({ role: 'assistant', content: 'partial' })
expect(mockPublishStatusChanged).toHaveBeenCalledWith({
workspaceId: 'ws-1',
@@ -217,28 +136,19 @@ describe('copilot chat stop route', () => {
})
it('republishes completed status when the assistant was already persisted', async () => {
mockLimit.mockResolvedValueOnce([
{
workspaceId: 'ws-1',
messages: [
{ id: 'stream-1', role: 'user', content: 'hello' },
{ id: 'assistant-1', role: 'assistant', content: 'partial' },
],
conversationId: null,
},
])
mockReads({
chat: { workspaceId: 'ws-1', conversationId: null, model: null },
last: { messageId: 'assistant-1', role: 'assistant' },
})
const response = await POST(
createRequest({
chatId: 'chat-1',
streamId: 'stream-1',
content: 'partial',
})
createRequest({ chatId: 'chat-1', streamId: 'stream-1', content: 'partial' })
)
expect(response.status).toBe(200)
expect(await response.json()).toEqual({ success: true })
expect(mockUpdate).not.toHaveBeenCalled()
expect(mockAppendCopilotChatMessages).not.toHaveBeenCalled()
expect(dbChainMockFns.set).not.toHaveBeenCalled()
expect(mockPublishStatusChanged).toHaveBeenCalledWith({
workspaceId: 'ws-1',
chatId: 'chat-1',
apps/sim/app/api/copilot/chat/update-messages/route.test.ts
+35 -14
View File
@@ -16,6 +16,7 @@ const {
mockSet,
mockUpdateWhere,
mockReturning,
mockReplaceCopilotChatMessages,
} = vi.hoisted(() => ({
mockSelect: vi.fn(),
mockFrom: vi.fn(),
@@ -25,15 +26,23 @@ const {
mockSet: vi.fn(),
mockUpdateWhere: vi.fn(),
mockReturning: vi.fn(),
mockReplaceCopilotChatMessages: vi.fn(),
}))
vi.mock('@sim/db', () => ({
db: {
select: mockSelect,
update: mockUpdate,
transaction: async (
cb: (tx: { update: typeof mockUpdate; select: typeof mockSelect }) => unknown
) => cb({ update: mockUpdate, select: mockSelect }),
},
}))
vi.mock('@/lib/copilot/chat/messages-store', () => ({
replaceCopilotChatMessages: mockReplaceCopilotChatMessages,
}))
vi.mock('drizzle-orm', () => ({
and: vi.fn((...conditions: unknown[]) => ({ conditions, type: 'and' })),
eq: vi.fn((field: unknown, value: unknown) => ({ field, value, type: 'eq' })),
@@ -257,10 +266,13 @@ describe('Copilot Chat Update Messages API Route', () => {
expect(mockSelect).toHaveBeenCalled()
expect(mockUpdate).toHaveBeenCalled()
expect(mockSet).toHaveBeenCalledWith({
expect(mockSet).toHaveBeenCalledWith({ updatedAt: expect.any(Date) })
expect(mockReplaceCopilotChatMessages).toHaveBeenCalledWith(
'chat-123',
messages,
updatedAt: expect.any(Date),
})
{ chatModel: 'gpt-4' },
expect.anything()
)
})
it('should successfully update chat messages with optional fields', async () => {
@@ -315,8 +327,10 @@ describe('Copilot Chat Update Messages API Route', () => {
messageCount: 2,
})
expect(mockSet).toHaveBeenCalledWith({
messages: [
expect(mockSet).toHaveBeenCalledWith({ updatedAt: expect.any(Date) })
expect(mockReplaceCopilotChatMessages).toHaveBeenCalledWith(
'chat-456',
[
{
id: 'msg-1',
role: 'user',
@@ -345,8 +359,9 @@ describe('Copilot Chat Update Messages API Route', () => {
],
},
],
updatedAt: expect.any(Date),
})
{ chatModel: 'gpt-4' },
expect.anything()
)
})
it('should handle empty messages array', async () => {
@@ -373,10 +388,13 @@ describe('Copilot Chat Update Messages API Route', () => {
messageCount: 0,
})
expect(mockSet).toHaveBeenCalledWith({
messages: [],
updatedAt: expect.any(Date),
})
expect(mockSet).toHaveBeenCalledWith({ updatedAt: expect.any(Date) })
expect(mockReplaceCopilotChatMessages).toHaveBeenCalledWith(
'chat-789',
[],
{ chatModel: 'gpt-4' },
expect.anything()
)
})
it('should handle database errors during chat lookup', async () => {
@@ -485,10 +503,13 @@ describe('Copilot Chat Update Messages API Route', () => {
messageCount: 100,
})
expect(mockSet).toHaveBeenCalledWith({
expect(mockSet).toHaveBeenCalledWith({ updatedAt: expect.any(Date) })
expect(mockReplaceCopilotChatMessages).toHaveBeenCalledWith(
'chat-large',
messages,
updatedAt: expect.any(Date),
})
{ chatModel: 'gpt-4' },
expect.anything()
)
})
it('should handle messages with both user and assistant roles', async () => {
apps/sim/app/api/copilot/chat/update-messages/route.ts
+15 -13
View File
@@ -6,7 +6,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { updateCopilotMessagesContract } from '@/lib/api/contracts/copilot'
import { parseRequest } from '@/lib/api/server'
import { getAccessibleCopilotChatAuth } from '@/lib/copilot/chat/lifecycle'
import { replaceCopilotChatMessages } from '@/lib/copilot/chat/messages-dual-write'
import { replaceCopilotChatMessages } from '@/lib/copilot/chat/messages-store'
import { normalizeMessage, type PersistedMessage } from '@/lib/copilot/chat/persisted-message'
import {
authenticateCopilotRequestSessionOnly,
@@ -73,9 +73,7 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
return createNotFoundResponse('Chat not found or unauthorized')
}
// Update chat with new messages, plan artifact, and config
const updateData: Record<string, unknown> = {
messages: normalizedMessages,
updatedAt: new Date(),
}
@@ -87,16 +85,20 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
updateData.config = config
}
const [updated] = await db
.update(copilotChats)
.set(updateData)
.where(eq(copilotChats.id, chatId))
.returning({ model: copilotChats.model })
if (updated) {
await replaceCopilotChatMessages(chatId, normalizedMessages, {
chatModel: updated.model ?? null,
})
}
await db.transaction(async (tx) => {
const [updated] = await tx
.update(copilotChats)
.set(updateData)
.where(eq(copilotChats.id, chatId))
.returning({ model: copilotChats.model })
if (!updated) return
await replaceCopilotChatMessages(
chatId,
normalizedMessages,
{ chatModel: updated.model ?? null },
tx
)
})
logger.info(`[${tracker.requestId}] Successfully updated chat`, {
chatId,
apps/sim/app/api/files/parse/route.test.ts
+62
View File
@@ -796,6 +796,68 @@ describe('Files Parse API - Path Traversal Security', () => {
}
})
it('should not treat .. inside external URLs as path traversal', async () => {
inputValidationMockFns.mockValidateUrlWithDNS.mockResolvedValue({
isValid: true,
resolvedIP: '203.0.113.10',
})
inputValidationMockFns.mockSecureFetchWithPinnedIP.mockResolvedValue(
new Response('slack file content', {
status: 200,
headers: { 'content-type': 'text/plain' },
})
)
permissionsMockFns.mockGetUserEntityPermissions.mockResolvedValue('write')
// Slack truncates long titles with a literal ellipsis, so the slug contains `..`
const slackUrl =
'https://files.slack.com/files-pri/T08-F0B/_other__no_invitation_messages_get_sent_-_sim_on_railway...txt'
const request = new NextRequest('http://localhost:3000/api/files/parse', {
method: 'POST',
body: JSON.stringify({ filePath: slackUrl, workspaceId: 'workspace-id' }),
})
const response = await POST(request)
const result = await response.json()
expect(result.success).toBe(true)
// The URL reaching the pinned fetch proves it passed validation and routed
// to external-URL handling rather than being rejected as a local path.
expect(inputValidationMockFns.mockSecureFetchWithPinnedIP).toHaveBeenCalledWith(
slackUrl,
'203.0.113.10',
expect.any(Object)
)
})
it('should still reject traversal in https URLs that look like internal serve URLs', async () => {
inputValidationMockFns.mockValidateUrlWithDNS.mockResolvedValue({
isValid: true,
resolvedIP: '203.0.113.10',
})
inputValidationMockFns.mockSecureFetchWithPinnedIP.mockResolvedValue(
new Response('should never be fetched', { status: 200 })
)
// Absolute https URL containing `/api/files/serve/` matches isInternalFileUrl and would
// route to handleCloudFile — so it must keep traversal protection, not be waved through
// as an external URL.
const request = new NextRequest('http://localhost:3000/api/files/parse', {
method: 'POST',
body: JSON.stringify({
filePath: 'https://attacker.com/api/files/serve/../../../etc/passwd',
}),
})
const response = await POST(request)
const result = await response.json()
expect(result.success).toBe(false)
expect(result.error).toMatch(/Access denied: path traversal detected/)
expect(inputValidationMockFns.mockSecureFetchWithPinnedIP).not.toHaveBeenCalled()
})
it('should handle encoded path traversal attempts', async () => {
const encodedMaliciousPaths = [
'/api/files/serve/%2e%2e%2f%2e%2e%2fetc%2fpasswd', // ../../../etc/passwd
apps/sim/app/api/files/parse/route.ts
+21 -1
View File
@@ -419,13 +419,33 @@ function assertParsedContentWithinLimit(content: string, maxBytes?: number): str
}
/**
* Validate file path for security - prevents null byte injection and path traversal attacks
* Validate file path for security - prevents null byte injection and path traversal attacks.
*
* External URLs (`http`/`https`) are fetched over HTTP — with SSRF protection applied
* downstream in `fetchExternalUrlToWorkspace` (DNS resolution + private/reserved IP blocking)
* — and are never resolved against the filesystem, so `..`/`~` are legal URL content and must
* not be rejected. Providers such as Slack routinely emit slugs containing a literal `...`.
*
* Internal file URLs (`/api/files/serve/...`) ARE resolved to storage keys and filesystem
* paths via `extractStorageKey`, so they keep full traversal protection. The external
* short-circuit explicitly excludes them: `parseFileSingle` routes anything matching
* `isInternalFileUrl` to `handleCloudFile` (even an absolute `https://host/api/files/serve/...`),
* so such inputs must stay subject to the `..`/`~` checks rather than being waved through as
* external URLs. Only the leading-`/` "outside allowed directory" check is relaxed for them,
* since that prefix is expected.
*/
function validateFilePath(filePath: string): { isValid: boolean; error?: string } {
if (filePath.includes('\0')) {
return { isValid: false, error: 'Invalid path: null byte detected' }
}
if (
(filePath.startsWith('http://') || filePath.startsWith('https://')) &&
!isInternalFileUrl(filePath)
) {
return { isValid: true }
}
if (filePath.includes('..')) {
return { isValid: false, error: 'Access denied: path traversal detected' }
}
apps/sim/app/api/knowledge/utils.test.ts
+6 -1
View File
@@ -259,7 +259,12 @@ describe('Knowledge Utils', () => {
{}
)
expect(dbOps.order).toEqual(['insert', 'updateDoc'])
// Embeddings are inserted first, then the document counter update. A
// usage_log billing insert (recordUsage) may trail after updateDoc and is
// irrelevant to this ordering invariant, so assert position rather than
// exact array equality.
expect(dbOps.order[0]).toBe('insert')
expect(dbOps.order.indexOf('updateDoc')).toBeGreaterThan(0)
expect(dbOps.updatePayloads[0]).toMatchObject({
processingStatus: 'completed',
apps/sim/app/api/logs/execution/[executionId]/route.ts
+12 -3
View File
@@ -13,6 +13,7 @@ import { executionIdParamsSchema } from '@/lib/api/contracts/logs'
import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { materializeExecutionData } from '@/lib/logs/execution/trace-store'
import type { TraceSpan, WorkflowExecutionLog } from '@/lib/logs/types'
const logger = createLogger('LogsByExecutionIdAPI')
@@ -39,13 +40,14 @@ export const GET = withRouteHandler(
.select({
id: workflowExecutionLogs.id,
workflowId: workflowExecutionLogs.workflowId,
workspaceId: workflowExecutionLogs.workspaceId,
executionId: workflowExecutionLogs.executionId,
stateSnapshotId: workflowExecutionLogs.stateSnapshotId,
trigger: workflowExecutionLogs.trigger,
startedAt: workflowExecutionLogs.startedAt,
endedAt: workflowExecutionLogs.endedAt,
totalDurationMs: workflowExecutionLogs.totalDurationMs,
cost: workflowExecutionLogs.cost,
costTotal: workflowExecutionLogs.costTotal,
executionData: workflowExecutionLogs.executionData,
})
.from(workflowExecutionLogs)
@@ -119,7 +121,14 @@ export const GET = withRouteHandler(
return NextResponse.json({ error: 'Workflow state snapshot not found' }, { status: 404 })
}
const executionData = workflowLog.executionData as WorkflowExecutionLog['executionData']
const executionData = (await materializeExecutionData(
workflowLog.executionData as Record<string, unknown> | null,
{
workspaceId: workflowLog.workspaceId,
workflowId: workflowLog.workflowId,
executionId: workflowLog.executionId,
}
)) as WorkflowExecutionLog['executionData']
const traceSpans = (executionData?.traceSpans as TraceSpan[]) || []
const childSnapshotIds = new Set<string>()
const collectSnapshotIds = (spans: TraceSpan[]) => {
@@ -163,7 +172,7 @@ export const GET = withRouteHandler(
startedAt: workflowLog.startedAt.toISOString(),
endedAt: workflowLog.endedAt?.toISOString(),
totalDurationMs: workflowLog.totalDurationMs,
cost: workflowLog.cost || null,
cost: workflowLog.costTotal != null ? { total: Number(workflowLog.costTotal) } : null,
},
}
apps/sim/app/api/logs/export/route.ts
+33 -8
View File
@@ -4,7 +4,9 @@ import { createLogger } from '@sim/logger'
import { and, desc, eq, sql } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server'
import { getSession } from '@/lib/auth'
import { MATERIALIZE_CONCURRENCY, mapWithConcurrency } from '@/lib/core/utils/concurrency'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { materializeExecutionData } from '@/lib/logs/execution/trace-store'
import { buildFilterConditions, LogFilterParamsSchema } from '@/lib/logs/filters'
import { expandFolderIdsWithDescendants } from '@/lib/logs/folder-expansion'
@@ -41,7 +43,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
startedAt: workflowExecutionLogs.startedAt,
endedAt: workflowExecutionLogs.endedAt,
totalDurationMs: workflowExecutionLogs.totalDurationMs,
cost: workflowExecutionLogs.cost,
costTotal: workflowExecutionLogs.costTotal,
executionData: workflowExecutionLogs.executionData,
workflowName: sql<string>`COALESCE(${workflow.name}, 'Deleted Workflow')`,
}
@@ -96,11 +98,29 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
if (!rows.length) break
for (const r of rows as any[]) {
// Heavy execution data may live in object storage; materialize per
// row with bounded concurrency so a 1000-row page doesn't fan out
// into 1000 simultaneous reads.
const materialized = await mapWithConcurrency(
rows as any[],
MATERIALIZE_CONCURRENCY,
(r) =>
materializeExecutionData(r.executionData as Record<string, unknown> | null, {
workspaceId: params.workspaceId,
workflowId: r.workflowId,
executionId: r.executionId,
})
)
for (let j = 0; j < rows.length; j++) {
const r = rows[j] as any
const ed = materialized[j] as Record<string, any>
// A single malformed/unserializable row must not abort the whole CSV
// stream — derive the message/trace columns defensively and fall back
// to empty on error so the row's metadata still exports.
let message = ''
let traces: any = null
let tracesJson = ''
try {
const ed = (r as any).executionData
if (ed) {
if (ed.finalOutput)
message =
@@ -108,20 +128,25 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
? ed.finalOutput
: JSON.stringify(ed.finalOutput)
if (ed.message) message = ed.message
if (ed.traceSpans) traces = ed.traceSpans
if (ed.traceSpans) tracesJson = JSON.stringify(ed.traceSpans)
}
} catch {}
} catch (rowError) {
logger.warn('Skipping unserializable execution data for export row', {
executionId: r.executionId,
error: rowError instanceof Error ? rowError.message : String(rowError),
})
}
const line = [
escapeCsv(r.startedAt?.toISOString?.() || r.startedAt),
escapeCsv(r.level),
escapeCsv(r.workflowName),
escapeCsv(r.trigger),
escapeCsv(r.totalDurationMs ?? ''),
escapeCsv(r.cost?.total ?? r.cost?.value?.total ?? ''),
escapeCsv(r.costTotal ?? ''),
escapeCsv(r.workflowId ?? ''),
escapeCsv(r.executionId ?? ''),
escapeCsv(message),
escapeCsv(traces ? JSON.stringify(traces) : ''),
escapeCsv(tracesJson),
].join(',')
controller.enqueue(encoder.encode(`${line}\n`))
}
apps/sim/app/api/logs/route.ts
+8 -4
View File
@@ -31,6 +31,7 @@ import { listLogsContract, type WorkflowLogSummary } from '@/lib/api/contracts/l
import { parseRequest } from '@/lib/api/server'
import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { jobCostTotal } from '@/lib/logs/fetch-log-detail'
import { buildFilterConditions } from '@/lib/logs/filters'
import { expandFolderIdsWithDescendants } from '@/lib/logs/folder-expansion'
@@ -81,7 +82,8 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
case 'duration':
return sql`${workflowExecutionLogs.totalDurationMs}`
case 'cost':
return sql`(${workflowExecutionLogs.cost}->>'total')::numeric`
// Indexed projection of the usage_log ledger (dollars); no live aggregation.
return sql`${workflowExecutionLogs.costTotal}`
case 'status':
return sql`${workflowExecutionLogs.status}`
default:
@@ -201,7 +203,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
startedAt: workflowExecutionLogs.startedAt,
endedAt: workflowExecutionLogs.endedAt,
totalDurationMs: workflowExecutionLogs.totalDurationMs,
cost: workflowExecutionLogs.cost,
costTotal: workflowExecutionLogs.costTotal,
createdAt: workflowExecutionLogs.createdAt,
workflowName: workflow.name,
workflowDescription: workflow.description,
@@ -379,7 +381,9 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
}
: null,
jobTitle: null,
cost: (log.cost as WorkflowLogSummary['cost']) ?? null,
// List cost is the cost_total projection (faithful ledger sum). Null until
// completion (running) or until the one-time legacy backfill populates it.
cost: log.costTotal != null ? { total: Number(log.costTotal) } : null,
pauseSummary: {
status: log.pausedStatus ?? null,
total: totalPauseCount,
@@ -405,7 +409,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
createdAt: log.startedAt.toISOString(),
workflow: null,
jobTitle: log.jobTitle ?? null,
cost: (log.cost as WorkflowLogSummary['cost']) ?? null,
cost: jobCostTotal(log.cost),
pauseSummary: { status: null, total: 0, resumed: 0 },
hasPendingPause: false,
}
apps/sim/app/api/mcp/copilot/.well-known/oauth-authorization-server/route.ts
+2 -10
View File
@@ -1,12 +1,4 @@
import type { NextRequest, NextResponse } from 'next/server'
import { mcpOauthAuthorizationServerMetadataContract } from '@/lib/api/contracts/mcp-oauth'
import { parseRequest } from '@/lib/api/server'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { createMcpAuthorizationServerMetadataResponse } from '@/lib/mcp/oauth-discovery'
import { copilotMcpDeprecatedResponse } from '@/lib/mcp/copilot-deprecated'
export const GET = withRouteHandler(async (request: NextRequest): Promise<NextResponse> => {
const parsed = await parseRequest(mcpOauthAuthorizationServerMetadataContract, request, {})
if (!parsed.success) return parsed.response as NextResponse
return createMcpAuthorizationServerMetadataResponse()
})
export const GET = withRouteHandler(async () => copilotMcpDeprecatedResponse())
apps/sim/app/api/mcp/copilot/.well-known/oauth-protected-resource/route.ts
+2 -10
View File
@@ -1,12 +1,4 @@
import type { NextRequest, NextResponse } from 'next/server'
import { mcpOauthProtectedResourceMetadataContract } from '@/lib/api/contracts/mcp-oauth'
import { parseRequest } from '@/lib/api/server'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { createMcpProtectedResourceMetadataResponse } from '@/lib/mcp/oauth-discovery'
import { copilotMcpDeprecatedResponse } from '@/lib/mcp/copilot-deprecated'
export const GET = withRouteHandler(async (request: NextRequest): Promise<NextResponse> => {
const parsed = await parseRequest(mcpOauthProtectedResourceMetadataContract, request, {})
if (!parsed.success) return parsed.response as NextResponse
return createMcpProtectedResourceMetadataResponse()
})
export const GET = withRouteHandler(async () => copilotMcpDeprecatedResponse())
apps/sim/app/api/mcp/copilot/route.test.ts
+62
View File
@@ -0,0 +1,62 @@
/**
* Tests for the deprecated Copilot MCP route
*
* @vitest-environment node
*/
import { NextRequest } from 'next/server'
import { describe, expect, it } from 'vitest'
import { GET as authServerDiscoveryGET } from '@/app/api/mcp/copilot/.well-known/oauth-authorization-server/route'
import { GET as protectedResourceDiscoveryGET } from '@/app/api/mcp/copilot/.well-known/oauth-protected-resource/route'
import { DELETE, GET, POST } from '@/app/api/mcp/copilot/route'
const URL = 'http://localhost:3000/api/mcp/copilot'
describe('Deprecated Copilot MCP route', () => {
it('GET returns 410', async () => {
const response = await GET(new NextRequest(URL))
expect(response.status).toBe(410)
})
it('POST returns 410 with a JSON-RPC error envelope', async () => {
const request = new NextRequest(URL, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'initialize' }),
})
const response = await POST(request)
expect(response.status).toBe(410)
const body = (await response.json()) as { jsonrpc?: string; error?: { message?: string } }
expect(body.jsonrpc).toBe('2.0')
expect(body.error?.message).toContain('deprecated')
})
it('POST still returns 410 when an x-api-key header is present', async () => {
const request = new NextRequest(URL, {
method: 'POST',
headers: { 'Content-Type': 'application/json', 'x-api-key': 'sk-sim-copilot-test' },
body: JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'tools/list' }),
})
const response = await POST(request)
expect(response.status).toBe(410)
})
it('DELETE returns 410', async () => {
const response = await DELETE(new NextRequest(URL, { method: 'DELETE' }))
expect(response.status).toBe(410)
})
it('copilot OAuth authorization-server discovery returns 410', async () => {
const response = await authServerDiscoveryGET(
new NextRequest(`${URL}/.well-known/oauth-authorization-server`)
)
expect(response.status).toBe(410)
})
it('copilot OAuth protected-resource discovery returns 410', async () => {
const response = await protectedResourceDiscoveryGET(
new NextRequest(`${URL}/.well-known/oauth-protected-resource`)
)
expect(response.status).toBe(410)
})
})
apps/sim/app/api/mcp/copilot/route.ts
+7 -734
View File
@@ -1,740 +1,13 @@
import { Server } from '@modelcontextprotocol/sdk/server/index.js'
import { WebStandardStreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/webStandardStreamableHttp.js'
import {
CallToolRequestSchema,
type CallToolResult,
ErrorCode,
type JSONRPCError,
ListToolsRequestSchema,
type ListToolsResult,
McpError,
type RequestId,
} from '@modelcontextprotocol/sdk/types.js'
import { createLogger } from '@sim/logger'
import { toError } from '@sim/utils/errors'
import { generateId } from '@sim/utils/id'
import { authorizeWorkflowByWorkspacePermission } from '@sim/workflow-authz'
import { type NextRequest, NextResponse } from 'next/server'
import { mcpRequestBodySchema, mcpToolCallParamsSchema } from '@/lib/api/contracts/mcp'
import { validateOAuthAccessToken } from '@/lib/auth/oauth-token'
import { getHighestPrioritySubscription } from '@/lib/billing/core/subscription'
import { generateWorkspaceContext } from '@/lib/copilot/chat/workspace-context'
import { ORCHESTRATION_TIMEOUT_MS, SIM_AGENT_API_URL } from '@/lib/copilot/constants'
import { createRequestId } from '@/lib/copilot/request/http'
import { runHeadlessCopilotLifecycle } from '@/lib/copilot/request/lifecycle/headless'
import { orchestrateSubagentStream } from '@/lib/copilot/request/subagent'
import { ensureHandlersRegistered, executeTool } from '@/lib/copilot/tool-executor'
import { ensureWorkspaceAccess } from '@/lib/copilot/tools/handlers/access'
import { prepareExecutionContext } from '@/lib/copilot/tools/handlers/context'
import { DIRECT_TOOL_DEFS, SUBAGENT_TOOL_DEFS } from '@/lib/copilot/tools/mcp/definitions'
import { env } from '@/lib/core/config/env'
import { RateLimiter } from '@/lib/core/rate-limiter'
import { getBaseUrl } from '@/lib/core/utils/urls'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { resolveWorkflowIdForUser } from '@/lib/workflows/utils'
const logger = createLogger('CopilotMcpAPI')
const mcpRateLimiter = new RateLimiter()
const DEFAULT_COPILOT_MODEL = 'claude-opus-4-6'
import {
copilotMcpDeprecatedJsonRpcResponse,
copilotMcpDeprecatedResponse,
} from '@/lib/mcp/copilot-deprecated'
export const dynamic = 'force-dynamic'
export const runtime = 'nodejs'
export const maxDuration = 3600
interface CopilotKeyAuthResult {
success: boolean
userId?: string
error?: string
}
export const GET = withRouteHandler(async () => copilotMcpDeprecatedResponse())
/**
* Validates a copilot API key by forwarding it to the Go copilot service's
* `/api/validate-key` endpoint. Returns the associated userId on success.
*/
async function authenticateCopilotApiKey(apiKey: string): Promise<CopilotKeyAuthResult> {
try {
const internalSecret = env.INTERNAL_API_SECRET
if (!internalSecret) {
logger.error('INTERNAL_API_SECRET not configured')
return { success: false, error: 'Server configuration error' }
}
export const POST = withRouteHandler(async () => copilotMcpDeprecatedJsonRpcResponse())
const { fetchGo } = await import('@/lib/copilot/request/go/fetch')
const res = await fetchGo(`${SIM_AGENT_API_URL}/api/validate-key`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-api-key': internalSecret,
},
body: JSON.stringify({ targetApiKey: apiKey }),
signal: AbortSignal.timeout(10_000),
spanName: 'sim → go /api/validate-key (mcp)',
operation: 'mcp_validate_key',
})
if (!res.ok) {
const body = await res.json().catch(() => null)
const upstream = (body as Record<string, unknown>)?.message
const status = res.status
if (status === 401 || status === 403) {
return {
success: false,
error: `Invalid Copilot API key. Generate a new key in Settings → Copilot and set it in the x-api-key header.`,
}
}
if (status === 402) {
return {
success: false,
error: `Usage limit exceeded for this Copilot API key. Upgrade your plan or wait for your quota to reset.`,
}
}
return {
success: false,
error: String(upstream ?? 'Copilot API key validation failed'),
}
}
const data = (await res.json()) as { ok?: boolean; userId?: string }
if (!data.ok || !data.userId) {
return {
success: false,
error: 'Invalid Copilot API key. Generate a new key in Settings → Copilot.',
}
}
return { success: true, userId: data.userId }
} catch (error) {
logger.error('Copilot API key validation failed', { error })
return {
success: false,
error:
'Could not validate Copilot API key — the authentication service is temporarily unreachable. This is NOT a problem with the API key itself; please retry shortly.',
}
}
}
/**
* MCP Server instructions that guide LLMs on how to use the Sim copilot tools.
* This is included in the initialize response to help external LLMs understand
* the workflow lifecycle and best practices.
*/
const MCP_SERVER_INSTRUCTIONS = `
## Sim Workflow Copilot
Sim is a workflow automation platform. Workflows are visual pipelines of connected blocks (Agent, Function, Condition, API, integrations, etc.). The Agent block is the core — an LLM with tools, memory, structured output, and knowledge bases.
### Workflow Lifecycle (Happy Path)
1. \`list_workspaces\` → know where to work
2. \`create_workflow(name, workspaceId)\` → get a workflowId
3. \`sim_workflow(request, workflowId)\` → plan and build in one pass
4. \`sim_test(request, workflowId)\` → verify it works
5. \`sim_deploy("deploy as api", workflowId)\` → make it accessible externally (optional)
### Working with Existing Workflows
When the user refers to a workflow by name or description ("the email one", "my Slack bot"):
1. Use \`sim_discovery\` to find it by functionality
2. Or use \`list_workflows\` and match by name
3. Then pass the workflowId to other tools
### Organization
- \`rename_workflow\` — rename a workflow
- \`move_workflow\` — move a workflow into a folder (or back to root by clearing the folder id)
- \`move_folder\` — nest a folder inside another (or move it back to root by clearing the parent id)
- \`create_folder(name, parentId)\` — create nested folder hierarchies
### Key Rules
- You can test workflows immediately after building — deployment is only needed for external access (API, chat, MCP).
- Tools that operate on a specific workflow such as \`sim_workflow\`, \`sim_test\`, \`sim_deploy\`, and workflow-scoped \`sim_info\` requests require \`workflowId\`.
- If the user reports errors, route through \`sim_workflow\` and ask it to reproduce, inspect logs, and fix the issue end to end.
- Variable syntax: \`<blockname.field>\` for block outputs, \`{{ENV_VAR}}\` for env vars.
`
type HeaderMap = Record<string, string | string[] | undefined>
function createError(id: RequestId, code: ErrorCode | number, message: string): JSONRPCError {
return {
jsonrpc: '2.0',
id,
error: { code, message },
}
}
function readHeader(headers: HeaderMap | undefined, name: string): string | undefined {
if (!headers) return undefined
const value = headers[name.toLowerCase()]
if (Array.isArray(value)) {
return value[0]
}
return value
}
function buildMcpServer(abortSignal?: AbortSignal): Server {
const server = new Server(
{
name: 'sim-copilot',
version: '1.0.0',
},
{
capabilities: { tools: {} },
instructions: MCP_SERVER_INSTRUCTIONS,
}
)
server.setRequestHandler(ListToolsRequestSchema, async () => {
const directTools = DIRECT_TOOL_DEFS.map((tool) => ({
name: tool.name,
description: tool.description,
inputSchema: tool.inputSchema,
...(tool.annotations && { annotations: tool.annotations }),
}))
const subagentTools = SUBAGENT_TOOL_DEFS.map((tool) => ({
name: tool.name,
description: tool.description,
inputSchema: tool.inputSchema,
...(tool.annotations && { annotations: tool.annotations }),
}))
const result: ListToolsResult = {
tools: [...directTools, ...subagentTools],
}
return result
})
server.setRequestHandler(CallToolRequestSchema, async (request, extra) => {
const headers = (extra.requestInfo?.headers || {}) as HeaderMap
const apiKeyHeader = readHeader(headers, 'x-api-key')
const authorizationHeader = readHeader(headers, 'authorization')
let authResult: CopilotKeyAuthResult = { success: false }
if (authorizationHeader?.startsWith('Bearer ')) {
const token = authorizationHeader.slice(7)
const oauthResult = await validateOAuthAccessToken(token)
if (oauthResult.success && oauthResult.userId) {
if (!oauthResult.scopes?.includes('mcp:tools')) {
return {
content: [
{
type: 'text' as const,
text: 'AUTHENTICATION ERROR: OAuth token is missing the required "mcp:tools" scope. Re-authorize with the correct scopes.',
},
],
isError: true,
}
}
authResult = { success: true, userId: oauthResult.userId }
} else {
return {
content: [
{
type: 'text' as const,
text: `AUTHENTICATION ERROR: ${oauthResult.error ?? 'Invalid OAuth access token'} Do NOT retry — re-authorize via OAuth.`,
},
],
isError: true,
}
}
} else if (apiKeyHeader) {
authResult = await authenticateCopilotApiKey(apiKeyHeader)
}
if (!authResult.success || !authResult.userId) {
const errorMsg = apiKeyHeader
? `AUTHENTICATION ERROR: ${authResult.error} Do NOT retry — this will fail until the user fixes their Copilot API key.`
: 'AUTHENTICATION ERROR: No authentication provided. Provide a Bearer token (OAuth 2.1) or an x-api-key header. Generate a Copilot API key in Settings → Copilot.'
logger.warn('MCP copilot auth failed', { method: request.method })
return {
content: [
{
type: 'text' as const,
text: errorMsg,
},
],
isError: true,
}
}
const rateLimitResult = await mcpRateLimiter.checkRateLimitWithSubscription(
authResult.userId,
await getHighestPrioritySubscription(authResult.userId),
'api-endpoint',
false
)
if (!rateLimitResult.allowed) {
return {
content: [
{
type: 'text' as const,
text: `RATE LIMIT: Too many requests. Please wait and retry after ${rateLimitResult.resetAt.toISOString()}.`,
},
],
isError: true,
}
}
const paramsValidation = mcpToolCallParamsSchema.safeParse(request.params)
if (!paramsValidation.success) {
throw new McpError(ErrorCode.InvalidParams, 'Tool name required')
}
const params = paramsValidation.data
const result = await handleToolsCall(
{
name: params.name,
arguments: params.arguments,
},
authResult.userId,
abortSignal
)
return result
})
return server
}
async function handleMcpRequestWithSdk(
request: NextRequest,
parsedBody: unknown
): Promise<Response> {
const server = buildMcpServer(request.signal)
const transport = new WebStandardStreamableHTTPServerTransport({
sessionIdGenerator: undefined,
enableJsonResponse: true,
})
await server.connect(transport)
try {
return await transport.handleRequest(request, { parsedBody })
} finally {
await server.close().catch(() => {})
await transport.close().catch(() => {})
}
}
export const GET = withRouteHandler(async () => {
// Return 405 to signal that server-initiated SSE notifications are not
// supported. Without this, clients like mcp-remote will repeatedly
// reconnect trying to open an SSE stream, flooding the logs with GETs.
return new NextResponse(null, { status: 405 })
})
export const POST = withRouteHandler(async (request: NextRequest) => {
const hasAuth = request.headers.has('authorization') || request.headers.has('x-api-key')
if (!hasAuth) {
const origin = getBaseUrl().replace(/\/$/, '')
const resourceMetadataUrl = `${origin}/.well-known/oauth-protected-resource/api/mcp/copilot`
return new NextResponse(JSON.stringify({ error: 'unauthorized' }), {
status: 401,
headers: {
'WWW-Authenticate': `Bearer resource_metadata="${resourceMetadataUrl}", scope="mcp:tools"`,
'Content-Type': 'application/json',
},
})
}
try {
let parsedBody: unknown
try {
parsedBody = await request.json()
} catch {
return NextResponse.json(createError(0, ErrorCode.ParseError, 'Invalid JSON body'), {
status: 400,
})
}
const bodyValidation = mcpRequestBodySchema.safeParse(parsedBody)
if (!bodyValidation.success) {
return NextResponse.json(
createError(0, ErrorCode.InvalidRequest, 'Invalid JSON-RPC message'),
{
status: 400,
}
)
}
return await handleMcpRequestWithSdk(request, bodyValidation.data)
} catch (error) {
if (request.signal.aborted || (error as Error)?.name === 'AbortError') {
return NextResponse.json(
createError(0, ErrorCode.ConnectionClosed, 'Client cancelled request'),
{ status: 499 }
)
}
logger.error('Error handling MCP request', { error })
return NextResponse.json(createError(0, ErrorCode.InternalError, 'Internal error'), {
status: 500,
})
}
})
export const DELETE = withRouteHandler(async (request: NextRequest) => {
void request
return NextResponse.json(createError(0, -32000, 'Method not allowed.'), { status: 405 })
})
async function handleToolsCall(
params: { name: string; arguments?: Record<string, unknown> },
userId: string,
abortSignal?: AbortSignal
): Promise<CallToolResult> {
const args = params.arguments || {}
const directTool = DIRECT_TOOL_DEFS.find((tool) => tool.name === params.name)
if (directTool) {
return handleDirectToolCall(directTool, args, userId)
}
const subagentTool = SUBAGENT_TOOL_DEFS.find((tool) => tool.name === params.name)
if (subagentTool) {
return handleSubagentToolCall(subagentTool, args, userId, abortSignal)
}
throw new McpError(ErrorCode.MethodNotFound, `Tool not found: ${params.name}`)
}
async function handleDirectToolCall(
toolDef: (typeof DIRECT_TOOL_DEFS)[number],
args: Record<string, unknown>,
userId: string
): Promise<CallToolResult> {
try {
const rawWorkflowId = (args.workflowId as string) || ''
let resolvedWorkspaceId: string | undefined
if (rawWorkflowId) {
const authorization = await authorizeWorkflowByWorkspacePermission({
workflowId: rawWorkflowId,
userId,
action: 'read',
})
if (!authorization.allowed) {
return {
content: [
{
type: 'text',
text: JSON.stringify(
{ success: false, error: 'Workflow not found or access denied' },
null,
2
),
},
],
isError: true,
}
}
resolvedWorkspaceId = authorization.workflow?.workspaceId || undefined
}
const execContext = await prepareExecutionContext(
userId,
rawWorkflowId,
(args.chatId as string) || undefined,
{ workspaceId: resolvedWorkspaceId }
)
const toolCall = {
id: generateId(),
name: toolDef.toolId,
status: 'pending' as const,
params: args as Record<string, any>,
startTime: Date.now(),
}
ensureHandlersRegistered()
const result = await executeTool(toolCall.name, toolCall.params || {}, execContext)
return {
content: [
{
type: 'text',
text: JSON.stringify(result.output ?? result, null, 2),
},
],
isError: !result.success,
}
} catch (error) {
logger.error('Direct tool execution failed', { tool: toolDef.name, error })
return {
content: [
{
type: 'text',
text: `Tool execution failed: ${toError(error).message}`,
},
],
isError: true,
}
}
}
/**
* Build mode uses the main /api/mcp orchestrator instead of /api/subagent/workflow.
* The main agent still delegates workflow work to the workflow subagent inside Go;
* this helper simply uses the full headless lifecycle so build requests behave like
* the primary MCP chat flow.
*/
async function handleBuildToolCall(
args: Record<string, unknown>,
userId: string,
abortSignal?: AbortSignal
): Promise<CallToolResult> {
try {
const requestText = (args.request as string) || JSON.stringify(args)
const workflowId = args.workflowId as string | undefined
let resolvedWorkflowName: string | undefined
let resolvedWorkspaceId: string | undefined
const resolved = workflowId
? await (async () => {
const authorization = await authorizeWorkflowByWorkspacePermission({
workflowId,
userId,
action: 'read',
})
resolvedWorkflowName = authorization.workflow?.name || undefined
resolvedWorkspaceId = authorization.workflow?.workspaceId || undefined
return authorization.allowed
? {
status: 'resolved' as const,
workflowId,
workflowName: resolvedWorkflowName,
}
: {
status: 'not_found' as const,
message: 'workflowId is required for build. Call create_workflow first.',
}
})()
: await resolveWorkflowIdForUser(userId)
if (resolved.status === 'resolved') {
resolvedWorkflowName ||= resolved.workflowName
}
if (!resolved || resolved.status !== 'resolved') {
return {
content: [
{
type: 'text',
text: JSON.stringify(
{
success: false,
error:
resolved?.message ??
'workflowId is required for build. Call create_workflow first.',
},
null,
2
),
},
],
isError: true,
}
}
const chatId = generateId()
const executionContext = await prepareExecutionContext(userId, resolved.workflowId, chatId, {
workspaceId: resolvedWorkspaceId,
})
resolvedWorkspaceId = executionContext.workspaceId
let workspaceContext: string | undefined
if (resolvedWorkspaceId) {
try {
workspaceContext = await generateWorkspaceContext(resolvedWorkspaceId, userId)
} catch (error) {
logger.warn('Failed to generate workspace context for build tool call', {
workflowId: resolved.workflowId,
workspaceId: resolvedWorkspaceId,
error: toError(error).message,
})
}
}
const requestPayload = {
message: requestText,
workflowId: resolved.workflowId,
...(resolvedWorkflowName ? { workflowName: resolvedWorkflowName } : {}),
...(resolvedWorkspaceId ? { workspaceId: resolvedWorkspaceId } : {}),
...(workspaceContext ? { workspaceContext } : {}),
userId,
model: DEFAULT_COPILOT_MODEL,
mode: 'agent',
commands: ['fast'],
messageId: generateId(),
chatId,
}
const result = await runHeadlessCopilotLifecycle(requestPayload, {
userId,
workflowId: resolved.workflowId,
workspaceId: resolvedWorkspaceId,
chatId,
goRoute: '/api/mcp',
executionContext,
autoExecuteTools: true,
timeout: ORCHESTRATION_TIMEOUT_MS,
interactive: false,
abortSignal,
})
const responseData = {
success: result.success,
content: result.content,
toolCalls: result.toolCalls,
error: result.error,
}
return {
content: [{ type: 'text', text: JSON.stringify(responseData, null, 2) }],
isError: !result.success,
}
} catch (error) {
logger.error('Build tool call failed', { error })
return {
content: [
{
type: 'text',
text: `Build failed: ${toError(error).message}`,
},
],
isError: true,
}
}
}
async function handleSubagentToolCall(
toolDef: (typeof SUBAGENT_TOOL_DEFS)[number],
args: Record<string, unknown>,
userId: string,
abortSignal?: AbortSignal
): Promise<CallToolResult> {
if (toolDef.agentId === 'workflow') {
return handleBuildToolCall(args, userId, abortSignal)
}
try {
const requestText =
(args.request as string) ||
(args.message as string) ||
(args.error as string) ||
JSON.stringify(args)
const simRequestId = createRequestId()
const context = (args.context as Record<string, unknown>) || {}
if (args.plan && !context.plan) {
context.plan = args.plan
}
// Authorize user-supplied workflowId / workspaceId before forwarding downstream
const rawWorkflowId = args.workflowId as string | undefined
const rawWorkspaceId = args.workspaceId as string | undefined
let resolvedWorkflowId: string | undefined
let resolvedWorkspaceId: string | undefined
if (rawWorkflowId) {
const authorization = await authorizeWorkflowByWorkspacePermission({
workflowId: rawWorkflowId,
userId,
action: 'read',
})
if (!authorization.allowed) {
return {
content: [
{
type: 'text',
text: JSON.stringify(
{ success: false, error: 'Workflow not found or access denied' },
null,
2
),
},
],
isError: true,
}
}
resolvedWorkflowId = rawWorkflowId
resolvedWorkspaceId = authorization.workflow?.workspaceId || undefined
} else if (rawWorkspaceId) {
await ensureWorkspaceAccess(rawWorkspaceId, userId, 'read')
resolvedWorkspaceId = rawWorkspaceId
}
const result = await orchestrateSubagentStream(
toolDef.agentId,
{
message: requestText,
workflowId: resolvedWorkflowId,
workspaceId: resolvedWorkspaceId,
context,
model: DEFAULT_COPILOT_MODEL,
headless: true,
source: 'mcp',
},
{
userId,
workflowId: resolvedWorkflowId,
workspaceId: resolvedWorkspaceId,
simRequestId,
abortSignal,
}
)
let responseData: unknown
if (result.structuredResult) {
responseData = {
success: result.structuredResult.success ?? result.success,
type: result.structuredResult.type,
summary: result.structuredResult.summary,
data: result.structuredResult.data,
}
} else if (result.error) {
responseData = {
success: false,
error: result.error,
errors: result.errors,
}
} else {
responseData = {
success: result.success,
content: result.content,
}
}
return {
content: [
{
type: 'text',
text: JSON.stringify(responseData, null, 2),
},
],
isError: !result.success,
}
} catch (error) {
logger.error('Subagent tool call failed', {
tool: toolDef.name,
agentId: toolDef.agentId,
error,
})
return {
content: [
{
type: 'text',
text: `Subagent call failed: ${toError(error).message}`,
},
],
isError: true,
}
}
}
export const DELETE = withRouteHandler(async () => copilotMcpDeprecatedJsonRpcResponse())
apps/sim/app/api/mothership/chats/[chatId]/fork/route.ts
+40 -27
View File
@@ -6,8 +6,8 @@ import { eq } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server'
import { forkMothershipChatContract } from '@/lib/api/contracts/mothership-tasks'
import { parseRequest } from '@/lib/api/server'
import { appendCopilotChatMessages } from '@/lib/copilot/chat/messages-dual-write'
import type { PersistedMessage } from '@/lib/copilot/chat/persisted-message'
import { loadCopilotChatMessages } from '@/lib/copilot/chat/lifecycle'
import { appendCopilotChatMessages } from '@/lib/copilot/chat/messages-store'
import { fetchGo } from '@/lib/copilot/request/go/fetch'
import {
authenticateCopilotRequestSessionOnly,
@@ -50,9 +50,19 @@ export const POST = withRouteHandler(
const { chatId } = parsed.data.params
const { upToMessageId } = parsed.data.body
// Load parent chat and verify ownership.
const [parent] = await db
.select()
.select({
id: copilotChats.id,
userId: copilotChats.userId,
type: copilotChats.type,
workspaceId: copilotChats.workspaceId,
title: copilotChats.title,
model: copilotChats.model,
resources: copilotChats.resources,
previewYaml: copilotChats.previewYaml,
planArtifact: copilotChats.planArtifact,
config: copilotChats.config,
})
.from(copilotChats)
.where(eq(copilotChats.id, chatId))
.limit(1)
@@ -65,8 +75,7 @@ export const POST = withRouteHandler(
await assertActiveWorkspaceAccess(parent.workspaceId, userId)
}
// Find the fork point in the Sim-side messages array.
const messages = Array.isArray(parent.messages) ? (parent.messages as PersistedMessage[]) : []
const messages = await loadCopilotChatMessages(chatId)
const forkIdx = messages.findIndex((m) => m.id === upToMessageId)
if (forkIdx < 0) {
return createBadRequestResponse('Message not found in chat')
@@ -83,32 +92,36 @@ export const POST = withRouteHandler(
const title = `Fork | ${baseTitle}`
const now = new Date()
const [newChat] = await db
.insert(copilotChats)
.values({
id: newId,
userId,
workspaceId: parent.workspaceId,
type: parent.type,
title,
model: parent.model,
messages: forkedMessages,
resources: parentResources,
previewYaml: parent.previewYaml,
planArtifact: parent.planArtifact,
config: parent.config,
conversationId: null,
updatedAt: now,
lastSeenAt: now,
})
.returning({ id: copilotChats.id, workspaceId: copilotChats.workspaceId })
const newChat = await db.transaction(async (tx) => {
const [row] = await tx
.insert(copilotChats)
.values({
id: newId,
userId,
workspaceId: parent.workspaceId,
type: parent.type,
title,
model: parent.model,
resources: parentResources,
previewYaml: parent.previewYaml,
planArtifact: parent.planArtifact,
config: parent.config,
conversationId: null,
updatedAt: now,
lastSeenAt: now,
})
.returning({ id: copilotChats.id, workspaceId: copilotChats.workspaceId })
if (!row) return null
await appendCopilotChatMessages(newId, forkedMessages, { chatModel: parent.model }, tx)
return row
})
if (!newChat) {
return createInternalServerErrorResponse('Failed to create forked chat')
}
await appendCopilotChatMessages(newId, forkedMessages, { chatModel: parent.model })
// Clone copilot-service conversation state (messages, active_messages, memory files).
// Best-effort: if the copilot service doesn't have a row for the source chat yet, skip.
try {
apps/sim/app/api/mothership/chats/route.ts
-1
View File
@@ -106,7 +106,6 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
type: 'mothership',
title: null,
model: 'claude-opus-4-6',
messages: [],
updatedAt: now,
lastSeenAt: now,
})
apps/sim/app/api/organizations/[id]/members/[memberId]/route.ts
+16
View File
@@ -11,6 +11,7 @@ import {
import { parseRequest } from '@/lib/api/server'
import { getSession } from '@/lib/auth'
import { setActiveOrganizationForCurrentSession } from '@/lib/auth/active-organization'
import { getOrgMemberLedgerByUser } from '@/lib/billing/core/organization'
import { getUserUsageData } from '@/lib/billing/core/usage'
import {
removeExternalUserFromOrganizationWorkspaces,
@@ -101,10 +102,25 @@ export const GET = withRouteHandler(
const computed = await getUserUsageData(memberId)
if (usageData.length > 0) {
// currentPeriodCost is only a baseline; add this member's attributed
// usage_log for the period. (getUserUsageData returns the org POOL for
// org-scoped members, so it can't supply the per-member figure.)
const memberLedger =
(
await getOrgMemberLedgerByUser(
organizationId,
computed.billingPeriodStart && computed.billingPeriodEnd
? { start: computed.billingPeriodStart, end: computed.billingPeriodEnd }
: null
)
).get(memberId) ?? 0
memberData = {
...memberData,
usage: {
...usageData[0],
currentPeriodCost: (
Number(usageData[0].currentPeriodCost ?? 0) + memberLedger
).toString(),
billingPeriodStart: computed.billingPeriodStart,
billingPeriodEnd: computed.billingPeriodEnd,
},
apps/sim/app/api/organizations/[id]/members/route.ts
+14
View File
@@ -17,6 +17,7 @@ import {
} from '@/lib/api/contracts/organization'
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { getSession } from '@/lib/auth'
import { getOrgMemberLedgerByUser } from '@/lib/billing/core/organization'
import { ENTITLED_SUBSCRIPTION_STATUSES } from '@/lib/billing/subscriptions/utils'
import { validateSeatAvailability } from '@/lib/billing/validation/seat-management'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
@@ -139,8 +140,21 @@ export const GET = withRouteHandler(
const billingPeriodStart = orgSub?.periodStart ?? null
const billingPeriodEnd = orgSub?.periodEnd ?? null
// currentPeriodCost is only a baseline; add each member's attributed
// usage_log for the period (batched, one query) so the roster shows real
// usage rather than the frozen baseline.
const usageByUser = await getOrgMemberLedgerByUser(
organizationId,
billingPeriodStart && billingPeriodEnd
? { start: billingPeriodStart, end: billingPeriodEnd }
: null
)
const membersWithUsage = base.map((row) => ({
...row,
currentPeriodCost: (
Number(row.currentPeriodCost ?? 0) + (usageByUser.get(row.userId) ?? 0)
).toString(),
billingPeriodStart,
billingPeriodEnd,
}))
apps/sim/app/api/providers/baseten/models/route.test.ts
+252
View File
@@ -0,0 +1,252 @@
/**
* @vitest-environment node
*/
import { createMockRequest } from '@sim/testing'
import { beforeEach, describe, expect, it, vi } from 'vitest'
const {
mockFilterBlacklistedModels,
mockIsProviderBlacklisted,
mockGetBYOKKey,
mockGetSession,
mockGetUserEntityPermissions,
mutableEnv,
} = vi.hoisted(() => ({
mockFilterBlacklistedModels: vi.fn(),
mockIsProviderBlacklisted: vi.fn(),
mockGetBYOKKey: vi.fn(),
mockGetSession: vi.fn(),
mockGetUserEntityPermissions: vi.fn(),
mutableEnv: { BASETEN_API_KEY: undefined as string | undefined },
}))
vi.mock('@/lib/core/config/env', () => ({ env: mutableEnv }))
vi.mock('@/providers/utils', () => ({
filterBlacklistedModels: mockFilterBlacklistedModels,
isProviderBlacklisted: mockIsProviderBlacklisted,
}))
vi.mock('@/lib/api-key/byok', () => ({
getBYOKKey: mockGetBYOKKey,
}))
vi.mock('@/lib/auth', () => ({
getSession: mockGetSession,
}))
vi.mock('@/lib/workspaces/permissions/utils', () => ({
getUserEntityPermissions: mockGetUserEntityPermissions,
}))
import { GET } from '@/app/api/providers/baseten/models/route'
const BASETEN_MODELS_URL = 'https://inference.baseten.co/v1/models'
function jsonResponse(body: unknown, init: { ok?: boolean; status?: number } = {}): Response {
const status = init.status ?? 200
const ok = init.ok ?? (status >= 200 && status < 300)
return {
ok,
status,
statusText: ok ? 'OK' : 'Error',
json: vi.fn(async () => body),
} as unknown as Response
}
function setEnvKey(value: string | undefined): void {
mutableEnv.BASETEN_API_KEY = value
}
function authHeaderFromLastFetch(mockFetch: ReturnType<typeof vi.fn>): unknown {
const init = mockFetch.mock.calls.at(-1)?.[1] as RequestInit | undefined
return (init?.headers as Record<string, string> | undefined)?.Authorization
}
describe('GET /api/providers/baseten/models', () => {
let mockFetch: ReturnType<typeof vi.fn>
beforeEach(() => {
vi.clearAllMocks()
mockFetch = vi.fn()
vi.stubGlobal('fetch', mockFetch)
mockIsProviderBlacklisted.mockReturnValue(false)
mockFilterBlacklistedModels.mockImplementation((models: string[]) => models)
mockGetBYOKKey.mockResolvedValue(null)
mockGetSession.mockResolvedValue(null)
mockGetUserEntityPermissions.mockResolvedValue(null)
setEnvKey(undefined)
})
it('returns empty models without fetching when the provider is blacklisted', async () => {
mockIsProviderBlacklisted.mockReturnValue(true)
setEnvKey('env-key')
const res = await GET(createMockRequest('GET'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
expect(mockFetch).not.toHaveBeenCalled()
})
it('returns empty models when no workspaceId and no env key are available', async () => {
const res = await GET(createMockRequest('GET'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
expect(mockFetch).not.toHaveBeenCalled()
})
it('fetches models with the env key and prefixes each id with baseten/', async () => {
setEnvKey('env-key')
mockFetch.mockResolvedValueOnce(
jsonResponse({
data: [{ id: 'openai/gpt-oss-120b' }, { id: 'deepseek-ai/DeepSeek-V3' }],
})
)
const res = await GET(createMockRequest('GET'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({
models: ['baseten/openai/gpt-oss-120b', 'baseten/deepseek-ai/DeepSeek-V3'],
})
expect(mockFetch).toHaveBeenCalledTimes(1)
const [url, init] = mockFetch.mock.calls[0]
expect(url).toBe(BASETEN_MODELS_URL)
expect((init.headers as Record<string, string>).Authorization).toBe('Bearer env-key')
})
it('uses the BYOK key when workspaceId, session, and permission are present', async () => {
setEnvKey('env-key')
mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
mockGetUserEntityPermissions.mockResolvedValue('admin')
mockGetBYOKKey.mockResolvedValue({ apiKey: 'byok-key', isBYOK: true })
mockFetch.mockResolvedValueOnce(jsonResponse({ data: [{ id: 'model-a' }] }))
const res = await GET(
createMockRequest('GET', undefined, {}, 'http://localhost:3000/api/test?workspaceId=ws-1')
)
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: ['baseten/model-a'] })
expect(mockGetBYOKKey).toHaveBeenCalledWith('ws-1', 'baseten')
expect(authHeaderFromLastFetch(mockFetch)).toBe('Bearer byok-key')
})
it('falls back to the env key when there is a workspaceId but no session', async () => {
setEnvKey('env-key')
mockGetSession.mockResolvedValue(null)
mockFetch.mockResolvedValueOnce(jsonResponse({ data: [{ id: 'model-a' }] }))
const res = await GET(
createMockRequest('GET', undefined, {}, 'http://localhost:3000/api/test?workspaceId=ws-1')
)
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: ['baseten/model-a'] })
expect(mockGetBYOKKey).not.toHaveBeenCalled()
expect(authHeaderFromLastFetch(mockFetch)).toBe('Bearer env-key')
})
it('falls back to the env key when the user lacks workspace permission', async () => {
setEnvKey('env-key')
mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
mockGetUserEntityPermissions.mockResolvedValue(null)
mockFetch.mockResolvedValueOnce(jsonResponse({ data: [{ id: 'model-a' }] }))
const res = await GET(
createMockRequest('GET', undefined, {}, 'http://localhost:3000/api/test?workspaceId=ws-1')
)
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: ['baseten/model-a'] })
expect(mockGetBYOKKey).not.toHaveBeenCalled()
expect(authHeaderFromLastFetch(mockFetch)).toBe('Bearer env-key')
})
it('returns empty models when the upstream responds 401', async () => {
setEnvKey('env-key')
mockFetch.mockResolvedValueOnce(jsonResponse({}, { ok: false, status: 401 }))
const res = await GET(createMockRequest('GET'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
})
it('returns empty models when the upstream responds 500', async () => {
setEnvKey('env-key')
mockFetch.mockResolvedValueOnce(jsonResponse({}, { ok: false, status: 500 }))
const res = await GET(createMockRequest('GET'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
})
it('returns empty models when fetch throws', async () => {
setEnvKey('env-key')
mockFetch.mockRejectedValueOnce(new Error('network down'))
const res = await GET(createMockRequest('GET'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
})
it('returns empty models when the upstream data array is empty', async () => {
setEnvKey('env-key')
mockFetch.mockResolvedValueOnce(jsonResponse({ data: [] }))
const res = await GET(createMockRequest('GET'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
})
it('returns empty models when the upstream omits the data field', async () => {
setEnvKey('env-key')
mockFetch.mockResolvedValueOnce(jsonResponse({ object: 'list' }))
const res = await GET(createMockRequest('GET'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
})
it('dedupes repeated model ids', async () => {
setEnvKey('env-key')
mockFetch.mockResolvedValueOnce(
jsonResponse({
data: [{ id: 'model-a' }, { id: 'model-a' }, { id: 'model-b' }],
})
)
const res = await GET(createMockRequest('GET'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: ['baseten/model-a', 'baseten/model-b'] })
})
it('drops models removed by the blacklist filter', async () => {
setEnvKey('env-key')
mockFilterBlacklistedModels.mockImplementation((models: string[]) =>
models.filter((m) => m !== 'baseten/blocked-model')
)
mockFetch.mockResolvedValueOnce(
jsonResponse({
data: [{ id: 'allowed-model' }, { id: 'blocked-model' }],
})
)
const res = await GET(createMockRequest('GET'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: ['baseten/allowed-model'] })
})
})
apps/sim/app/api/providers/baseten/models/route.ts
+93
View File
@@ -0,0 +1,93 @@
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { type NextRequest, NextResponse } from 'next/server'
import {
basetenProviderModelsQuerySchema,
basetenUpstreamResponseSchema,
providerModelsResponseSchema,
} from '@/lib/api/contracts/providers'
import { validationErrorResponse } from '@/lib/api/server'
import { getBYOKKey } from '@/lib/api-key/byok'
import { getSession } from '@/lib/auth'
import { env } from '@/lib/core/config/env'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
import { filterBlacklistedModels, isProviderBlacklisted } from '@/providers/utils'
const logger = createLogger('BasetenModelsAPI')
export const GET = withRouteHandler(async (request: NextRequest) => {
if (isProviderBlacklisted('baseten')) {
logger.info('Baseten provider is blacklisted, returning empty models')
return NextResponse.json({ models: [] })
}
let apiKey: string | undefined
const queryValidation = basetenProviderModelsQuerySchema.safeParse({
workspaceId: request.nextUrl.searchParams.get('workspaceId') ?? undefined,
})
if (!queryValidation.success) return validationErrorResponse(queryValidation.error)
const { workspaceId } = queryValidation.data
if (workspaceId) {
const session = await getSession()
if (session?.user?.id) {
const permission = await getUserEntityPermissions(session.user.id, 'workspace', workspaceId)
if (permission) {
const byokResult = await getBYOKKey(workspaceId, 'baseten')
if (byokResult) {
apiKey = byokResult.apiKey
}
}
}
}
if (!apiKey) {
apiKey = env.BASETEN_API_KEY
}
if (!apiKey) {
logger.info('No Baseten API key available, returning empty models')
return NextResponse.json({ models: [] })
}
try {
const response = await fetch('https://inference.baseten.co/v1/models', {
headers: {
Authorization: `Bearer ${apiKey}`,
'Content-Type': 'application/json',
},
cache: 'no-store',
})
if (!response.ok) {
logger.warn('Failed to fetch Baseten models', {
status: response.status,
statusText: response.statusText,
})
return NextResponse.json({ models: [] })
}
const data = basetenUpstreamResponseSchema.parse(await response.json())
const allModels: string[] = []
for (const model of data.data ?? []) {
allModels.push(`baseten/${model.id}`)
}
const uniqueModels = Array.from(new Set(allModels))
const models = filterBlacklistedModels(uniqueModels)
logger.info('Successfully fetched Baseten models', {
count: models.length,
filtered: uniqueModels.length - models.length,
})
return NextResponse.json(providerModelsResponseSchema.parse({ models }))
} catch (error) {
logger.error('Error fetching Baseten models', {
error: getErrorMessage(error, 'Unknown error'),
})
return NextResponse.json({ models: [] })
}
})
apps/sim/app/api/providers/ollama-cloud/models/route.test.ts
+238
View File
@@ -0,0 +1,238 @@
/**
* @vitest-environment node
*/
import { createMockRequest } from '@sim/testing'
import { beforeEach, describe, expect, it, vi } from 'vitest'
const {
mockFilterBlacklistedModels,
mockIsProviderBlacklisted,
mockGetBYOKKey,
mockGetSession,
mockGetUserEntityPermissions,
mockFetch,
} = vi.hoisted(() => ({
mockFilterBlacklistedModels: vi.fn(),
mockIsProviderBlacklisted: vi.fn(),
mockGetBYOKKey: vi.fn(),
mockGetSession: vi.fn(),
mockGetUserEntityPermissions: vi.fn(),
mockFetch: vi.fn(),
}))
vi.mock('@/providers/utils', () => ({
filterBlacklistedModels: mockFilterBlacklistedModels,
isProviderBlacklisted: mockIsProviderBlacklisted,
}))
vi.mock('@/lib/api-key/byok', () => ({
getBYOKKey: mockGetBYOKKey,
}))
vi.mock('@/lib/auth', () => ({
getSession: mockGetSession,
}))
vi.mock('@/lib/workspaces/permissions/utils', () => ({
getUserEntityPermissions: mockGetUserEntityPermissions,
}))
import { GET } from '@/app/api/providers/ollama-cloud/models/route'
const OLLAMA_CLOUD_TAGS_URL = 'https://ollama.com/api/tags'
const okResponse = (body: unknown) => ({
ok: true,
status: 200,
statusText: 'OK',
json: vi.fn().mockResolvedValue(body),
})
const errorResponse = (status: number, statusText = 'Unauthorized') => ({
ok: false,
status,
statusText,
json: vi.fn().mockResolvedValue({}),
})
/**
* Builds a request whose query string carries the given workspaceId. Passing
* `undefined` omits the param entirely; passing `''` produces `?workspaceId=`.
*/
const requestWithWorkspace = (workspaceId?: string) => {
const url = new URL('http://localhost:3000/api/providers/ollama-cloud/models')
if (workspaceId !== undefined) {
url.searchParams.set('workspaceId', workspaceId)
}
return createMockRequest('GET', undefined, {}, url.toString())
}
const fetchAuthHeader = () => {
const init = mockFetch.mock.calls[0]?.[1] as RequestInit | undefined
const headers = init?.headers as Record<string, string> | undefined
return headers?.Authorization
}
/** Grants a session + workspace permission so the BYOK lookup is reached. */
const grantWorkspaceAccess = () => {
mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
mockGetUserEntityPermissions.mockResolvedValue('admin')
}
describe('GET /api/providers/ollama-cloud/models', () => {
beforeEach(() => {
vi.clearAllMocks()
vi.stubGlobal('fetch', mockFetch)
mockIsProviderBlacklisted.mockReturnValue(false)
mockFilterBlacklistedModels.mockImplementation((models: string[]) => models)
mockGetBYOKKey.mockResolvedValue(null)
mockGetSession.mockResolvedValue(null)
mockGetUserEntityPermissions.mockResolvedValue(null)
})
it('returns empty models without calling fetch when the provider is blacklisted', async () => {
mockIsProviderBlacklisted.mockReturnValue(true)
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
expect(mockFetch).not.toHaveBeenCalled()
})
it('returns empty models when there is no workspaceId (BYOK only, no env fallback)', async () => {
const res = await GET(requestWithWorkspace())
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
expect(mockFetch).not.toHaveBeenCalled()
expect(mockGetBYOKKey).not.toHaveBeenCalled()
})
it('returns empty models when the workspace has no stored BYOK key (never falls back to a hosted key)', async () => {
grantWorkspaceAccess()
mockGetBYOKKey.mockResolvedValue(null)
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
expect(mockGetBYOKKey).toHaveBeenCalledWith('ws-1', 'ollama-cloud')
expect(mockFetch).not.toHaveBeenCalled()
})
it('fetches /api/tags with the BYOK key and prefixes each model name with ollama-cloud/', async () => {
grantWorkspaceAccess()
mockGetBYOKKey.mockResolvedValue({ apiKey: 'byok-ollama-key' })
mockFetch.mockResolvedValue(
okResponse({
models: [{ name: 'gpt-oss:120b' }, { name: 'deepseek-v3.1:671b' }],
})
)
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({
models: ['ollama-cloud/gpt-oss:120b', 'ollama-cloud/deepseek-v3.1:671b'],
})
expect(mockFetch).toHaveBeenCalledTimes(1)
expect(mockFetch.mock.calls[0][0]).toBe(OLLAMA_CLOUD_TAGS_URL)
expect(fetchAuthHeader()).toBe('Bearer byok-ollama-key')
})
it('does not call getBYOKKey when there is a workspaceId but no session', async () => {
mockGetSession.mockResolvedValue(null)
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
expect(mockGetBYOKKey).not.toHaveBeenCalled()
expect(mockFetch).not.toHaveBeenCalled()
})
it('does not call getBYOKKey when the session user lacks workspace permission', async () => {
mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
mockGetUserEntityPermissions.mockResolvedValue(null)
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
expect(mockGetBYOKKey).not.toHaveBeenCalled()
expect(mockFetch).not.toHaveBeenCalled()
})
it('returns empty models when the upstream fetch responds non-ok', async () => {
grantWorkspaceAccess()
mockGetBYOKKey.mockResolvedValue({ apiKey: 'byok-ollama-key' })
mockFetch.mockResolvedValue(errorResponse(401, 'Unauthorized'))
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
})
it('returns empty models when the upstream fetch throws', async () => {
grantWorkspaceAccess()
mockGetBYOKKey.mockResolvedValue({ apiKey: 'byok-ollama-key' })
mockFetch.mockRejectedValue(new Error('network down'))
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
})
it('returns a validation error for an empty workspaceId query param', async () => {
const res = await GET(requestWithWorkspace(''))
expect(res.status).toBe(400)
const body = (await res.json()) as { error: string }
expect(body.error).toBe('Validation error')
expect(mockFetch).not.toHaveBeenCalled()
})
it('dedupes duplicate model names from the upstream response', async () => {
grantWorkspaceAccess()
mockGetBYOKKey.mockResolvedValue({ apiKey: 'byok-ollama-key' })
mockFetch.mockResolvedValue(
okResponse({
models: [{ name: 'gpt-oss:120b' }, { name: 'gpt-oss:120b' }, { name: 'qwen3-coder:480b' }],
})
)
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({
models: ['ollama-cloud/gpt-oss:120b', 'ollama-cloud/qwen3-coder:480b'],
})
})
it('applies the blacklist filter to the deduped model list', async () => {
grantWorkspaceAccess()
mockGetBYOKKey.mockResolvedValue({ apiKey: 'byok-ollama-key' })
mockFilterBlacklistedModels.mockImplementation((models: string[]) =>
models.filter((m) => !m.includes('qwen'))
)
mockFetch.mockResolvedValue(
okResponse({
models: [{ name: 'gpt-oss:120b' }, { name: 'qwen3-coder:480b' }],
})
)
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: ['ollama-cloud/gpt-oss:120b'] })
expect(mockFilterBlacklistedModels).toHaveBeenCalledWith([
'ollama-cloud/gpt-oss:120b',
'ollama-cloud/qwen3-coder:480b',
])
})
})
apps/sim/app/api/providers/ollama-cloud/models/route.ts
+91
View File
@@ -0,0 +1,91 @@
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { type NextRequest, NextResponse } from 'next/server'
import {
ollamaCloudProviderModelsQuerySchema,
ollamaUpstreamResponseSchema,
providerModelsResponseSchema,
} from '@/lib/api/contracts/providers'
import { validationErrorResponse } from '@/lib/api/server'
import { getBYOKKey } from '@/lib/api-key/byok'
import { getSession } from '@/lib/auth'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
import { filterBlacklistedModels, isProviderBlacklisted } from '@/providers/utils'
const logger = createLogger('OllamaCloudModelsAPI')
/**
* Get available Ollama Cloud models.
*
* Ollama Cloud is BYOK-only — Sim never supplies a hosted key and never bills
* usage. Models are listed only when the workspace has stored its own Ollama
* API key, which is used to authenticate against the cloud `/api/tags` endpoint.
*/
export const GET = withRouteHandler(async (request: NextRequest) => {
if (isProviderBlacklisted('ollama-cloud')) {
logger.info('Ollama Cloud provider is blacklisted, returning empty models')
return NextResponse.json({ models: [] })
}
const queryValidation = ollamaCloudProviderModelsQuerySchema.safeParse({
workspaceId: request.nextUrl.searchParams.get('workspaceId') ?? undefined,
})
if (!queryValidation.success) return validationErrorResponse(queryValidation.error)
const { workspaceId } = queryValidation.data
let apiKey: string | undefined
if (workspaceId) {
const session = await getSession()
if (session?.user?.id) {
const permission = await getUserEntityPermissions(session.user.id, 'workspace', workspaceId)
if (permission) {
const byokResult = await getBYOKKey(workspaceId, 'ollama-cloud')
if (byokResult) {
apiKey = byokResult.apiKey
}
}
}
}
if (!apiKey) {
logger.info('No Ollama Cloud API key available, returning empty models')
return NextResponse.json({ models: [] })
}
try {
const response = await fetch('https://ollama.com/api/tags', {
headers: {
Authorization: `Bearer ${apiKey}`,
'Content-Type': 'application/json',
},
cache: 'no-store',
})
if (!response.ok) {
logger.warn('Failed to fetch Ollama Cloud models', {
status: response.status,
statusText: response.statusText,
})
return NextResponse.json({ models: [] })
}
const data = ollamaUpstreamResponseSchema.parse(await response.json())
const allModels = data.models.map((model) => `ollama-cloud/${model.name}`)
const uniqueModels = Array.from(new Set(allModels))
const models = filterBlacklistedModels(uniqueModels)
logger.info('Successfully fetched Ollama Cloud models', {
count: models.length,
filtered: uniqueModels.length - models.length,
})
return NextResponse.json(providerModelsResponseSchema.parse({ models }))
} catch (error) {
logger.error('Error fetching Ollama Cloud models', {
error: getErrorMessage(error, 'Unknown error'),
})
return NextResponse.json({ models: [] })
}
})
apps/sim/app/api/providers/together/models/route.test.ts
+259
View File
@@ -0,0 +1,259 @@
/**
* @vitest-environment node
*/
import { createMockRequest } from '@sim/testing'
import { beforeEach, describe, expect, it, vi } from 'vitest'
const {
mockFilterBlacklistedModels,
mockIsProviderBlacklisted,
mockGetBYOKKey,
mockGetSession,
mockGetUserEntityPermissions,
mockFetch,
mutableEnv,
} = vi.hoisted(() => ({
mockFilterBlacklistedModels: vi.fn(),
mockIsProviderBlacklisted: vi.fn(),
mockGetBYOKKey: vi.fn(),
mockGetSession: vi.fn(),
mockGetUserEntityPermissions: vi.fn(),
mockFetch: vi.fn(),
mutableEnv: { TOGETHER_API_KEY: undefined as string | undefined },
}))
vi.mock('@/providers/utils', () => ({
filterBlacklistedModels: mockFilterBlacklistedModels,
isProviderBlacklisted: mockIsProviderBlacklisted,
}))
vi.mock('@/lib/api-key/byok', () => ({
getBYOKKey: mockGetBYOKKey,
}))
vi.mock('@/lib/auth', () => ({
getSession: mockGetSession,
}))
vi.mock('@/lib/workspaces/permissions/utils', () => ({
getUserEntityPermissions: mockGetUserEntityPermissions,
}))
vi.mock('@/lib/core/config/env', () => ({
env: mutableEnv,
}))
import { GET } from '@/app/api/providers/together/models/route'
const TOGETHER_MODELS_URL = 'https://api.together.ai/v1/models'
const okResponse = (body: unknown) => ({
ok: true,
status: 200,
statusText: 'OK',
json: vi.fn().mockResolvedValue(body),
})
const errorResponse = (status: number, statusText = 'Unauthorized') => ({
ok: false,
status,
statusText,
json: vi.fn().mockResolvedValue({}),
})
/**
* Builds a request whose query string carries the given workspaceId. Passing
* `undefined` omits the param entirely; passing `''` produces `?workspaceId=`.
*/
const requestWithWorkspace = (workspaceId?: string) => {
const url = new URL('http://localhost:3000/api/providers/together/models')
if (workspaceId !== undefined) {
url.searchParams.set('workspaceId', workspaceId)
}
return createMockRequest('GET', undefined, {}, url.toString())
}
const fetchAuthHeader = () => {
const init = mockFetch.mock.calls[0]?.[1] as RequestInit | undefined
const headers = init?.headers as Record<string, string> | undefined
return headers?.Authorization
}
describe('GET /api/providers/together/models', () => {
beforeEach(() => {
vi.clearAllMocks()
vi.stubGlobal('fetch', mockFetch)
mutableEnv.TOGETHER_API_KEY = undefined
mockIsProviderBlacklisted.mockReturnValue(false)
mockFilterBlacklistedModels.mockImplementation((models: string[]) => models)
mockGetBYOKKey.mockResolvedValue(null)
mockGetSession.mockResolvedValue(null)
mockGetUserEntityPermissions.mockResolvedValue(null)
})
it('returns empty models without calling fetch when the provider is blacklisted', async () => {
mockIsProviderBlacklisted.mockReturnValue(true)
const res = await GET(requestWithWorkspace())
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
expect(mockFetch).not.toHaveBeenCalled()
})
it('returns empty models when there is no workspaceId and no env key', async () => {
const res = await GET(requestWithWorkspace())
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
expect(mockFetch).not.toHaveBeenCalled()
})
it('fetches with the env key and prefixes each model id with together/', async () => {
mutableEnv.TOGETHER_API_KEY = 'env-together-key'
mockFetch.mockResolvedValue(
okResponse([{ id: 'moonshotai/Kimi-K2-Instruct' }, { id: 'Qwen/Qwen2.5-72B-Instruct-Turbo' }])
)
const res = await GET(requestWithWorkspace())
expect(res.status).toBe(200)
expect(await res.json()).toEqual({
models: ['together/moonshotai/Kimi-K2-Instruct', 'together/Qwen/Qwen2.5-72B-Instruct-Turbo'],
})
expect(mockFetch).toHaveBeenCalledTimes(1)
expect(mockFetch.mock.calls[0][0]).toBe(TOGETHER_MODELS_URL)
expect(fetchAuthHeader()).toBe('Bearer env-together-key')
})
it('uses the BYOK key when a workspace, session, and permission are present', async () => {
mutableEnv.TOGETHER_API_KEY = 'env-together-key'
mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
mockGetUserEntityPermissions.mockResolvedValue('admin')
mockGetBYOKKey.mockResolvedValue({ apiKey: 'byok-together-key' })
mockFetch.mockResolvedValue(okResponse([{ id: 'moonshotai/Kimi-K2-Instruct' }]))
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: ['together/moonshotai/Kimi-K2-Instruct'] })
expect(mockGetBYOKKey).toHaveBeenCalledWith('ws-1', 'together')
expect(fetchAuthHeader()).toBe('Bearer byok-together-key')
})
it('falls back to the env key when a workspaceId is given but there is no session', async () => {
mutableEnv.TOGETHER_API_KEY = 'env-together-key'
mockGetSession.mockResolvedValue(null)
mockFetch.mockResolvedValue(okResponse([{ id: 'moonshotai/Kimi-K2-Instruct' }]))
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: ['together/moonshotai/Kimi-K2-Instruct'] })
expect(mockGetBYOKKey).not.toHaveBeenCalled()
expect(fetchAuthHeader()).toBe('Bearer env-together-key')
})
it('falls back to the env key when the session user lacks workspace permission', async () => {
mutableEnv.TOGETHER_API_KEY = 'env-together-key'
mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
mockGetUserEntityPermissions.mockResolvedValue(null)
mockFetch.mockResolvedValue(okResponse([{ id: 'moonshotai/Kimi-K2-Instruct' }]))
const res = await GET(requestWithWorkspace('ws-1'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: ['together/moonshotai/Kimi-K2-Instruct'] })
expect(mockGetBYOKKey).not.toHaveBeenCalled()
expect(fetchAuthHeader()).toBe('Bearer env-together-key')
})
it('returns empty models when the upstream fetch responds non-ok', async () => {
mutableEnv.TOGETHER_API_KEY = 'env-together-key'
mockFetch.mockResolvedValue(errorResponse(401, 'Unauthorized'))
const res = await GET(requestWithWorkspace())
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
})
it('returns empty models when the upstream fetch throws', async () => {
mutableEnv.TOGETHER_API_KEY = 'env-together-key'
mockFetch.mockRejectedValue(new Error('network down'))
const res = await GET(requestWithWorkspace())
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: [] })
})
it('returns a validation error for an empty workspaceId query param', async () => {
const res = await GET(requestWithWorkspace(''))
expect(res.status).toBe(400)
const body = (await res.json()) as { error: string }
expect(body.error).toBe('Validation error')
expect(mockFetch).not.toHaveBeenCalled()
})
it('dedupes duplicate model ids from the upstream array', async () => {
mutableEnv.TOGETHER_API_KEY = 'env-together-key'
mockFetch.mockResolvedValue(
okResponse([
{ id: 'moonshotai/Kimi-K2-Instruct' },
{ id: 'moonshotai/Kimi-K2-Instruct' },
{ id: 'Qwen/Qwen2.5-72B-Instruct-Turbo' },
])
)
const res = await GET(requestWithWorkspace())
expect(res.status).toBe(200)
expect(await res.json()).toEqual({
models: ['together/moonshotai/Kimi-K2-Instruct', 'together/Qwen/Qwen2.5-72B-Instruct-Turbo'],
})
})
it('applies the blacklist filter to the deduped model list', async () => {
mutableEnv.TOGETHER_API_KEY = 'env-together-key'
mockFilterBlacklistedModels.mockImplementation((models: string[]) =>
models.filter((m) => !m.includes('Qwen'))
)
mockFetch.mockResolvedValue(
okResponse([{ id: 'moonshotai/Kimi-K2-Instruct' }, { id: 'Qwen/Qwen2.5-72B-Instruct-Turbo' }])
)
const res = await GET(requestWithWorkspace())
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ models: ['together/moonshotai/Kimi-K2-Instruct'] })
expect(mockFilterBlacklistedModels).toHaveBeenCalledWith([
'together/moonshotai/Kimi-K2-Instruct',
'together/Qwen/Qwen2.5-72B-Instruct-Turbo',
])
})
it('filters out non-chat model types (image, embedding, rerank, etc.)', async () => {
mutableEnv.TOGETHER_API_KEY = 'env-together-key'
mockFetch.mockResolvedValue(
okResponse([
{ id: 'meta-llama/Llama-3.3-70B-Instruct-Turbo', type: 'chat' },
{ id: 'black-forest-labs/FLUX.1-schnell', type: 'image' },
{ id: 'BAAI/bge-large-en-v1.5', type: 'embedding' },
{ id: 'Salesforce/Llama-Rank-V1', type: 'rerank' },
{ id: 'openai/whisper-large-v3', type: 'transcribe' },
])
)
const res = await GET(requestWithWorkspace())
expect(res.status).toBe(200)
expect(await res.json()).toEqual({
models: ['together/meta-llama/Llama-3.3-70B-Instruct-Turbo'],
})
})
})
apps/sim/app/api/providers/together/models/route.ts
+105
View File
@@ -0,0 +1,105 @@
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { type NextRequest, NextResponse } from 'next/server'
import {
providerModelsResponseSchema,
togetherProviderModelsQuerySchema,
togetherUpstreamResponseSchema,
} from '@/lib/api/contracts/providers'
import { validationErrorResponse } from '@/lib/api/server'
import { getBYOKKey } from '@/lib/api-key/byok'
import { getSession } from '@/lib/auth'
import { env } from '@/lib/core/config/env'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
import { filterBlacklistedModels, isProviderBlacklisted } from '@/providers/utils'
const logger = createLogger('TogetherModelsAPI')
/** Together's catalog includes non-text models; only chat models work with chat completions. */
const NON_CHAT_MODEL_TYPES = new Set([
'image',
'video',
'audio',
'transcribe',
'embedding',
'moderation',
'rerank',
])
export const GET = withRouteHandler(async (request: NextRequest) => {
if (isProviderBlacklisted('together')) {
logger.info('Together provider is blacklisted, returning empty models')
return NextResponse.json({ models: [] })
}
let apiKey: string | undefined
const queryValidation = togetherProviderModelsQuerySchema.safeParse({
workspaceId: request.nextUrl.searchParams.get('workspaceId') ?? undefined,
})
if (!queryValidation.success) return validationErrorResponse(queryValidation.error)
const { workspaceId } = queryValidation.data
if (workspaceId) {
const session = await getSession()
if (session?.user?.id) {
const permission = await getUserEntityPermissions(session.user.id, 'workspace', workspaceId)
if (permission) {
const byokResult = await getBYOKKey(workspaceId, 'together')
if (byokResult) {
apiKey = byokResult.apiKey
}
}
}
}
if (!apiKey) {
apiKey = env.TOGETHER_API_KEY
}
if (!apiKey) {
logger.info('No Together API key available, returning empty models')
return NextResponse.json({ models: [] })
}
try {
const response = await fetch('https://api.together.ai/v1/models', {
headers: {
Authorization: `Bearer ${apiKey}`,
'Content-Type': 'application/json',
},
cache: 'no-store',
})
if (!response.ok) {
logger.warn('Failed to fetch Together models', {
status: response.status,
statusText: response.statusText,
})
return NextResponse.json({ models: [] })
}
const data = togetherUpstreamResponseSchema.parse(await response.json())
const allModels: string[] = []
for (const model of data) {
if (model.type && NON_CHAT_MODEL_TYPES.has(model.type)) continue
allModels.push(`together/${model.id}`)
}
const uniqueModels = Array.from(new Set(allModels))
const models = filterBlacklistedModels(uniqueModels)
logger.info('Successfully fetched Together models', {
count: models.length,
filtered: uniqueModels.length - models.length,
})
return NextResponse.json(providerModelsResponseSchema.parse({ models }))
} catch (error) {
logger.error('Error fetching Together models', {
error: getErrorMessage(error, 'Unknown error'),
})
return NextResponse.json({ models: [] })
}
})
apps/sim/app/api/superuser/import-workflow/route.ts
+36 -24
View File
@@ -7,8 +7,8 @@ import { type NextRequest, NextResponse } from 'next/server'
import { importWorkflowAsSuperuserContract } from '@/lib/api/contracts/workflows'
import { parseRequest } from '@/lib/api/server'
import { getSession } from '@/lib/auth'
import { appendCopilotChatMessages } from '@/lib/copilot/chat/messages-dual-write'
import type { PersistedMessage } from '@/lib/copilot/chat/persisted-message'
import { loadCopilotChatMessages } from '@/lib/copilot/chat/lifecycle'
import { appendCopilotChatMessages } from '@/lib/copilot/chat/messages-store'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { verifyEffectiveSuperUser } from '@/lib/permissions/super-user'
import { parseWorkflowJson } from '@/lib/workflows/operations/import-export'
@@ -167,34 +167,46 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
// Copy copilot chats associated with the source workflow
const sourceCopilotChats = await db
.select()
.select({
id: copilotChats.id,
title: copilotChats.title,
model: copilotChats.model,
previewYaml: copilotChats.previewYaml,
planArtifact: copilotChats.planArtifact,
config: copilotChats.config,
})
.from(copilotChats)
.where(eq(copilotChats.workflowId, workflowId))
let copilotChatsImported = 0
for (const chat of sourceCopilotChats) {
const [imported] = await db
.insert(copilotChats)
.values({
userId: session.user.id,
workflowId: newWorkflowId,
title: chat.title ? `[Import] ${chat.title}` : null,
messages: chat.messages,
model: chat.model,
conversationId: null, // Don't copy conversation ID
previewYaml: chat.previewYaml,
planArtifact: chat.planArtifact,
config: chat.config,
createdAt: new Date(),
updatedAt: new Date(),
})
.returning({ id: copilotChats.id })
if (imported && Array.isArray(chat.messages) && chat.messages.length > 0) {
await appendCopilotChatMessages(imported.id, chat.messages as PersistedMessage[], {
chatModel: chat.model,
})
}
const sourceMessages = await loadCopilotChatMessages(chat.id)
await db.transaction(async (tx) => {
const [imported] = await tx
.insert(copilotChats)
.values({
userId: session.user.id,
workflowId: newWorkflowId,
title: chat.title ? `[Import] ${chat.title}` : null,
model: chat.model,
conversationId: null, // Don't copy conversation ID
previewYaml: chat.previewYaml,
planArtifact: chat.planArtifact,
config: chat.config,
createdAt: new Date(),
updatedAt: new Date(),
})
.returning({ id: copilotChats.id })
if (imported && sourceMessages.length > 0) {
await appendCopilotChatMessages(
imported.id,
sourceMessages,
{ chatModel: chat.model },
tx
)
}
})
copilotChatsImported++
}
apps/sim/app/api/table/[tableId]/export/route.ts
+16 -1
View File
@@ -53,7 +53,9 @@ export const GET = withRouteHandler(async (request: NextRequest, { params }: Rou
const encoder = new TextEncoder()
try {
if (format === 'csv') {
controller.enqueue(encoder.encode(`${toCsvRow(columns.map((c) => c.name))}\n`))
controller.enqueue(
encoder.encode(`${toCsvRow(columns.map((c) => neutralizeCsvFormula(c.name)))}\n`)
)
} else {
controller.enqueue(encoder.encode('['))
}
@@ -111,10 +113,23 @@ function sanitizeFilename(name: string): string {
return cleaned || 'table'
}
/**
* Prefixes a single quote to values starting with a spreadsheet formula trigger
* (`=`, `+`, `-`, `@`, tab, CR), neutralizing CSV injection in Excel/Sheets.
*/
function neutralizeCsvFormula(value: string): string {
return /^[=+\-@\t\r]/.test(value) ? `'${value}` : value
}
/**
* Serializes a cell for CSV. Only string cells are formula-neutralized; numbers,
* booleans, dates, and JSON objects can never form a trigger and pass through verbatim.
*/
function formatCsvValue(value: unknown): string {
if (value === null || value === undefined) return ''
if (value instanceof Date) return value.toISOString()
if (typeof value === 'object') return JSON.stringify(value)
if (typeof value === 'string') return neutralizeCsvFormula(value)
return String(value)
}
apps/sim/app/api/tools/airtable/bases/route.ts
+81 -19
View File
@@ -11,6 +11,71 @@ const logger = createLogger('AirtableBasesAPI')
export const dynamic = 'force-dynamic'
const AIRTABLE_MAX_BASES_PAGES = 50
interface AirtableBase {
id: string
name: string
}
/**
* Lists all Airtable bases, following the `offset` continuation token the Meta
* API returns (an opaque string, passed back verbatim as `?offset=`) so the
* full set is returned. Bounded by `AIRTABLE_MAX_BASES_PAGES`; logs a warning
* rather than silently dropping bases when the cap is hit.
*/
async function fetchAllBases(accessToken: string): Promise<AirtableBase[]> {
const bases: AirtableBase[] = []
let offset: string | undefined
for (let page = 0; page < AIRTABLE_MAX_BASES_PAGES; page++) {
const url = new URL('https://api.airtable.com/v0/meta/bases')
if (offset) {
url.searchParams.set('offset', offset)
}
const response = await fetch(url.toString(), {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
throw new AirtableFetchError(response.status, errorData)
}
const data = (await response.json()) as { bases?: AirtableBase[]; offset?: string }
if (Array.isArray(data.bases)) {
bases.push(...data.bases)
}
offset = data.offset || undefined
if (!offset) {
return bases
}
if (page === AIRTABLE_MAX_BASES_PAGES - 1) {
logger.warn('Airtable bases listing hit pagination cap; base list may be incomplete', {
pages: AIRTABLE_MAX_BASES_PAGES,
})
}
}
return bases
}
class AirtableFetchError extends Error {
constructor(
readonly status: number,
readonly details: unknown
) {
super('Failed to fetch Airtable bases')
this.name = 'AirtableFetchError'
}
}
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
try {
@@ -45,27 +110,24 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch('https://api.airtable.com/v0/meta/bases', {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch Airtable bases', {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ error: 'Failed to fetch Airtable bases', details: errorData },
{ status: response.status }
)
let allBases: AirtableBase[]
try {
allBases = await fetchAllBases(accessToken)
} catch (error) {
if (error instanceof AirtableFetchError) {
logger.error('Failed to fetch Airtable bases', {
status: error.status,
error: error.details,
})
return NextResponse.json(
{ error: 'Failed to fetch Airtable bases', details: error.details },
{ status: error.status }
)
}
throw error
}
const data = await response.json()
const bases = (data.bases || []).map((base: { id: string; name: string }) => ({
const bases = allBases.map((base) => ({
id: base.id,
name: base.name,
}))
apps/sim/app/api/tools/asana/workspaces/route.ts
+91 -19
View File
@@ -11,6 +11,81 @@ const logger = createLogger('AsanaWorkspacesAPI')
export const dynamic = 'force-dynamic'
const ASANA_PAGE_LIMIT = 100
const ASANA_MAX_WORKSPACES_PAGES = 50
interface AsanaWorkspace {
gid: string
name: string
}
interface AsanaWorkspacesPage {
data?: AsanaWorkspace[]
next_page?: {
offset?: string
} | null
}
/**
* Lists all Asana workspaces using `limit`/`offset` pagination, following
* `next_page.offset` (an opaque token, passed back verbatim as `?offset=`)
* until `next_page` is null so the full set is returned. Bounded by
* `ASANA_MAX_WORKSPACES_PAGES`; logs a warning rather than silently dropping
* workspaces when the cap is hit.
*/
async function fetchAllWorkspaces(accessToken: string): Promise<AsanaWorkspace[]> {
const workspaces: AsanaWorkspace[] = []
let offset: string | undefined
for (let page = 0; page < ASANA_MAX_WORKSPACES_PAGES; page++) {
const url = new URL('https://app.asana.com/api/1.0/workspaces')
url.searchParams.set('limit', String(ASANA_PAGE_LIMIT))
if (offset) {
url.searchParams.set('offset', offset)
}
const response = await fetch(url.toString(), {
headers: {
Authorization: `Bearer ${accessToken}`,
Accept: 'application/json',
},
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
throw new AsanaFetchError(response.status, errorData)
}
const data = (await response.json()) as AsanaWorkspacesPage
if (Array.isArray(data.data)) {
workspaces.push(...data.data)
}
offset = data.next_page?.offset || undefined
if (!offset) {
return workspaces
}
if (page === ASANA_MAX_WORKSPACES_PAGES - 1) {
logger.warn('Asana workspaces listing hit pagination cap; workspace list may be incomplete', {
pages: ASANA_MAX_WORKSPACES_PAGES,
})
}
}
return workspaces
}
class AsanaFetchError extends Error {
constructor(
readonly status: number,
readonly details: unknown
) {
super('Failed to fetch Asana workspaces')
this.name = 'AsanaFetchError'
}
}
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
try {
@@ -42,27 +117,24 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch('https://app.asana.com/api/1.0/workspaces', {
headers: {
Authorization: `Bearer ${accessToken}`,
Accept: 'application/json',
},
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch Asana workspaces', {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ error: 'Failed to fetch Asana workspaces', details: errorData },
{ status: response.status }
)
let allWorkspaces: AsanaWorkspace[]
try {
allWorkspaces = await fetchAllWorkspaces(accessToken)
} catch (error) {
if (error instanceof AsanaFetchError) {
logger.error('Failed to fetch Asana workspaces', {
status: error.status,
error: error.details,
})
return NextResponse.json(
{ error: 'Failed to fetch Asana workspaces', details: error.details },
{ status: error.status }
)
}
throw error
}
const data = await response.json()
const workspaces = (data.data || []).map((workspace: { gid: string; name: string }) => ({
const workspaces = allWorkspaces.map((workspace) => ({
id: workspace.gid,
name: workspace.name,
}))
apps/sim/app/api/tools/cloudwatch/describe-log-groups/route.ts
+52 -14
View File
@@ -10,6 +10,12 @@ import { createCloudWatchLogsClient } from '@/app/api/tools/cloudwatch/utils'
const logger = createLogger('CloudWatchDescribeLogGroups')
/** AWS DescribeLogGroups caps `limit` at 50 items per page. */
const LOG_GROUPS_PAGE_SIZE = 50
/** Upper bound on pages drained to avoid unbounded loops on very large accounts. */
const MAX_LOG_GROUPS_PAGES = 20
export const POST = withRouteHandler(async (request: NextRequest) => {
try {
const auth = await checkSessionOrInternalAuth(request)
@@ -33,26 +39,58 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
})
try {
const command = new DescribeLogGroupsCommand({
...(validatedData.prefix && { logGroupNamePrefix: validatedData.prefix }),
...(validatedData.limit !== undefined && { limit: validatedData.limit }),
})
const totalLimit = validatedData.limit
const logGroups: {
logGroupName: string
arn: string
storedBytes: number
retentionInDays: number | undefined
creationTime: number | undefined
}[] = []
let nextToken: string | undefined
const response = await client.send(command)
for (let page = 0; page < MAX_LOG_GROUPS_PAGES; page++) {
const pageLimit =
totalLimit !== undefined
? Math.min(LOG_GROUPS_PAGE_SIZE, totalLimit - logGroups.length)
: LOG_GROUPS_PAGE_SIZE
const logGroups = (response.logGroups ?? []).map((lg) => ({
logGroupName: lg.logGroupName ?? '',
arn: lg.arn ?? '',
storedBytes: lg.storedBytes ?? 0,
retentionInDays: lg.retentionInDays,
creationTime: lg.creationTime,
}))
const command = new DescribeLogGroupsCommand({
...(validatedData.prefix && { logGroupNamePrefix: validatedData.prefix }),
limit: pageLimit,
...(nextToken && { nextToken }),
})
logger.info(`Successfully described ${logGroups.length} log groups`)
const response = await client.send(command)
for (const lg of response.logGroups ?? []) {
logGroups.push({
logGroupName: lg.logGroupName ?? '',
arn: lg.arn ?? '',
storedBytes: lg.storedBytes ?? 0,
retentionInDays: lg.retentionInDays,
creationTime: lg.creationTime,
})
}
nextToken = response.nextToken
if (!nextToken) break
if (totalLimit !== undefined && logGroups.length >= totalLimit) break
if (page === MAX_LOG_GROUPS_PAGES - 1) {
logger.warn(
`DescribeLogGroups hit pagination cap of ${MAX_LOG_GROUPS_PAGES} pages; log group list may be incomplete`
)
}
}
const cappedLogGroups = totalLimit !== undefined ? logGroups.slice(0, totalLimit) : logGroups
logger.info(`Successfully described ${cappedLogGroups.length} log groups`)
return NextResponse.json({
success: true,
output: { logGroups },
output: { logGroups: cappedLogGroups },
})
} finally {
client.destroy()
apps/sim/app/api/tools/cloudwatch/utils.ts
+70 -24
View File
@@ -5,6 +5,7 @@ import {
GetQueryResultsCommand,
type ResultField,
} from '@aws-sdk/client-cloudwatch-logs'
import { createLogger } from '@sim/logger'
import { sleep } from '@sim/utils/helpers'
import { DEFAULT_EXECUTION_TIMEOUT_MS } from '@/lib/core/execution-limits'
@@ -96,37 +97,82 @@ export async function pollQueryResults(
}
}
/** AWS DescribeLogStreams caps `limit` at 50 items per page. */
const LOG_STREAMS_PAGE_SIZE = 50
/** Upper bound on pages drained to avoid unbounded loops on log groups with many streams. */
const MAX_LOG_STREAMS_PAGES = 20
const logger = createLogger('CloudWatchUtils')
interface DescribedLogStream {
logStreamName: string
lastEventTimestamp: number | undefined
firstEventTimestamp: number | undefined
creationTime: number | undefined
storedBytes: number
}
/**
* Lists log streams for a log group, following `nextToken` so the complete set
* is returned rather than just the first page. Bounded by
* `MAX_LOG_STREAMS_PAGES`; logs a warning rather than silently dropping streams
* when the cap is hit. Ordering/prefix inputs are preserved across all pages.
*
* When `limit` is provided it is treated as a total result cap: draining stops
* once enough streams have been collected. When omitted, every page is drained.
*/
export async function describeLogStreams(
client: CloudWatchLogsClient,
logGroupName: string,
options?: { prefix?: string; limit?: number }
): Promise<{
logStreams: {
logStreamName: string
lastEventTimestamp: number | undefined
firstEventTimestamp: number | undefined
creationTime: number | undefined
storedBytes: number
}[]
}> {
): Promise<{ logStreams: DescribedLogStream[] }> {
const hasPrefix = Boolean(options?.prefix)
const command = new DescribeLogStreamsCommand({
logGroupName,
...(hasPrefix
? { orderBy: 'LogStreamName', logStreamNamePrefix: options!.prefix }
: { orderBy: 'LastEventTime', descending: true }),
...(options?.limit !== undefined && { limit: options.limit }),
})
const totalLimit = options?.limit
const logStreams: DescribedLogStream[] = []
let nextToken: string | undefined
for (let page = 0; page < MAX_LOG_STREAMS_PAGES; page++) {
const pageLimit =
totalLimit !== undefined
? Math.min(LOG_STREAMS_PAGE_SIZE, totalLimit - logStreams.length)
: LOG_STREAMS_PAGE_SIZE
const command = new DescribeLogStreamsCommand({
logGroupName,
...(hasPrefix
? { orderBy: 'LogStreamName', logStreamNamePrefix: options!.prefix }
: { orderBy: 'LastEventTime', descending: true }),
limit: pageLimit,
...(nextToken && { nextToken }),
})
const response = await client.send(command)
for (const ls of response.logStreams ?? []) {
logStreams.push({
logStreamName: ls.logStreamName ?? '',
lastEventTimestamp: ls.lastEventTimestamp,
firstEventTimestamp: ls.firstEventTimestamp,
creationTime: ls.creationTime,
storedBytes: ls.storedBytes ?? 0,
})
}
nextToken = response.nextToken
if (!nextToken) break
if (totalLimit !== undefined && logStreams.length >= totalLimit) break
if (page === MAX_LOG_STREAMS_PAGES - 1) {
logger.warn(
`DescribeLogStreams hit pagination cap of ${MAX_LOG_STREAMS_PAGES} pages; log stream list may be incomplete`,
{ logGroupName }
)
}
}
const response = await client.send(command)
return {
logStreams: (response.logStreams ?? []).map((ls) => ({
logStreamName: ls.logStreamName ?? '',
lastEventTimestamp: ls.lastEventTimestamp,
firstEventTimestamp: ls.firstEventTimestamp,
creationTime: ls.creationTime,
storedBytes: ls.storedBytes ?? 0,
})),
logStreams: totalLimit !== undefined ? logStreams.slice(0, totalLimit) : logStreams,
}
}
apps/sim/app/api/tools/drive/files/route.ts
+54 -23
View File
@@ -7,12 +7,21 @@ import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
import { validateAlphanumericId } from '@/lib/core/security/input-validation'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { drainGooglePagedList, GooglePageError } from '@/lib/oauth/google-pagination'
import { getScopesForService } from '@/lib/oauth/utils'
import { refreshAccessTokenIfNeeded, ServiceAccountTokenError } from '@/app/api/auth/oauth/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('GoogleDriveFilesAPI')
const MAX_DRIVE_FILE_PAGES = 20
const DRIVE_FILE_PAGE_SIZE = 100
interface DriveFilesResponse {
files?: DriveFile[]
nextPageToken?: string
}
function escapeForDriveQuery(value: string): string {
return value.replace(/\\/g, '\\\\').replace(/'/g, "\\'")
}
@@ -138,34 +147,56 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
if (query) {
qParts.push(`name contains '${escapeForDriveQuery(query)}'`)
}
const q = encodeURIComponent(qParts.join(' and '))
const q = qParts.join(' and ')
const response = await fetch(
`https://www.googleapis.com/drive/v3/files?q=${q}&corpora=allDrives&supportsAllDrives=true&includeItemsFromAllDrives=true&fields=files(id,name,mimeType,iconLink,webViewLink,thumbnailLink,createdTime,modifiedTime,size,owners,parents)`,
{
headers: {
Authorization: `Bearer ${accessToken}`,
let files: DriveFile[]
try {
const drained = await drainGooglePagedList<DriveFile, DriveFilesResponse>({
buildUrl: (pageToken) => {
const url = new URL('https://www.googleapis.com/drive/v3/files')
url.searchParams.set('q', q)
url.searchParams.set('corpora', 'allDrives')
url.searchParams.set('supportsAllDrives', 'true')
url.searchParams.set('includeItemsFromAllDrives', 'true')
url.searchParams.set('pageSize', String(DRIVE_FILE_PAGE_SIZE))
url.searchParams.set(
'fields',
'nextPageToken,files(id,name,mimeType,iconLink,webViewLink,thumbnailLink,createdTime,modifiedTime,size,owners,parents)'
)
if (pageToken) url.searchParams.set('pageToken', pageToken)
return url.toString()
},
}
)
if (!response.ok) {
const error = await response.json().catch(() => ({ error: { message: 'Unknown error' } }))
logger.error(`[${requestId}] Google Drive API error`, {
status: response.status,
error: error.error?.message || 'Failed to fetch files from Google Drive',
fetch: (url) =>
fetch(url, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
}),
parseError: (response) =>
response.json().catch(() => ({ error: { message: 'Unknown error' } })),
getItems: (body) => body.files,
getNextPageToken: (body) => body.nextPageToken,
maxPages: MAX_DRIVE_FILE_PAGES,
label: 'Google Drive files',
})
return NextResponse.json(
{
error: error.error?.message || 'Failed to fetch files from Google Drive',
},
{ status: response.status }
)
files = drained.items
} catch (error) {
if (error instanceof GooglePageError) {
const errorBody = error.body as { error?: { message?: string } }
logger.error(`[${requestId}] Google Drive API error`, {
status: error.status,
error: errorBody?.error?.message || 'Failed to fetch files from Google Drive',
})
return NextResponse.json(
{
error: errorBody?.error?.message || 'Failed to fetch files from Google Drive',
},
{ status: error.status }
)
}
throw error
}
const data = await response.json()
let files: DriveFile[] = data.files || []
if (mimeType === 'application/vnd.google-apps.spreadsheet') {
files = files.filter(
(file: DriveFile) => file.mimeType === 'application/vnd.google-apps.spreadsheet'
apps/sim/app/api/tools/google_bigquery/datasets/route.ts
+50 -31
View File
@@ -5,6 +5,7 @@ import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { drainGooglePagedList, GooglePageError } from '@/lib/oauth/google-pagination'
import { getScopesForService } from '@/lib/oauth/utils'
import { refreshAccessTokenIfNeeded, ServiceAccountTokenError } from '@/app/api/auth/oauth/utils'
@@ -12,6 +13,19 @@ const logger = createLogger('GoogleBigQueryDatasetsAPI')
export const dynamic = 'force-dynamic'
const MAX_DATASET_PAGES = 20
const DATASET_PAGE_SIZE = 200
interface BigQueryDataset {
datasetReference: { datasetId: string; projectId: string }
friendlyName?: string
}
interface BigQueryDatasetsResponse {
datasets?: BigQueryDataset[]
nextPageToken?: string
}
/**
* POST /api/tools/google_bigquery/datasets
*
@@ -71,41 +85,46 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch(
`https://bigquery.googleapis.com/bigquery/v2/projects/${encodeURIComponent(projectId)}/datasets?maxResults=200`,
{
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
}
)
const { items } = await drainGooglePagedList<BigQueryDataset, BigQueryDatasetsResponse>({
buildUrl: (pageToken) => {
const url = new URL(
`https://bigquery.googleapis.com/bigquery/v2/projects/${encodeURIComponent(projectId)}/datasets`
)
url.searchParams.set('maxResults', String(DATASET_PAGE_SIZE))
if (pageToken) url.searchParams.set('pageToken', pageToken)
return url.toString()
},
fetch: (url) =>
fetch(url, {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
}),
parseError: (response) => response.json().catch(() => ({})),
getItems: (body) => body.datasets,
getNextPageToken: (body) => body.nextPageToken,
maxPages: MAX_DATASET_PAGES,
label: 'BigQuery datasets',
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch BigQuery datasets', {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ error: 'Failed to fetch BigQuery datasets', details: errorData },
{ status: response.status }
)
}
const data = await response.json()
const datasets = (data.datasets || []).map(
(ds: {
datasetReference: { datasetId: string; projectId: string }
friendlyName?: string
}) => ({
datasetReference: ds.datasetReference,
friendlyName: ds.friendlyName,
})
)
const datasets = items.map((ds) => ({
datasetReference: ds.datasetReference,
friendlyName: ds.friendlyName,
}))
return NextResponse.json({ datasets })
} catch (error) {
if (error instanceof GooglePageError) {
logger.error('Failed to fetch BigQuery datasets', {
status: error.status,
error: error.body,
})
return NextResponse.json(
{ error: 'Failed to fetch BigQuery datasets', details: error.body },
{ status: error.status }
)
}
if (error instanceof ServiceAccountTokenError) {
return NextResponse.json({ error: error.message }, { status: 400 })
}
apps/sim/app/api/tools/google_bigquery/tables/route.ts
+50 -28
View File
@@ -5,6 +5,7 @@ import { parseRequest } from '@/lib/api/server'
import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { drainGooglePagedList, GooglePageError } from '@/lib/oauth/google-pagination'
import { getScopesForService } from '@/lib/oauth/utils'
import { refreshAccessTokenIfNeeded, ServiceAccountTokenError } from '@/app/api/auth/oauth/utils'
@@ -12,6 +13,19 @@ const logger = createLogger('GoogleBigQueryTablesAPI')
export const dynamic = 'force-dynamic'
const MAX_TABLE_PAGES = 20
const TABLE_PAGE_SIZE = 200
interface BigQueryTable {
tableReference: { tableId: string }
friendlyName?: string
}
interface BigQueryTablesResponse {
tables?: BigQueryTable[]
nextPageToken?: string
}
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
try {
@@ -68,38 +82,46 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch(
`https://bigquery.googleapis.com/bigquery/v2/projects/${encodeURIComponent(projectId)}/datasets/${encodeURIComponent(datasetId)}/tables?maxResults=200`,
{
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
}
)
const { items } = await drainGooglePagedList<BigQueryTable, BigQueryTablesResponse>({
buildUrl: (pageToken) => {
const url = new URL(
`https://bigquery.googleapis.com/bigquery/v2/projects/${encodeURIComponent(projectId)}/datasets/${encodeURIComponent(datasetId)}/tables`
)
url.searchParams.set('maxResults', String(TABLE_PAGE_SIZE))
if (pageToken) url.searchParams.set('pageToken', pageToken)
return url.toString()
},
fetch: (url) =>
fetch(url, {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
}),
parseError: (response) => response.json().catch(() => ({})),
getItems: (body) => body.tables,
getNextPageToken: (body) => body.nextPageToken,
maxPages: MAX_TABLE_PAGES,
label: 'BigQuery tables',
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch BigQuery tables', {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ error: 'Failed to fetch BigQuery tables', details: errorData },
{ status: response.status }
)
}
const data = await response.json()
const tables = (data.tables || []).map(
(t: { tableReference: { tableId: string }; friendlyName?: string }) => ({
tableReference: t.tableReference,
friendlyName: t.friendlyName,
})
)
const tables = items.map((t) => ({
tableReference: t.tableReference,
friendlyName: t.friendlyName,
}))
return NextResponse.json({ tables })
} catch (error) {
if (error instanceof GooglePageError) {
logger.error('Failed to fetch BigQuery tables', {
status: error.status,
error: error.body,
})
return NextResponse.json(
{ error: 'Failed to fetch BigQuery tables', details: error.body },
{ status: error.status }
)
}
if (error instanceof ServiceAccountTokenError) {
return NextResponse.json({ error: error.message }, { status: 400 })
}
apps/sim/app/api/tools/google_calendar/calendars/route.ts
+49 -25
View File
@@ -5,12 +5,16 @@ import { parseRequest } from '@/lib/api/server'
import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { drainGooglePagedList, GooglePageError } from '@/lib/oauth/google-pagination'
import { getScopesForService } from '@/lib/oauth/utils'
import { refreshAccessTokenIfNeeded, ServiceAccountTokenError } from '@/app/api/auth/oauth/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('GoogleCalendarAPI')
const MAX_CALENDAR_PAGES = 20
const CALENDAR_PAGE_SIZE = 250
interface CalendarListItem {
id: string
summary: string
@@ -21,6 +25,11 @@ interface CalendarListItem {
foregroundColor?: string
}
interface CalendarListResponse {
items?: CalendarListItem[]
nextPageToken?: string
}
/**
* Get calendars from Google Calendar
*/
@@ -64,35 +73,50 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
}
logger.info(`[${requestId}] Fetching calendars from Google Calendar API`)
const calendarResponse = await fetch(
'https://www.googleapis.com/calendar/v3/users/me/calendarList',
{
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
let calendars: CalendarListItem[]
try {
const drained = await drainGooglePagedList<CalendarListItem, CalendarListResponse>({
buildUrl: (pageToken) => {
const url = new URL('https://www.googleapis.com/calendar/v3/users/me/calendarList')
url.searchParams.set('maxResults', String(CALENDAR_PAGE_SIZE))
if (pageToken) url.searchParams.set('pageToken', pageToken)
return url.toString()
},
}
)
if (!calendarResponse.ok) {
const errorData = await calendarResponse
.text()
.then((text) => JSON.parse(text))
.catch(() => ({ error: { message: 'Unknown error' } }))
logger.error(`[${requestId}] Google Calendar API error`, {
status: calendarResponse.status,
error: errorData.error?.message || 'Failed to fetch calendars',
fetch: (url) =>
fetch(url, {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
}),
parseError: (response) =>
response
.text()
.then((text) => JSON.parse(text))
.catch(() => ({ error: { message: 'Unknown error' } })),
getItems: (body) => body.items,
getNextPageToken: (body) => body.nextPageToken,
maxPages: MAX_CALENDAR_PAGES,
label: 'Google Calendar calendars',
})
return NextResponse.json(
{ error: errorData.error?.message || 'Failed to fetch calendars' },
{ status: calendarResponse.status }
)
calendars = drained.items
} catch (error) {
if (error instanceof GooglePageError) {
const errorData = error.body as { error?: { message?: string } }
logger.error(`[${requestId}] Google Calendar API error`, {
status: error.status,
error: errorData?.error?.message || 'Failed to fetch calendars',
})
return NextResponse.json(
{ error: errorData?.error?.message || 'Failed to fetch calendars' },
{ status: error.status }
)
}
throw error
}
const data = await calendarResponse.json()
const calendars: CalendarListItem[] = data.items || []
calendars.sort((a, b) => {
if (a.primary && !b.primary) return -1
if (!a.primary && b.primary) return 1
apps/sim/app/api/tools/google_tasks/task-lists/route.ts
+43 -18
View File
@@ -5,6 +5,7 @@ import { parseRequest } from '@/lib/api/server'
import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { drainGooglePagedList, GooglePageError } from '@/lib/oauth/google-pagination'
import { getScopesForService } from '@/lib/oauth/utils'
import { refreshAccessTokenIfNeeded, ServiceAccountTokenError } from '@/app/api/auth/oauth/utils'
@@ -12,6 +13,19 @@ const logger = createLogger('GoogleTasksTaskListsAPI')
export const dynamic = 'force-dynamic'
const MAX_TASK_LIST_PAGES = 20
const TASK_LIST_PAGE_SIZE = 1000
interface GoogleTaskList {
id: string
title: string
}
interface GoogleTaskListsResponse {
items?: GoogleTaskList[]
nextPageToken?: string
}
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
try {
@@ -56,33 +70,44 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch('https://tasks.googleapis.com/tasks/v1/users/@me/lists', {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
const { items } = await drainGooglePagedList<GoogleTaskList, GoogleTaskListsResponse>({
buildUrl: (pageToken) => {
const url = new URL('https://tasks.googleapis.com/tasks/v1/users/@me/lists')
url.searchParams.set('maxResults', String(TASK_LIST_PAGE_SIZE))
if (pageToken) url.searchParams.set('pageToken', pageToken)
return url.toString()
},
fetch: (url) =>
fetch(url, {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
}),
parseError: (response) => response.json().catch(() => ({})),
getItems: (body) => body.items,
getNextPageToken: (body) => body.nextPageToken,
maxPages: MAX_TASK_LIST_PAGES,
label: 'Google Tasks task lists',
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch Google Tasks task lists', {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ error: 'Failed to fetch Google Tasks task lists', details: errorData },
{ status: response.status }
)
}
const data = await response.json()
const taskLists = (data.items || []).map((list: { id: string; title: string }) => ({
const taskLists = items.map((list) => ({
id: list.id,
title: list.title,
}))
return NextResponse.json({ taskLists })
} catch (error) {
if (error instanceof GooglePageError) {
logger.error('Failed to fetch Google Tasks task lists', {
status: error.status,
error: error.body,
})
return NextResponse.json(
{ error: 'Failed to fetch Google Tasks task lists', details: error.body },
{ status: error.status }
)
}
if (error instanceof ServiceAccountTokenError) {
return NextResponse.json({ error: error.message }, { status: 400 })
}
apps/sim/app/api/tools/jira/projects/route.ts
+86 -22
View File
@@ -14,6 +14,77 @@ export const dynamic = 'force-dynamic'
const logger = createLogger('JiraProjectsAPI')
const JIRA_PROJECTS_PAGE_SIZE = 50
const MAX_JIRA_PROJECTS_PAGES = 40
interface JiraProjectSearchPage {
values?: unknown[]
isLast?: boolean
maxResults?: number
}
/**
* Drains the offset-paginated Jira `/project/search` endpoint, advancing
* `startAt` by the server-returned page size until `isLast === true` (or a short
* page is seen). Bounded by `MAX_JIRA_PROJECTS_PAGES`; emits a `logger.warn` and
* returns the partial set rather than looping unbounded when the cap is hit.
*/
async function fetchAllJiraProjects(
apiUrl: string,
baseParams: URLSearchParams,
accessToken: string
): Promise<{ values: unknown[]; lastResponse: Response }> {
const values: unknown[] = []
let startAt = 0
let lastResponse: Response
for (let page = 0; page < MAX_JIRA_PROJECTS_PAGES; page++) {
const params = new URLSearchParams(baseParams)
params.set('startAt', String(startAt))
params.set('maxResults', String(JIRA_PROJECTS_PAGE_SIZE))
const finalUrl = `${apiUrl}?${params.toString()}`
logger.info(`Fetching Jira projects from: ${finalUrl}`)
const response = await fetch(finalUrl, {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
Accept: 'application/json',
},
})
logger.info(`Response status: ${response.status} ${response.statusText}`)
if (!response.ok) {
return { values, lastResponse: response }
}
const data = (await response.json()) as JiraProjectSearchPage
lastResponse = response
const pageValues = data.values ?? []
values.push(...pageValues)
const pageSize =
data.maxResults && data.maxResults > 0 ? data.maxResults : JIRA_PROJECTS_PAGE_SIZE
if (data.isLast === true || pageValues.length < pageSize) {
return { values, lastResponse }
}
startAt += pageValues.length
if (page === MAX_JIRA_PROJECTS_PAGES - 1) {
logger.warn('Jira project search hit pagination cap; project list may be incomplete', {
pages: MAX_JIRA_PROJECTS_PAGES,
collected: values.length,
})
}
}
return { values, lastResponse: lastResponse! }
}
export const GET = withRouteHandler(async (request: NextRequest) => {
try {
const auth = await checkSessionOrInternalAuth(request)
@@ -51,35 +122,28 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
queryParams.append('orderBy', 'name')
queryParams.append('expand', 'description,lead,url,projectKeys')
const finalUrl = `${apiUrl}?${queryParams.toString()}`
logger.info(`Fetching Jira projects from: ${finalUrl}`)
const { values, lastResponse } = await fetchAllJiraProjects(apiUrl, queryParams, accessToken)
const response = await fetch(finalUrl, {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
Accept: 'application/json',
},
})
logger.info(`Response status: ${response.status} ${response.statusText}`)
if (!response.ok) {
const errorText = await response.text()
logger.error('Jira API error:', { status: response.status, error: errorText })
if (!lastResponse.ok) {
const errorText = await lastResponse.text()
logger.error('Jira API error:', { status: lastResponse.status, error: errorText })
return NextResponse.json(
{ error: parseAtlassianErrorMessage(response.status, response.statusText, errorText) },
{ status: response.status }
{
error: parseAtlassianErrorMessage(
lastResponse.status,
lastResponse.statusText,
errorText
),
},
{ status: lastResponse.status }
)
}
const data = await response.json()
logger.info(`Jira API Response Status: ${response.status}`)
logger.info(`Found projects: ${data.values?.length || 0}`)
logger.info(`Jira API Response Status: ${lastResponse.status}`)
logger.info(`Found projects: ${values.length}`)
const projects =
data.values?.map((project: any) => ({
values.map((project: any) => ({
id: project.id,
key: project.key,
name: project.name,
apps/sim/app/api/tools/jsm/selector-requesttypes/route.ts
+81 -13
View File
@@ -14,6 +14,72 @@ const logger = createLogger('JsmSelectorRequestTypesAPI')
export const dynamic = 'force-dynamic'
const JSM_REQUEST_TYPES_PAGE_SIZE = 100
const MAX_JSM_REQUEST_TYPES_PAGES = 50
interface JsmPagedResponse<T> {
values?: T[]
isLastPage?: boolean
_links?: { next?: string }
}
interface JsmRequestTypeValue {
id: string
name: string
}
/**
* Drains the offset-paginated JSM `/servicedesk/{id}/requesttype` endpoint,
* advancing `start` by the number of rows actually returned until
* `isLastPage === true` (or `_links.next` is absent, or a page comes back
* empty). Advancing by the real row count — not the requested `limit` —
* prevents skipping items if the server returns a short non-final page. Bounded
* by `MAX_JSM_REQUEST_TYPES_PAGES`; emits a `logger.warn` and returns the
* partial set rather than looping unbounded when the cap is hit.
*/
async function fetchAllJsmRequestTypes(
requestTypeUrl: string,
accessToken: string
): Promise<{ values: JsmRequestTypeValue[]; lastResponse: Response }> {
const values: JsmRequestTypeValue[] = []
let start = 0
let lastResponse: Response
for (let page = 0; page < MAX_JSM_REQUEST_TYPES_PAGES; page++) {
const url = `${requestTypeUrl}?start=${start}&limit=${JSM_REQUEST_TYPES_PAGE_SIZE}`
const response = await fetch(url, {
method: 'GET',
headers: getJsmHeaders(accessToken),
})
if (!response.ok) {
return { values, lastResponse: response }
}
const data = (await response.json()) as JsmPagedResponse<JsmRequestTypeValue>
lastResponse = response
const pageValues = data.values ?? []
values.push(...pageValues)
if (data.isLastPage === true || !data._links?.next || pageValues.length === 0) {
return { values, lastResponse }
}
start += pageValues.length
if (page === MAX_JSM_REQUEST_TYPES_PAGES - 1) {
logger.warn('JSM request type list hit pagination cap; list may be incomplete', {
pages: MAX_JSM_REQUEST_TYPES_PAGES,
collected: values.length,
})
}
}
return { values, lastResponse: lastResponse! }
}
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
try {
@@ -72,28 +138,30 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
}
const baseUrl = getJsmApiBaseUrl(cloudIdValidation.sanitized!)
const url = `${baseUrl}/servicedesk/${serviceDeskIdValidation.sanitized}/requesttype?limit=100`
const requestTypeUrl = `${baseUrl}/servicedesk/${serviceDeskIdValidation.sanitized}/requesttype`
const response = await fetch(url, {
method: 'GET',
headers: getJsmHeaders(accessToken),
})
const { values, lastResponse } = await fetchAllJsmRequestTypes(requestTypeUrl, accessToken)
if (!response.ok) {
const errorText = await response.text()
if (!lastResponse.ok) {
const errorText = await lastResponse.text()
logger.error('JSM API error:', {
status: response.status,
statusText: response.statusText,
status: lastResponse.status,
statusText: lastResponse.statusText,
error: errorText,
})
return NextResponse.json(
{ error: parseAtlassianErrorMessage(response.status, response.statusText, errorText) },
{ status: response.status }
{
error: parseAtlassianErrorMessage(
lastResponse.status,
lastResponse.statusText,
errorText
),
},
{ status: lastResponse.status }
)
}
const data = await response.json()
const requestTypes = (data.values || []).map((rt: { id: string; name: string }) => ({
const requestTypes = values.map((rt) => ({
id: rt.id,
name: rt.name,
}))
apps/sim/app/api/tools/jsm/selector-servicedesks/route.ts
+80 -13
View File
@@ -14,6 +14,72 @@ const logger = createLogger('JsmSelectorServiceDesksAPI')
export const dynamic = 'force-dynamic'
const JSM_SERVICE_DESKS_PAGE_SIZE = 100
const MAX_JSM_SERVICE_DESKS_PAGES = 50
interface JsmPagedResponse<T> {
values?: T[]
isLastPage?: boolean
_links?: { next?: string }
}
interface JsmServiceDeskValue {
id: string
projectName: string
}
/**
* Drains the offset-paginated JSM `/servicedesk` endpoint, advancing `start` by
* the number of rows actually returned until `isLastPage === true` (or
* `_links.next` is absent, or a page comes back empty). Advancing by the real
* row count — not the requested `limit` — prevents skipping items if the server
* returns a short non-final page. Bounded by `MAX_JSM_SERVICE_DESKS_PAGES`;
* emits a `logger.warn` and returns the partial set rather than looping
* unbounded when the cap is hit.
*/
async function fetchAllJsmServiceDesks(
baseUrl: string,
accessToken: string
): Promise<{ values: JsmServiceDeskValue[]; lastResponse: Response }> {
const values: JsmServiceDeskValue[] = []
let start = 0
let lastResponse: Response
for (let page = 0; page < MAX_JSM_SERVICE_DESKS_PAGES; page++) {
const url = `${baseUrl}/servicedesk?start=${start}&limit=${JSM_SERVICE_DESKS_PAGE_SIZE}`
const response = await fetch(url, {
method: 'GET',
headers: getJsmHeaders(accessToken),
})
if (!response.ok) {
return { values, lastResponse: response }
}
const data = (await response.json()) as JsmPagedResponse<JsmServiceDeskValue>
lastResponse = response
const pageValues = data.values ?? []
values.push(...pageValues)
if (data.isLastPage === true || !data._links?.next || pageValues.length === 0) {
return { values, lastResponse }
}
start += pageValues.length
if (page === MAX_JSM_SERVICE_DESKS_PAGES - 1) {
logger.warn('JSM service desk list hit pagination cap; list may be incomplete', {
pages: MAX_JSM_SERVICE_DESKS_PAGES,
collected: values.length,
})
}
}
return { values, lastResponse: lastResponse! }
}
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
try {
@@ -63,28 +129,29 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
}
const baseUrl = getJsmApiBaseUrl(cloudIdValidation.sanitized!)
const url = `${baseUrl}/servicedesk?limit=100`
const response = await fetch(url, {
method: 'GET',
headers: getJsmHeaders(accessToken),
})
const { values, lastResponse } = await fetchAllJsmServiceDesks(baseUrl, accessToken)
if (!response.ok) {
const errorText = await response.text()
if (!lastResponse.ok) {
const errorText = await lastResponse.text()
logger.error('JSM API error:', {
status: response.status,
statusText: response.statusText,
status: lastResponse.status,
statusText: lastResponse.statusText,
error: errorText,
})
return NextResponse.json(
{ error: parseAtlassianErrorMessage(response.status, response.statusText, errorText) },
{ status: response.status }
{
error: parseAtlassianErrorMessage(
lastResponse.status,
lastResponse.statusText,
errorText
),
},
{ status: lastResponse.status }
)
}
const data = await response.json()
const serviceDesks = (data.values || []).map((sd: { id: string; projectName: string }) => ({
const serviceDesks = values.map((sd) => ({
id: sd.id,
name: sd.projectName,
}))
apps/sim/app/api/tools/linear/projects/route.ts
+47 -3
View File
@@ -1,4 +1,4 @@
import type { Project } from '@linear/sdk'
import type { Project, Team } from '@linear/sdk'
import { LinearClient } from '@linear/sdk'
import { createLogger } from '@sim/logger'
import { type NextRequest, NextResponse } from 'next/server'
@@ -13,6 +13,50 @@ export const dynamic = 'force-dynamic'
const logger = createLogger('LinearProjectsAPI')
/** Linear's maximum page size for a single connection request. */
const LINEAR_PAGE_SIZE = 250
/**
* Upper bound on pages to drain from a single team's projects connection. At
* 250 projects/page this covers 2,500 projects per team; the cap guards
* against runaway loops on a broken `hasNextPage` rather than a realistic
* limit.
*/
const MAX_PROJECTS_PAGES = 10
/**
* Drains a single team's projects connection by following
* `pageInfo.endCursor` until `hasNextPage` is false. Bounded by
* `MAX_PROJECTS_PAGES`; logs a warning if the cap is hit so a truncated list
* is visible rather than silently dropped.
*/
async function fetchAllTeamProjects(team: Team): Promise<Project[]> {
const projects: Project[] = []
let after: string | undefined
for (let page = 0; page < MAX_PROJECTS_PAGES; page++) {
const result = await team.projects({ first: LINEAR_PAGE_SIZE, after })
projects.push(...result.nodes)
if (!result.pageInfo.hasNextPage) {
return projects
}
after = result.pageInfo.endCursor ?? undefined
if (!after) {
return projects
}
if (page === MAX_PROJECTS_PAGES - 1) {
logger.warn('Linear projects pagination hit cap; project list may be incomplete', {
teamId: team.id,
cap: MAX_PROJECTS_PAGES,
fetched: projects.length,
})
}
}
return projects
}
export const POST = withRouteHandler(async (request: NextRequest) => {
try {
const parsed = await parseRequest(linearProjectsSelectorContract, request, {})
@@ -59,8 +103,8 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
const perTeam = await Promise.all(
teamIds.map(async (id) => {
const team = await linearClient.team(id)
const result = await team.projects()
return result.nodes.map((project: Project) => ({
const teamProjects = await fetchAllTeamProjects(team)
return teamProjects.map((project: Project) => ({
id: project.id,
name: project.name,
}))
apps/sim/app/api/tools/linear/teams/route.ts
+44 -2
View File
@@ -13,6 +13,48 @@ export const dynamic = 'force-dynamic'
const logger = createLogger('LinearTeamsAPI')
/** Linear's maximum page size for a single connection request. */
const LINEAR_PAGE_SIZE = 250
/**
* Upper bound on pages to drain from the teams connection. At 250 teams/page
* this covers 2,500 teams; the cap guards against runaway loops on a broken
* `hasNextPage` rather than a realistic limit.
*/
const MAX_TEAMS_PAGES = 10
/**
* Drains the full Linear teams connection by following
* `pageInfo.endCursor` until `hasNextPage` is false. Bounded by
* `MAX_TEAMS_PAGES`; logs a warning if the cap is hit so a truncated list is
* visible rather than silently dropped.
*/
async function fetchAllTeams(linearClient: LinearClient): Promise<Team[]> {
const teams: Team[] = []
let after: string | undefined
for (let page = 0; page < MAX_TEAMS_PAGES; page++) {
const result = await linearClient.teams({ first: LINEAR_PAGE_SIZE, after })
teams.push(...result.nodes)
if (!result.pageInfo.hasNextPage) {
return teams
}
after = result.pageInfo.endCursor ?? undefined
if (!after) {
return teams
}
if (page === MAX_TEAMS_PAGES - 1) {
logger.warn('Linear teams pagination hit cap; team list may be incomplete', {
cap: MAX_TEAMS_PAGES,
fetched: teams.length,
})
}
}
return teams
}
export const POST = withRouteHandler(async (request: NextRequest) => {
try {
const requestId = generateRequestId()
@@ -45,8 +87,8 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
}
const linearClient = new LinearClient({ accessToken })
const teamsResult = await linearClient.teams()
const teams = teamsResult.nodes.map((team: Team) => ({
const allTeams = await fetchAllTeams(linearClient)
const teams = allTeams.map((team: Team) => ({
id: team.id,
name: team.name,
}))
apps/sim/app/api/tools/microsoft-teams/channels/route.ts
+57 -24
View File
@@ -7,11 +7,25 @@ import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { validateMicrosoftGraphId } from '@/lib/core/security/input-validation'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('TeamsChannelsAPI')
/**
* Upper bound on Microsoft Graph pages drained when listing a team's channels.
* The `teams/{id}/channels` endpoint does not support `$top`, so paging is
* driven entirely by the server via `@odata.nextLink`. The cap prevents an
* unbounded loop; hitting it is logged as a warning.
*/
const MAX_CHANNELS_PAGES = 20
interface GraphChannel {
id: string
displayName?: string
}
export const POST = withRouteHandler(async (request: NextRequest) => {
try {
const parsed = await parseRequest(microsoftChannelsSelectorContract, request, {})
@@ -52,40 +66,59 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch(
`https://graph.microsoft.com/v1.0/teams/${encodeURIComponent(teamId)}/channels`,
{
const channels: GraphChannel[] = []
let nextPageUrl: string | undefined =
`https://graph.microsoft.com/v1.0/teams/${encodeURIComponent(teamId)}/channels`
for (let page = 0; page < MAX_CHANNELS_PAGES; page++) {
const response = await fetch(nextPageUrl, {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
}
)
if (!response.ok) {
const errorData = await response.json()
logger.error('Microsoft Graph API error getting channels', {
status: response.status,
error: errorData,
endpoint: `https://graph.microsoft.com/v1.0/teams/${teamId}/channels`,
})
if (response.status === 401) {
return NextResponse.json(
{
error: 'Authentication failed. Please reconnect your Microsoft Teams account.',
authRequired: true,
},
{ status: 401 }
)
if (!response.ok) {
const errorData = await response.json()
logger.error('Microsoft Graph API error getting channels', {
status: response.status,
error: errorData,
endpoint: nextPageUrl,
})
if (response.status === 401) {
return NextResponse.json(
{
error: 'Authentication failed. Please reconnect your Microsoft Teams account.',
authRequired: true,
},
{ status: 401 }
)
}
throw new Error(`Microsoft Graph API error: ${JSON.stringify(errorData)}`)
}
throw new Error(`Microsoft Graph API error: ${JSON.stringify(errorData)}`)
}
const data = await response.json()
if (Array.isArray(data.value)) {
channels.push(...(data.value as GraphChannel[]))
}
const data = await response.json()
const channels = data.value
const rawNextLink = getGraphNextPageUrl(data)
if (!rawNextLink) {
nextPageUrl = undefined
break
}
nextPageUrl = assertGraphNextPageUrl(rawNextLink)
if (page === MAX_CHANNELS_PAGES - 1) {
logger.warn(
'Hit Microsoft Graph channels pagination cap; channel list may be incomplete',
{ maxPages: MAX_CHANNELS_PAGES, collected: channels.length }
)
}
}
return NextResponse.json({
channels: channels,
apps/sim/app/api/tools/microsoft-teams/chats/route.ts
+66 -25
View File
@@ -7,11 +7,29 @@ import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { validateMicrosoftGraphId } from '@/lib/core/security/input-validation'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('TeamsChatsAPI')
/**
* Largest page size the `me/chats` Microsoft Graph endpoint permits via `$top`.
*/
const CHATS_PAGE_SIZE = 50
/**
* Upper bound on Microsoft Graph pages drained when listing the user's chats.
* Paging follows `@odata.nextLink`. The cap prevents an unbounded loop; hitting
* it is logged as a warning.
*/
const MAX_CHATS_PAGES = 20
interface GraphChat {
id: string
topic?: string
}
/**
* Helper function to get chat members and create a meaningful name
*
@@ -153,39 +171,62 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
return NextResponse.json({ error: 'Could not retrieve access token' }, { status: 401 })
}
const response = await fetch('https://graph.microsoft.com/v1.0/me/chats', {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
const rawChats: GraphChat[] = []
let nextPageUrl: string | undefined =
`https://graph.microsoft.com/v1.0/me/chats?$top=${CHATS_PAGE_SIZE}`
if (!response.ok) {
const errorData = await response.json()
logger.error('Microsoft Graph API error getting chats', {
status: response.status,
error: errorData,
endpoint: 'https://graph.microsoft.com/v1.0/me/chats',
for (let page = 0; page < MAX_CHATS_PAGES; page++) {
const response = await fetch(nextPageUrl, {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
if (response.status === 401) {
return NextResponse.json(
{
error: 'Authentication failed. Please reconnect your Microsoft Teams account.',
authRequired: true,
},
{ status: 401 }
)
if (!response.ok) {
const errorData = await response.json()
logger.error('Microsoft Graph API error getting chats', {
status: response.status,
error: errorData,
endpoint: nextPageUrl,
})
if (response.status === 401) {
return NextResponse.json(
{
error: 'Authentication failed. Please reconnect your Microsoft Teams account.',
authRequired: true,
},
{ status: 401 }
)
}
throw new Error(`Microsoft Graph API error: ${JSON.stringify(errorData)}`)
}
throw new Error(`Microsoft Graph API error: ${JSON.stringify(errorData)}`)
const data = await response.json()
if (Array.isArray(data.value)) {
rawChats.push(...(data.value as GraphChat[]))
}
const rawNextLink = getGraphNextPageUrl(data)
if (!rawNextLink) {
nextPageUrl = undefined
break
}
nextPageUrl = assertGraphNextPageUrl(rawNextLink)
if (page === MAX_CHATS_PAGES - 1) {
logger.warn('Hit Microsoft Graph chats pagination cap; chat list may be incomplete', {
maxPages: MAX_CHATS_PAGES,
collected: rawChats.length,
})
}
}
const data = await response.json()
const chats = await Promise.all(
data.value.map(async (chat: any) => ({
rawChats.map(async (chat) => ({
id: chat.id,
displayName: await getChatDisplayName(chat.id, accessToken, chat.topic),
}))
apps/sim/app/api/tools/microsoft-teams/teams/route.ts
+61 -26
View File
@@ -6,11 +6,25 @@ import { parseRequest } from '@/lib/api/server'
import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('TeamsTeamsAPI')
/**
* Upper bound on Microsoft Graph pages drained when listing the user's joined
* teams. The `me/joinedTeams` endpoint does not support `$top`, so paging is
* driven entirely by the server via `@odata.nextLink`. The cap prevents an
* unbounded loop; hitting it is logged as a warning.
*/
const MAX_TEAMS_PAGES = 20
interface GraphTeam {
id: string
displayName?: string
}
export const POST = withRouteHandler(async (request: NextRequest) => {
try {
const parsed = await parseRequest(microsoftTeamsSelectorContract, request, {})
@@ -46,38 +60,59 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch('https://graph.microsoft.com/v1.0/me/joinedTeams', {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
const teams: GraphTeam[] = []
let nextPageUrl: string | undefined = 'https://graph.microsoft.com/v1.0/me/joinedTeams'
if (!response.ok) {
const errorData = await response.json()
logger.error('Microsoft Graph API error getting teams', {
status: response.status,
error: errorData,
endpoint: 'https://graph.microsoft.com/v1.0/me/joinedTeams',
for (let page = 0; page < MAX_TEAMS_PAGES; page++) {
const response = await fetch(nextPageUrl, {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
// Check for auth errors specifically
if (response.status === 401) {
return NextResponse.json(
{
error: 'Authentication failed. Please reconnect your Microsoft Teams account.',
authRequired: true,
},
{ status: 401 }
)
if (!response.ok) {
const errorData = await response.json()
logger.error('Microsoft Graph API error getting teams', {
status: response.status,
error: errorData,
endpoint: nextPageUrl,
})
// Check for auth errors specifically
if (response.status === 401) {
return NextResponse.json(
{
error: 'Authentication failed. Please reconnect your Microsoft Teams account.',
authRequired: true,
},
{ status: 401 }
)
}
throw new Error(`Microsoft Graph API error: ${JSON.stringify(errorData)}`)
}
throw new Error(`Microsoft Graph API error: ${JSON.stringify(errorData)}`)
}
const data = await response.json()
if (Array.isArray(data.value)) {
teams.push(...(data.value as GraphTeam[]))
}
const data = await response.json()
const teams = data.value
const rawNextLink = getGraphNextPageUrl(data)
if (!rawNextLink) {
nextPageUrl = undefined
break
}
nextPageUrl = assertGraphNextPageUrl(rawNextLink)
if (page === MAX_TEAMS_PAGES - 1) {
logger.warn('Hit Microsoft Graph teams pagination cap; team list may be incomplete', {
maxPages: MAX_TEAMS_PAGES,
collected: teams.length,
})
}
}
return NextResponse.json({
teams: teams,
apps/sim/app/api/tools/microsoft_excel/drives/route.ts
+39 -15
View File
@@ -8,11 +8,19 @@ import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { extractGraphError, GRAPH_ID_PATTERN } from '@/tools/microsoft_excel/utils'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('MicrosoftExcelDrivesAPI')
/**
* Upper bound on Microsoft Graph pages drained when listing site drives.
* Each page returns up to `$top=999` drives, so this caps the result set at
* roughly 10k drives while preventing an unbounded server-side loop.
*/
const MAX_DRIVES_PAGES = 10
interface GraphDrive {
id: string
name: string
@@ -88,25 +96,41 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
}
// List all drives for the site
const url = `https://graph.microsoft.com/v1.0/sites/${siteId}/drives?$select=id,name,driveType,webUrl`
let nextUrl: string | undefined =
`https://graph.microsoft.com/v1.0/sites/${siteId}/drives?$select=id,name,driveType,webUrl&$top=999`
const response = await fetch(url, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
if (!response.ok) {
const errorMessage = await extractGraphError(response)
logger.error(`[${requestId}] Microsoft Graph API error fetching drives`, {
status: response.status,
error: errorMessage,
const rawDrives: GraphDrive[] = []
for (let page = 0; page < MAX_DRIVES_PAGES && nextUrl; page++) {
const response = await fetch(nextUrl, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
return NextResponse.json({ error: errorMessage }, { status: response.status })
if (!response.ok) {
const errorMessage = await extractGraphError(response)
logger.error(`[${requestId}] Microsoft Graph API error fetching drives`, {
status: response.status,
error: errorMessage,
})
return NextResponse.json({ error: errorMessage }, { status: response.status })
}
const data = await response.json()
if (Array.isArray(data.value)) {
rawDrives.push(...data.value)
}
const nextLink = getGraphNextPageUrl(data)
nextUrl = nextLink ? assertGraphNextPageUrl(nextLink) : undefined
if (nextUrl && page === MAX_DRIVES_PAGES - 1) {
logger.warn(
`[${requestId}] Site drives pagination hit ${MAX_DRIVES_PAGES}-page cap; result may be incomplete`
)
}
}
const data = await response.json()
const drives = (data.value || []).map((drive: GraphDrive) => ({
const drives = rawDrives.map((drive: GraphDrive) => ({
id: drive.id,
name: drive.name,
driveType: drive.driveType,
apps/sim/app/api/tools/microsoft_planner/plans/route.ts
+39 -16
View File
@@ -6,11 +6,19 @@ import { authorizeCredentialUse } from '@/lib/auth/credential-access'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
const logger = createLogger('MicrosoftPlannerPlansAPI')
export const dynamic = 'force-dynamic'
/**
* Upper bound on Microsoft Graph pages drained when listing Planner plans.
* Planner uses server-side paging (`$top` is generally ignored), so this caps
* the `@odata.nextLink` follow loop to prevent an unbounded drain.
*/
const MAX_PLANS_PAGES = 20
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
@@ -40,25 +48,40 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch('https://graph.microsoft.com/v1.0/me/planner/plans', {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
let nextUrl: string | undefined = 'https://graph.microsoft.com/v1.0/me/planner/plans'
if (!response.ok) {
const errorText = await response.text()
logger.error(`[${requestId}] Microsoft Graph API error:`, errorText)
return NextResponse.json(
{ error: 'Failed to fetch plans from Microsoft Graph' },
{ status: response.status }
)
const rawPlans: { id: string; title: string }[] = []
for (let page = 0; page < MAX_PLANS_PAGES && nextUrl; page++) {
const response = await fetch(nextUrl, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
if (!response.ok) {
const errorText = await response.text()
logger.error(`[${requestId}] Microsoft Graph API error:`, errorText)
return NextResponse.json(
{ error: 'Failed to fetch plans from Microsoft Graph' },
{ status: response.status }
)
}
const data = await response.json()
if (Array.isArray(data.value)) {
rawPlans.push(...data.value)
}
const nextLink = getGraphNextPageUrl(data)
nextUrl = nextLink ? assertGraphNextPageUrl(nextLink) : undefined
if (nextUrl && page === MAX_PLANS_PAGES - 1) {
logger.warn(
`[${requestId}] Planner plans pagination hit ${MAX_PLANS_PAGES}-page cap; result may be incomplete`
)
}
}
const data = await response.json()
const plans = data.value || []
const filteredPlans = plans.map((plan: { id: string; title: string }) => ({
const filteredPlans = rawPlans.map((plan: { id: string; title: string }) => ({
id: plan.id,
title: plan.title,
}))
apps/sim/app/api/tools/microsoft_planner/tasks/route.ts
+37 -16
View File
@@ -8,11 +8,19 @@ import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import type { PlannerTask } from '@/tools/microsoft_planner/types'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
const logger = createLogger('MicrosoftPlannerTasksAPI')
export const dynamic = 'force-dynamic'
/**
* Upper bound on Microsoft Graph pages drained when listing a plan's tasks.
* Planner uses server-side paging (`$top` is generally ignored), so this caps
* the `@odata.nextLink` follow loop to prevent an unbounded drain.
*/
const MAX_TASKS_PAGES = 20
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
@@ -48,28 +56,41 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch(
`https://graph.microsoft.com/v1.0/planner/plans/${planIdValidation.sanitized}/tasks`,
{
let nextUrl: string | undefined =
`https://graph.microsoft.com/v1.0/planner/plans/${planIdValidation.sanitized}/tasks`
const rawTasks: PlannerTask[] = []
for (let page = 0; page < MAX_TASKS_PAGES && nextUrl; page++) {
const response = await fetch(nextUrl, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
}
)
})
if (!response.ok) {
const errorText = await response.text()
logger.error(`[${requestId}] Microsoft Graph API error:`, errorText)
return NextResponse.json(
{ error: 'Failed to fetch tasks from Microsoft Graph' },
{ status: response.status }
)
if (!response.ok) {
const errorText = await response.text()
logger.error(`[${requestId}] Microsoft Graph API error:`, errorText)
return NextResponse.json(
{ error: 'Failed to fetch tasks from Microsoft Graph' },
{ status: response.status }
)
}
const data = await response.json()
if (Array.isArray(data.value)) {
rawTasks.push(...data.value)
}
const nextLink = getGraphNextPageUrl(data)
nextUrl = nextLink ? assertGraphNextPageUrl(nextLink) : undefined
if (nextUrl && page === MAX_TASKS_PAGES - 1) {
logger.warn(
`[${requestId}] Planner tasks pagination hit ${MAX_TASKS_PAGES}-page cap; result may be incomplete`
)
}
}
const data = await response.json()
const tasks = data.value || []
const filteredTasks = tasks.map((task: PlannerTask) => ({
const filteredTasks = rawTasks.map((task: PlannerTask) => ({
id: task.id,
title: task.title,
planId: task.planId,
apps/sim/app/api/tools/monday/boards/route.ts
+73 -28
View File
@@ -11,18 +11,29 @@ export const dynamic = 'force-dynamic'
const logger = createLogger('MondayBoardsAPI')
/**
* Monday's GraphQL `boards(limit: N, page: P, state: active)` has no cursor:
* `page` starts at 1 and you stop once a page returns fewer than `limit` items
* (or an empty page). We request the largest page (`MONDAY_BOARDS_LIMIT`) and
* bound the drain with `MAX_MONDAY_PAGES`.
*/
const MONDAY_BOARDS_LIMIT = 100
const MAX_MONDAY_PAGES = 50
interface MondayGraphQLError {
message?: string
}
interface MondayBoard {
id: string
name: string
}
interface MondayBoardsResponse {
errors?: MondayGraphQLError[]
error_message?: string
data?: {
boards?: Array<{
id: string
name: string
}>
boards?: MondayBoard[]
}
}
@@ -57,34 +68,68 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch('https://api.monday.com/v2', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: accessToken,
'API-Version': '2024-10',
},
body: JSON.stringify({
query: '{ boards(limit: 100, state: active) { id name } }',
}),
})
const allBoards: MondayBoard[] = []
let page = 1
const data = (await response.json()) as MondayBoardsResponse
for (; page <= MAX_MONDAY_PAGES; page++) {
const response = await fetch('https://api.monday.com/v2', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: accessToken,
'API-Version': '2024-10',
},
body: JSON.stringify({
query: `{ boards(limit: ${MONDAY_BOARDS_LIMIT}, page: ${page}, state: active) { id name } }`,
}),
})
if (data.errors?.length) {
logger.error('Monday.com API error', { errors: data.errors })
return NextResponse.json(
{ error: data.errors[0].message || 'Monday.com API error' },
{ status: 500 }
)
if (!response.ok) {
const details = await response.text().catch(() => '')
logger.error('Monday.com API HTTP error', {
status: response.status,
statusText: response.statusText,
details,
})
return NextResponse.json(
{ error: `Monday.com API error: ${response.status} ${response.statusText}` },
{ status: 500 }
)
}
const data = (await response.json()) as MondayBoardsResponse
if (data.errors?.length) {
logger.error('Monday.com API error', { errors: data.errors })
return NextResponse.json(
{ error: data.errors[0].message || 'Monday.com API error' },
{ status: 500 }
)
}
if (data.error_message) {
logger.error('Monday.com API error', { error_message: data.error_message })
return NextResponse.json({ error: data.error_message }, { status: 500 })
}
const pageBoards = data.data?.boards || []
allBoards.push(...pageBoards)
if (pageBoards.length < MONDAY_BOARDS_LIMIT) {
break
}
if (page === MAX_MONDAY_PAGES) {
logger.warn(
'Monday boards pagination hit MAX_MONDAY_PAGES cap; board list may be incomplete',
{
maxPages: MAX_MONDAY_PAGES,
}
)
}
}
if (data.error_message) {
logger.error('Monday.com API error', { error_message: data.error_message })
return NextResponse.json({ error: data.error_message }, { status: 500 })
}
const boards = (data.data?.boards || []).map((board) => ({
const boards = allBoards.map((board) => ({
id: board.id,
name: board.name,
}))
apps/sim/app/api/tools/notion/databases/route.ts
+57 -24
View File
@@ -1,4 +1,5 @@
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { type NextRequest, NextResponse } from 'next/server'
import { notionDatabasesSelectorContract } from '@/lib/api/contracts/selectors'
import { parseRequest } from '@/lib/api/server'
@@ -12,6 +13,16 @@ const logger = createLogger('NotionDatabasesAPI')
export const dynamic = 'force-dynamic'
const NOTION_PAGE_SIZE = 100
/**
* Notion's `POST /v1/search` returns at most `page_size` results per call and
* exposes `has_more`/`next_cursor` for pagination. This caps the number of
* pages drained so a tenant with a very large workspace cannot make this route
* loop unbounded. With `NOTION_PAGE_SIZE` of 100 this covers up to 2,000 items.
*/
const MAX_DATABASE_PAGES = 20
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
try {
@@ -43,33 +54,55 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch('https://api.notion.com/v1/search', {
method: 'POST',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
'Notion-Version': '2022-06-28',
},
body: JSON.stringify({
filter: { value: 'database', property: 'object' },
page_size: 100,
}),
})
const results: Record<string, unknown>[] = []
let startCursor: string | undefined
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch Notion databases', {
status: response.status,
error: errorData,
for (let page = 0; page < MAX_DATABASE_PAGES; page++) {
const response = await fetch('https://api.notion.com/v1/search', {
method: 'POST',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
'Notion-Version': '2022-06-28',
},
body: JSON.stringify({
filter: { value: 'database', property: 'object' },
page_size: NOTION_PAGE_SIZE,
...(startCursor ? { start_cursor: startCursor } : {}),
}),
})
return NextResponse.json(
{ error: 'Failed to fetch Notion databases', details: errorData },
{ status: response.status }
)
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch Notion databases', {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ error: 'Failed to fetch Notion databases', details: errorData },
{ status: response.status }
)
}
const data = await response.json()
if (Array.isArray(data.results)) {
results.push(...(data.results as Record<string, unknown>[]))
}
if (!data.has_more || !data.next_cursor) {
break
}
startCursor = data.next_cursor as string
if (page === MAX_DATABASE_PAGES - 1) {
logger.warn('Notion databases search hit pagination cap; results may be incomplete', {
maxPages: MAX_DATABASE_PAGES,
fetched: results.length,
})
}
}
const data = await response.json()
const databases = (data.results || []).map((db: Record<string, unknown>) => ({
const databases = results.map((db) => ({
id: db.id as string,
name: extractTitleFromItem(db),
}))
@@ -78,7 +111,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
} catch (error) {
logger.error('Error processing Notion databases request:', error)
return NextResponse.json(
{ error: 'Failed to retrieve Notion databases', details: (error as Error).message },
{ error: 'Failed to retrieve Notion databases', details: getErrorMessage(error) },
{ status: 500 }
)
}
apps/sim/app/api/tools/notion/pages/route.ts
+57 -24
View File
@@ -1,4 +1,5 @@
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { type NextRequest, NextResponse } from 'next/server'
import { notionPagesSelectorContract } from '@/lib/api/contracts/selectors'
import { parseRequest } from '@/lib/api/server'
@@ -12,6 +13,16 @@ const logger = createLogger('NotionPagesAPI')
export const dynamic = 'force-dynamic'
const NOTION_PAGE_SIZE = 100
/**
* Notion's `POST /v1/search` returns at most `page_size` results per call and
* exposes `has_more`/`next_cursor` for pagination. This caps the number of
* pages drained so a tenant with a very large workspace cannot make this route
* loop unbounded. With `NOTION_PAGE_SIZE` of 100 this covers up to 2,000 items.
*/
const MAX_NOTION_PAGES = 20
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
try {
@@ -43,33 +54,55 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch('https://api.notion.com/v1/search', {
method: 'POST',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
'Notion-Version': '2022-06-28',
},
body: JSON.stringify({
filter: { value: 'page', property: 'object' },
page_size: 100,
}),
})
const results: Record<string, unknown>[] = []
let startCursor: string | undefined
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch Notion pages', {
status: response.status,
error: errorData,
for (let page = 0; page < MAX_NOTION_PAGES; page++) {
const response = await fetch('https://api.notion.com/v1/search', {
method: 'POST',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
'Notion-Version': '2022-06-28',
},
body: JSON.stringify({
filter: { value: 'page', property: 'object' },
page_size: NOTION_PAGE_SIZE,
...(startCursor ? { start_cursor: startCursor } : {}),
}),
})
return NextResponse.json(
{ error: 'Failed to fetch Notion pages', details: errorData },
{ status: response.status }
)
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch Notion pages', {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ error: 'Failed to fetch Notion pages', details: errorData },
{ status: response.status }
)
}
const data = await response.json()
if (Array.isArray(data.results)) {
results.push(...(data.results as Record<string, unknown>[]))
}
if (!data.has_more || !data.next_cursor) {
break
}
startCursor = data.next_cursor as string
if (page === MAX_NOTION_PAGES - 1) {
logger.warn('Notion pages search hit pagination cap; results may be incomplete', {
maxPages: MAX_NOTION_PAGES,
fetched: results.length,
})
}
}
const data = await response.json()
const pages = (data.results || []).map((page: Record<string, unknown>) => ({
const pages = results.map((page) => ({
id: page.id as string,
name: extractTitleFromItem(page),
}))
@@ -78,7 +111,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
} catch (error) {
logger.error('Error processing Notion pages request:', error)
return NextResponse.json(
{ error: 'Failed to retrieve Notion pages', details: (error as Error).message },
{ error: 'Failed to retrieve Notion pages', details: getErrorMessage(error) },
{ status: 500 }
)
}
apps/sim/app/api/tools/onedrive/files/route.ts
+49 -20
View File
@@ -8,11 +8,21 @@ import { validateMicrosoftGraphId } from '@/lib/core/security/input-validation'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import type { MicrosoftGraphDriveItem } from '@/tools/onedrive/types'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('OneDriveFilesAPI')
/**
* Microsoft Graph paginates drive item collections via the `@odata.nextLink`
* absolute URL in the response body. Request the largest page (`$top` caps at
* 999) and drain following nextLink, bounded by a page cap.
* See https://learn.microsoft.com/en-us/graph/paging
*/
const ONEDRIVE_FILES_PAGE_SIZE = 999
const MAX_ONEDRIVE_FILES_PAGES = 20
/**
* Get files (not folders) from Microsoft OneDrive
*/
@@ -71,7 +81,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
'$select',
'id,name,file,webUrl,size,createdDateTime,lastModifiedDateTime,createdBy,thumbnails'
)
searchParams_new.append('$top', '50')
searchParams_new.append('$top', String(ONEDRIVE_FILES_PAGE_SIZE))
url = `https://graph.microsoft.com/v1.0/me/drive/root/search(q='${encodeURIComponent(query)}')?${searchParams_new.toString()}`
} else {
const searchParams_new = new URLSearchParams()
@@ -79,34 +89,53 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
'$select',
'id,name,file,folder,webUrl,size,createdDateTime,lastModifiedDateTime,createdBy,thumbnails'
)
searchParams_new.append('$top', '50')
searchParams_new.append('$top', String(ONEDRIVE_FILES_PAGE_SIZE))
url = `https://graph.microsoft.com/v1.0/me/drive/root/children?${searchParams_new.toString()}`
}
logger.info(`[${requestId}] Fetching files from Microsoft Graph`, { url })
const response = await fetch(url, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
const rawItems: MicrosoftGraphDriveItem[] = []
let nextUrl: string | undefined = url
if (!response.ok) {
const errorData = await response.json().catch(() => ({ error: { message: 'Unknown error' } }))
logger.error(`[${requestId}] Microsoft Graph API error`, {
status: response.status,
error: errorData.error?.message || 'Failed to fetch files from OneDrive',
for (let page = 0; page < MAX_ONEDRIVE_FILES_PAGES && nextUrl; page++) {
const response = await fetch(nextUrl, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
return NextResponse.json(
{ error: errorData.error?.message || 'Failed to fetch files from OneDrive' },
{ status: response.status }
)
if (!response.ok) {
const errorData = await response
.json()
.catch(() => ({ error: { message: 'Unknown error' } }))
logger.error(`[${requestId}] Microsoft Graph API error`, {
status: response.status,
error: errorData.error?.message || 'Failed to fetch files from OneDrive',
})
return NextResponse.json(
{ error: errorData.error?.message || 'Failed to fetch files from OneDrive' },
{ status: response.status }
)
}
const data = await response.json()
rawItems.push(...((data.value as MicrosoftGraphDriveItem[]) || []))
const nextLink = getGraphNextPageUrl(data)
nextUrl = nextLink ? assertGraphNextPageUrl(nextLink) : undefined
if (nextUrl && page === MAX_ONEDRIVE_FILES_PAGES - 1) {
logger.warn(`[${requestId}] OneDrive files hit pagination cap; list may be incomplete`, {
pages: MAX_ONEDRIVE_FILES_PAGES,
collected: rawItems.length,
})
}
}
const data = await response.json()
logger.info(`[${requestId}] Received ${data.value?.length || 0} items from Microsoft Graph`)
logger.info(`[${requestId}] Received ${rawItems.length} items from Microsoft Graph`)
const files = (data.value || [])
const files = rawItems
.filter((item: MicrosoftGraphDriveItem) => !!item.file && !item.folder)
.map((file: MicrosoftGraphDriveItem) => ({
id: file.id,
@@ -129,7 +158,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
}))
logger.info(`[${requestId}] Returning ${files.length} files`, {
totalItems: data.value?.length || 0,
totalItems: rawItems.length,
})
return NextResponse.json({ files }, { status: 200 })
apps/sim/app/api/tools/onedrive/folders/route.ts
+43 -14
View File
@@ -8,11 +8,21 @@ import { validateMicrosoftGraphId } from '@/lib/core/security/input-validation'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import type { MicrosoftGraphDriveItem } from '@/tools/onedrive/types'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('OneDriveFoldersAPI')
/**
* Microsoft Graph paginates drive item collections via the `@odata.nextLink`
* absolute URL in the response body. Request the largest page (`$top` caps at
* 999) and drain following nextLink, bounded by a page cap.
* See https://learn.microsoft.com/en-us/graph/paging
*/
const ONEDRIVE_FOLDERS_PAGE_SIZE = 999
const MAX_ONEDRIVE_FOLDERS_PAGES = 20
/**
* Get folders from Microsoft OneDrive
*/
@@ -60,28 +70,47 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
return NextResponse.json({ error: 'Failed to obtain valid access token' }, { status: 401 })
}
let url = `https://graph.microsoft.com/v1.0/me/drive/root/children?$filter=folder ne null&$select=id,name,folder,webUrl,createdDateTime,lastModifiedDateTime&$top=50`
let url = `https://graph.microsoft.com/v1.0/me/drive/root/children?$filter=folder ne null&$select=id,name,folder,webUrl,createdDateTime,lastModifiedDateTime&$top=${ONEDRIVE_FOLDERS_PAGE_SIZE}`
if (query) {
url += `&$search="${encodeURIComponent(query)}"`
}
const response = await fetch(url, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
const rawItems: MicrosoftGraphDriveItem[] = []
let nextUrl: string | undefined = url
if (!response.ok) {
const errorData = await response.json().catch(() => ({ error: { message: 'Unknown error' } }))
return NextResponse.json(
{ error: errorData.error?.message || 'Failed to fetch folders from OneDrive' },
{ status: response.status }
)
for (let page = 0; page < MAX_ONEDRIVE_FOLDERS_PAGES && nextUrl; page++) {
const response = await fetch(nextUrl, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
if (!response.ok) {
const errorData = await response
.json()
.catch(() => ({ error: { message: 'Unknown error' } }))
return NextResponse.json(
{ error: errorData.error?.message || 'Failed to fetch folders from OneDrive' },
{ status: response.status }
)
}
const data = await response.json()
rawItems.push(...((data.value as MicrosoftGraphDriveItem[]) || []))
const nextLink = getGraphNextPageUrl(data)
nextUrl = nextLink ? assertGraphNextPageUrl(nextLink) : undefined
if (nextUrl && page === MAX_ONEDRIVE_FOLDERS_PAGES - 1) {
logger.warn(`[${requestId}] OneDrive folders hit pagination cap; list may be incomplete`, {
pages: MAX_ONEDRIVE_FOLDERS_PAGES,
collected: rawItems.length,
})
}
}
const data = await response.json()
const folders = (data.value || [])
const folders = rawItems
.filter((item: MicrosoftGraphDriveItem) => item.folder)
.map((folder: MicrosoftGraphDriveItem) => ({
id: folder.id,
apps/sim/app/api/tools/onepassword/utils.test.ts
+108
View File
@@ -0,0 +1,108 @@
/**
* @vitest-environment node
*/
import { beforeEach, describe, expect, it, vi } from 'vitest'
const { mockDnsLookup, hostedFlag } = vi.hoisted(() => ({
mockDnsLookup: vi.fn(),
hostedFlag: { value: false },
}))
vi.mock('@/lib/core/config/feature-flags', () => ({
get isHosted() {
return hostedFlag.value
},
}))
vi.mock('dns/promises', () => ({
default: { lookup: mockDnsLookup },
}))
import { validateConnectServerUrl } from '@/app/api/tools/onepassword/utils'
describe('validateConnectServerUrl', () => {
beforeEach(() => {
vi.clearAllMocks()
hostedFlag.value = false
})
it('rejects a non-URL string', async () => {
await expect(validateConnectServerUrl('not a url')).rejects.toThrow('is not a valid URL')
})
describe('hosted deployment', () => {
beforeEach(() => {
hostedFlag.value = true
})
it.each([
['loopback', 'http://127.0.0.1:8080'],
['RFC1918 10.x', 'http://10.0.0.5'],
['RFC1918 192.168.x', 'http://192.168.1.1:8443'],
['RFC1918 172.16.x', 'http://172.16.0.9'],
['link-local metadata', 'http://169.254.169.254'],
['IPv4-mapped IPv6 private', 'http://[::ffff:10.0.0.1]'],
['IPv6 loopback', 'http://[::1]'],
])('blocks %s', async (_label, url) => {
await expect(validateConnectServerUrl(url)).rejects.toThrow(
'cannot point to a private or reserved IP address'
)
})
it('allows a public IP literal', async () => {
await expect(validateConnectServerUrl('https://8.8.8.8')).resolves.toBe('8.8.8.8')
})
it('blocks a hostname that resolves to a private IP', async () => {
mockDnsLookup.mockResolvedValue({ address: '10.1.2.3', family: 4 })
await expect(validateConnectServerUrl('https://connect.internal')).rejects.toThrow(
'cannot point to a private or reserved IP address'
)
})
it('allows a hostname that resolves to a public IP', async () => {
mockDnsLookup.mockResolvedValue({ address: '93.184.216.34', family: 4 })
await expect(validateConnectServerUrl('https://connect.example.com')).resolves.toBe(
'93.184.216.34'
)
})
})
describe('self-hosted deployment', () => {
beforeEach(() => {
hostedFlag.value = false
})
it.each([
['loopback', 'http://127.0.0.1:8080', '127.0.0.1'],
['RFC1918 10.x', 'http://10.0.0.5', '10.0.0.5'],
['RFC1918 192.168.x', 'http://192.168.1.1:8443', '192.168.1.1'],
])('allows %s (private Connect server)', async (_label, url, expected) => {
await expect(validateConnectServerUrl(url)).resolves.toBe(expected)
})
it('still blocks link-local metadata', async () => {
await expect(validateConnectServerUrl('http://169.254.169.254')).rejects.toThrow(
'cannot point to a link-local address'
)
})
it('still blocks IPv6 link-local', async () => {
await expect(validateConnectServerUrl('http://[fe80::1]')).rejects.toThrow(
'cannot point to a link-local address'
)
})
it('allows a hostname that resolves to a private IP', async () => {
mockDnsLookup.mockResolvedValue({ address: '10.1.2.3', family: 4 })
await expect(validateConnectServerUrl('https://connect.internal')).resolves.toBe('10.1.2.3')
})
})
it('rejects when DNS resolution fails', async () => {
mockDnsLookup.mockRejectedValue(new Error('ENOTFOUND'))
await expect(validateConnectServerUrl('https://nope.invalid')).rejects.toThrow(
'could not be resolved'
)
})
})
apps/sim/app/api/tools/onepassword/utils.ts
+48 -20
View File
@@ -12,7 +12,11 @@ import type {
import { createLogger } from '@sim/logger'
import { toError } from '@sim/utils/errors'
import * as ipaddr from 'ipaddr.js'
import { secureFetchWithPinnedIP } from '@/lib/core/security/input-validation.server'
import { isHosted } from '@/lib/core/config/feature-flags'
import {
isPrivateOrReservedIP,
secureFetchWithPinnedIP,
} from '@/lib/core/security/input-validation.server'
/** Connect-format field type strings returned by normalization. */
type ConnectFieldType =
@@ -246,12 +250,44 @@ export async function createOnePasswordClient(serviceAccountToken: string) {
const connectLogger = createLogger('OnePasswordConnect')
/**
* Validates that a Connect server URL does not target cloud metadata endpoints.
* Allows private IPs and localhost since 1Password Connect is designed to be self-hosted.
* Returns the resolved IP for DNS pinning to prevent TOCTOU rebinding.
* @throws Error if the URL is invalid, points to a link-local address, or DNS fails.
* Enforces the SSRF policy for a resolved Connect server IP.
*
* On the hosted service, all private and reserved IPs are blocked — a tenant has
* no legitimate reason to point Connect at the platform's internal network. On
* self-hosted deployments only link-local (cloud metadata) is blocked, since the
* operator controls both the workflows and the network and Connect servers
* legitimately live on private (RFC1918) addresses.
*
* @throws Error if the IP is not permitted under the active policy.
*/
async function validateConnectServerUrl(serverUrl: string): Promise<string> {
function assertConnectIpAllowed(ip: string, hostname: string): void {
if (isHosted) {
if (isPrivateOrReservedIP(ip)) {
connectLogger.warn('1Password Connect server URL resolves to a private or reserved IP', {
hostname,
resolvedIP: ip,
})
throw new Error('1Password server URL cannot point to a private or reserved IP address')
}
return
}
if (ipaddr.isValid(ip) && ipaddr.process(ip).range() === 'linkLocal') {
connectLogger.warn('1Password Connect server URL resolves to a link-local IP', {
hostname,
resolvedIP: ip,
})
throw new Error('1Password server URL cannot point to a link-local address')
}
}
/**
* Validates a Connect server URL against the SSRF policy and returns the resolved
* IP for DNS pinning to prevent TOCTOU rebinding. See {@link assertConnectIpAllowed}
* for the hosted vs. self-hosted policy.
* @throws Error if the URL is invalid, fails the IP policy, or DNS fails.
*/
export async function validateConnectServerUrl(serverUrl: string): Promise<string> {
let hostname: string
try {
hostname = new URL(serverUrl).hostname
@@ -263,31 +299,23 @@ async function validateConnectServerUrl(serverUrl: string): Promise<string> {
hostname.startsWith('[') && hostname.endsWith(']') ? hostname.slice(1, -1) : hostname
if (ipaddr.isValid(clean)) {
const addr = ipaddr.process(clean)
if (addr.range() === 'linkLocal') {
throw new Error('1Password server URL cannot point to a link-local address')
}
assertConnectIpAllowed(clean, clean)
return clean
}
let address: string
try {
const { address } = await dns.lookup(clean, { verbatim: true })
if (ipaddr.isValid(address) && ipaddr.process(address).range() === 'linkLocal') {
connectLogger.warn('1Password Connect server URL resolves to link-local IP', {
hostname: clean,
resolvedIP: address,
})
throw new Error('1Password server URL resolves to a link-local address')
}
return address
;({ address } = await dns.lookup(clean, { verbatim: true }))
} catch (error) {
if (error instanceof Error && error.message.startsWith('1Password')) throw error
connectLogger.warn('DNS lookup failed for 1Password Connect server URL', {
hostname: clean,
error: toError(error).message,
})
throw new Error('1Password server URL hostname could not be resolved')
}
assertConnectIpAllowed(address, clean)
return address
}
/** Minimal response shape used by all connectRequest callers. */
apps/sim/app/api/tools/outlook/folders/route.ts
+51 -25
View File
@@ -8,11 +8,21 @@ import { validateAlphanumericId } from '@/lib/core/security/input-validation'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('OutlookFoldersAPI')
/**
* Microsoft Graph paginates `mailFolders` via the `@odata.nextLink` absolute
* URL in the response body (default page size is ~10). Bound the drain so a
* pathological account can't loop unbounded; `$top` is capped at 999 by Graph.
* See https://learn.microsoft.com/en-us/graph/paging
*/
const OUTLOOK_FOLDERS_PAGE_SIZE = 999
const MAX_OUTLOOK_FOLDERS_PAGES = 20
interface OutlookFolder {
id: string
displayName: string
@@ -65,37 +75,53 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch('https://graph.microsoft.com/v1.0/me/mailFolders', {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
const folders: OutlookFolder[] = []
let nextUrl: string | undefined =
`https://graph.microsoft.com/v1.0/me/mailFolders?$top=${OUTLOOK_FOLDERS_PAGE_SIZE}`
if (!response.ok) {
const errorData = await response.json()
logger.error('Microsoft Graph API error getting folders', {
status: response.status,
error: errorData,
endpoint: 'https://graph.microsoft.com/v1.0/me/mailFolders',
for (let page = 0; page < MAX_OUTLOOK_FOLDERS_PAGES && nextUrl; page++) {
const response = await fetch(nextUrl, {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
if (response.status === 401) {
return NextResponse.json(
{
error: 'Authentication failed. Please reconnect your Outlook account.',
authRequired: true,
},
{ status: 401 }
)
if (!response.ok) {
const errorData = await response.json()
logger.error('Microsoft Graph API error getting folders', {
status: response.status,
error: errorData,
endpoint: nextUrl,
})
if (response.status === 401) {
return NextResponse.json(
{
error: 'Authentication failed. Please reconnect your Outlook account.',
authRequired: true,
},
{ status: 401 }
)
}
throw new Error(`Microsoft Graph API error: ${JSON.stringify(errorData)}`)
}
throw new Error(`Microsoft Graph API error: ${JSON.stringify(errorData)}`)
}
const data = await response.json()
folders.push(...((data.value as OutlookFolder[]) || []))
const data = await response.json()
const folders = data.value || []
const nextLink = getGraphNextPageUrl(data)
nextUrl = nextLink ? assertGraphNextPageUrl(nextLink) : undefined
if (nextUrl && page === MAX_OUTLOOK_FOLDERS_PAGES - 1) {
logger.warn('Outlook mailFolders hit pagination cap; folder list may be incomplete', {
pages: MAX_OUTLOOK_FOLDERS_PAGES,
collected: folders.length,
})
}
}
const transformedFolders = folders.map((folder: OutlookFolder) => ({
id: folder.id,
apps/sim/app/api/tools/pipedrive/pipelines/route.ts
+96 -19
View File
@@ -11,6 +11,86 @@ const logger = createLogger('PipedrivePipelinesAPI')
export const dynamic = 'force-dynamic'
const PIPEDRIVE_PAGE_LIMIT = 500
const PIPEDRIVE_MAX_PIPELINES_PAGES = 50
interface PipedrivePipeline {
id: number
name: string
}
interface PipedrivePipelinesPage {
data?: PipedrivePipeline[]
additional_data?: {
pagination?: {
more_items_in_collection?: boolean
next_start?: number
}
}
}
/**
* Lists all Pipedrive pipelines using v1 offset pagination (`start`/`limit`),
* following `additional_data.pagination.next_start` while
* `more_items_in_collection` is true so the full set is returned. Bounded by
* `PIPEDRIVE_MAX_PIPELINES_PAGES`; logs a warning rather than silently dropping
* pipelines when the cap is hit.
*/
async function fetchAllPipelines(accessToken: string): Promise<PipedrivePipeline[]> {
const pipelines: PipedrivePipeline[] = []
let start = 0
for (let page = 0; page < PIPEDRIVE_MAX_PIPELINES_PAGES; page++) {
const url = new URL('https://api.pipedrive.com/v1/pipelines')
url.searchParams.set('start', String(start))
url.searchParams.set('limit', String(PIPEDRIVE_PAGE_LIMIT))
const response = await fetch(url.toString(), {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
throw new PipedriveFetchError(response.status, errorData)
}
const data = (await response.json()) as PipedrivePipelinesPage
if (Array.isArray(data.data)) {
pipelines.push(...data.data)
}
const pagination = data.additional_data?.pagination
if (!pagination?.more_items_in_collection || typeof pagination.next_start !== 'number') {
return pipelines
}
start = pagination.next_start
if (page === PIPEDRIVE_MAX_PIPELINES_PAGES - 1) {
logger.warn(
'Pipedrive pipelines listing hit pagination cap; pipeline list may be incomplete',
{
pages: PIPEDRIVE_MAX_PIPELINES_PAGES,
}
)
}
}
return pipelines
}
class PipedriveFetchError extends Error {
constructor(
readonly status: number,
readonly details: unknown
) {
super('Failed to fetch Pipedrive pipelines')
this.name = 'PipedriveFetchError'
}
}
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
try {
@@ -42,27 +122,24 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch('https://api.pipedrive.com/v1/pipelines', {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch Pipedrive pipelines', {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ error: 'Failed to fetch Pipedrive pipelines', details: errorData },
{ status: response.status }
)
let allPipelines: PipedrivePipeline[]
try {
allPipelines = await fetchAllPipelines(accessToken)
} catch (error) {
if (error instanceof PipedriveFetchError) {
logger.error('Failed to fetch Pipedrive pipelines', {
status: error.status,
error: error.details,
})
return NextResponse.json(
{ error: 'Failed to fetch Pipedrive pipelines', details: error.details },
{ status: error.status }
)
}
throw error
}
const data = await response.json()
const pipelines = (data.data || []).map((pipeline: { id: number; name: string }) => ({
const pipelines = allPipelines.map((pipeline) => ({
id: String(pipeline.id),
name: pipeline.name,
}))
apps/sim/app/api/tools/sharepoint/lists/route.ts
+40 -14
View File
@@ -7,11 +7,19 @@ import { validateSharePointSiteId } from '@/lib/core/security/input-validation'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('SharePointListsAPI')
/**
* Upper bound on Microsoft Graph pages drained when listing SharePoint lists.
* Each page returns up to `$top=999` lists, so this caps the result set at
* roughly 10k lists while preventing an unbounded server-side loop.
*/
const MAX_LISTS_PAGES = 10
interface SharePointList {
id: string
displayName: string
@@ -60,24 +68,42 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const url = `https://graph.microsoft.com/v1.0/sites/${siteIdValidation.sanitized}/lists?$select=id,displayName,description,webUrl&$expand=list($select=hidden)&$top=100`
let nextUrl: string | undefined =
`https://graph.microsoft.com/v1.0/sites/${siteIdValidation.sanitized}/lists?$select=id,displayName,description,webUrl&$expand=list($select=hidden)&$top=999`
const response = await fetch(url, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
const rawLists: SharePointList[] = []
for (let page = 0; page < MAX_LISTS_PAGES && nextUrl; page++) {
const response = await fetch(nextUrl, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({ error: { message: 'Unknown error' } }))
return NextResponse.json(
{ error: errorData.error?.message || 'Failed to fetch lists from SharePoint' },
{ status: response.status }
)
if (!response.ok) {
const errorData = await response
.json()
.catch(() => ({ error: { message: 'Unknown error' } }))
return NextResponse.json(
{ error: errorData.error?.message || 'Failed to fetch lists from SharePoint' },
{ status: response.status }
)
}
const data = await response.json()
if (Array.isArray(data.value)) {
rawLists.push(...data.value)
}
const nextLink = getGraphNextPageUrl(data)
nextUrl = nextLink ? assertGraphNextPageUrl(nextLink) : undefined
if (nextUrl && page === MAX_LISTS_PAGES - 1) {
logger.warn(
`[${requestId}] SharePoint lists pagination hit ${MAX_LISTS_PAGES}-page cap; result may be incomplete`
)
}
}
const data = await response.json()
const lists = (data.value || [])
const lists = rawLists
.filter((list: SharePointList) => list.list?.hidden !== true)
.map((list: SharePointList) => ({
id: list.id,
apps/sim/app/api/tools/sharepoint/sites/route.ts
+40 -14
View File
@@ -7,11 +7,19 @@ import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { refreshAccessTokenIfNeeded } from '@/app/api/auth/oauth/utils'
import type { SharepointSite } from '@/tools/sharepoint/types'
import { assertGraphNextPageUrl, getGraphNextPageUrl } from '@/tools/sharepoint/utils'
export const dynamic = 'force-dynamic'
const logger = createLogger('SharePointSitesAPI')
/**
* Upper bound on Microsoft Graph pages drained when listing SharePoint sites.
* Each page returns up to `$top=999` sites, so this caps the result set at
* roughly 10k sites while preventing an unbounded server-side loop.
*/
const MAX_SITES_PAGES = 10
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
@@ -45,24 +53,42 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
}
const searchQuery = query || '*'
const url = `https://graph.microsoft.com/v1.0/sites?search=${encodeURIComponent(searchQuery)}&$select=id,name,displayName,webUrl,createdDateTime,lastModifiedDateTime&$top=50`
let nextUrl: string | undefined =
`https://graph.microsoft.com/v1.0/sites?search=${encodeURIComponent(searchQuery)}&$select=id,name,displayName,webUrl,createdDateTime,lastModifiedDateTime&$top=999`
const response = await fetch(url, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
const rawSites: SharepointSite[] = []
for (let page = 0; page < MAX_SITES_PAGES && nextUrl; page++) {
const response = await fetch(nextUrl, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({ error: { message: 'Unknown error' } }))
return NextResponse.json(
{ error: errorData.error?.message || 'Failed to fetch sites from SharePoint' },
{ status: response.status }
)
if (!response.ok) {
const errorData = await response
.json()
.catch(() => ({ error: { message: 'Unknown error' } }))
return NextResponse.json(
{ error: errorData.error?.message || 'Failed to fetch sites from SharePoint' },
{ status: response.status }
)
}
const data = await response.json()
if (Array.isArray(data.value)) {
rawSites.push(...data.value)
}
const nextLink = getGraphNextPageUrl(data)
nextUrl = nextLink ? assertGraphNextPageUrl(nextLink) : undefined
if (nextUrl && page === MAX_SITES_PAGES - 1) {
logger.warn(
`[${requestId}] SharePoint sites pagination hit ${MAX_SITES_PAGES}-page cap; result may be incomplete`
)
}
}
const data = await response.json()
const sites = (data.value || []).map((site: SharepointSite) => ({
const sites = rawSites.map((site: SharepointSite) => ({
id: site.id,
name: site.displayName || site.name,
mimeType: 'application/vnd.microsoft.graph.site',
apps/sim/app/api/tools/slack/users/route.ts
+56 -19
View File
@@ -12,6 +12,9 @@ export const dynamic = 'force-dynamic'
const logger = createLogger('SlackUsersAPI')
const SLACK_PAGE_LIMIT = 200
const SLACK_MAX_USER_PAGES = 10
interface SlackUser {
id: string
name: string
@@ -20,6 +23,11 @@ interface SlackUser {
is_bot: boolean
}
interface SlackUsersResult {
members: SlackUser[]
truncated: boolean
}
export const POST = withRouteHandler(async (request: NextRequest) => {
try {
const requestId = generateRequestId()
@@ -86,6 +94,9 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
}
const data = await fetchSlackUsers(accessToken)
if (data.truncated) {
logger.warn('users.list hit pagination cap; user list may be incomplete')
}
const users = (data.members || [])
.filter((user: SlackUser) => !user.deleted && !user.is_bot)
@@ -134,27 +145,53 @@ async function fetchSlackUser(accessToken: string, userId: string) {
return data
}
async function fetchSlackUsers(accessToken: string) {
const url = new URL('https://slack.com/api/users.list')
url.searchParams.append('limit', '200')
/**
* Lists Slack workspace members, following `response_metadata.next_cursor` so
* the full set is returned. Bounded by `SLACK_MAX_USER_PAGES`; sets `truncated`
* rather than silently dropping members when the cap is hit.
*/
async function fetchSlackUsers(accessToken: string): Promise<SlackUsersResult> {
const members: SlackUser[] = []
let cursor: string | undefined
let truncated = false
const response = await fetch(url.toString(), {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
for (let page = 0; page < SLACK_MAX_USER_PAGES; page++) {
const url = new URL('https://slack.com/api/users.list')
url.searchParams.append('limit', String(SLACK_PAGE_LIMIT))
if (cursor) {
url.searchParams.append('cursor', cursor)
}
if (!response.ok) {
throw new Error(`Slack API error: ${response.status} ${response.statusText}`)
const response = await fetch(url.toString(), {
method: 'GET',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
if (!response.ok) {
throw new Error(`Slack API error: ${response.status} ${response.statusText}`)
}
const data = await response.json()
if (!data.ok) {
throw new Error(data.error || 'Failed to fetch users')
}
if (Array.isArray(data.members)) {
members.push(...data.members)
}
cursor = data.response_metadata?.next_cursor?.trim() || undefined
if (!cursor) {
return { members, truncated }
}
if (page === SLACK_MAX_USER_PAGES - 1) {
truncated = true
}
}
const data = await response.json()
if (!data.ok) {
throw new Error(data.error || 'Failed to fetch users')
}
return data
return { members, truncated }
}
apps/sim/app/api/tools/wealthbox/items/route.ts
+80 -39
View File
@@ -12,6 +12,18 @@ export const dynamic = 'force-dynamic'
const logger = createLogger('WealthboxItemsAPI')
/**
* Wealthbox `GET /v1/contacts` paginates with `?page=` / `?per_page=`, starting
* at page 1. Wealthbox documents no `per_page` maximum and its contacts response
* carries no pagination `meta`, so termination relies on the short-page check:
* we stop once a page returns fewer items than `WEALTHBOX_PAGE_SIZE` (the
* `meta.total_pages` / `meta.current_page` check is a defensive fallback for if
* Wealthbox ever adds that block). Bounded by `MAX_WEALTHBOX_PAGES` so a runaway
* response can't loop forever.
*/
const WEALTHBOX_PAGE_SIZE = 50
const MAX_WEALTHBOX_PAGES = 50
interface WealthboxItem {
id: string
name: string
@@ -21,6 +33,15 @@ interface WealthboxItem {
updatedAt: string
}
interface WealthboxContactsPage {
contacts?: Array<Record<string, unknown>>
meta?: {
total_count?: number
total_pages?: number
current_page?: number
}
}
/**
* Get items (notes, contacts, tasks) from Wealthbox
*/
@@ -69,63 +90,83 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
}
const endpoint = endpoints[type as keyof typeof endpoints]
const url = new URL(`https://api.crmworkspace.com/v1/${endpoint}`)
logger.info(`[${requestId}] Fetching ${type}s from Wealthbox`, {
endpoint,
url: url.toString(),
hasQuery: !!query.trim(),
})
const response = await fetch(url.toString(), {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
const allContacts: Array<Record<string, unknown>> = []
let page = 1
if (!response.ok) {
const errorText = await response.text()
logger.error(
`[${requestId}] Wealthbox API error: ${response.status} ${response.statusText}`,
{
error: errorText,
endpoint,
url: url.toString(),
}
)
return NextResponse.json(
{ error: `Failed to fetch ${type}s from Wealthbox` },
{ status: response.status }
)
}
for (; page <= MAX_WEALTHBOX_PAGES; page++) {
const url = new URL(`https://api.crmworkspace.com/v1/${endpoint}`)
url.searchParams.set('per_page', String(WEALTHBOX_PAGE_SIZE))
url.searchParams.set('page', String(page))
const data = (await response.json()) as { contacts?: Array<Record<string, unknown>> } & Record<
string,
unknown
>
const response = await fetch(url.toString(), {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
})
logger.info(`[${requestId}] Wealthbox API raw response`, {
type,
status: response.status,
dataKeys: Object.keys(data || {}),
hasContacts: !!data.contacts,
dataStructure: typeof data === 'object' ? Object.keys(data) : 'not an object',
})
if (!response.ok) {
const errorText = await response.text()
logger.error(
`[${requestId}] Wealthbox API error: ${response.status} ${response.statusText}`,
{
error: errorText,
endpoint,
url: url.toString(),
}
)
return NextResponse.json(
{ error: `Failed to fetch ${type}s from Wealthbox` },
{ status: response.status }
)
}
let items: WealthboxItem[] = []
const data = (await response.json()) as WealthboxContactsPage
if (type === 'contact') {
const contacts = data.contacts || []
if (!Array.isArray(contacts)) {
logger.warn(`[${requestId}] Contacts is not an array`, {
contacts,
dataType: typeof contacts,
})
return NextResponse.json({ items: [] }, { status: 200 })
break
}
items = contacts.map((item) => {
allContacts.push(...contacts)
const totalPages = data.meta?.total_pages
const currentPage = data.meta?.current_page ?? page
const reachedLastByMeta =
typeof totalPages === 'number' && totalPages > 0 && currentPage >= totalPages
const reachedLastByCount = contacts.length < WEALTHBOX_PAGE_SIZE
if (reachedLastByMeta || reachedLastByCount) {
break
}
if (page === MAX_WEALTHBOX_PAGES) {
logger.warn(
`[${requestId}] Wealthbox pagination hit MAX_WEALTHBOX_PAGES cap; contact list may be incomplete`,
{ endpoint, maxPages: MAX_WEALTHBOX_PAGES }
)
}
}
logger.info(`[${requestId}] Wealthbox API drained`, {
type,
pagesFetched: Math.min(page, MAX_WEALTHBOX_PAGES),
totalContacts: allContacts.length,
})
let items: WealthboxItem[] = []
if (type === 'contact') {
items = allContacts.map((item) => {
const firstName = typeof item.first_name === 'string' ? item.first_name : ''
const lastName = typeof item.last_name === 'string' ? item.last_name : ''
return {
apps/sim/app/api/tools/webflow/items/route.ts
+86 -23
View File
@@ -12,6 +12,9 @@ const logger = createLogger('WebflowItemsAPI')
export const dynamic = 'force-dynamic'
const WEBFLOW_PAGE_LIMIT = 100
const WEBFLOW_MAX_ITEMS_PAGES = 50
interface WebflowItem {
id: string
fieldData?: {
@@ -21,6 +24,74 @@ interface WebflowItem {
}
}
interface WebflowItemsPage {
items?: WebflowItem[]
pagination?: {
total?: number
limit?: number
offset?: number
}
}
/**
* Lists all items in a Webflow collection using `offset`/`limit` pagination
* (limit capped at 100), advancing the numeric `offset` until the accumulated
* count reaches `pagination.total` so the full set is returned. Bounded by
* `WEBFLOW_MAX_ITEMS_PAGES`; logs a warning rather than silently dropping items
* when the cap is hit.
*/
async function fetchAllItems(accessToken: string, collectionId: string): Promise<WebflowItem[]> {
const items: WebflowItem[] = []
let offset = 0
for (let page = 0; page < WEBFLOW_MAX_ITEMS_PAGES; page++) {
const url = new URL(`https://api.webflow.com/v2/collections/${collectionId}/items`)
url.searchParams.set('limit', String(WEBFLOW_PAGE_LIMIT))
url.searchParams.set('offset', String(offset))
const response = await fetch(url.toString(), {
headers: {
Authorization: `Bearer ${accessToken}`,
accept: 'application/json',
},
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
throw new WebflowFetchError(response.status, errorData)
}
const data = (await response.json()) as WebflowItemsPage
const pageItems = data.items || []
items.push(...pageItems)
const total = data.pagination?.total
offset += pageItems.length
if (pageItems.length === 0 || (typeof total === 'number' && items.length >= total)) {
return items
}
if (page === WEBFLOW_MAX_ITEMS_PAGES - 1) {
logger.warn('Webflow items listing hit pagination cap; item list may be incomplete', {
collectionId,
pages: WEBFLOW_MAX_ITEMS_PAGES,
})
}
}
return items
}
class WebflowFetchError extends Error {
constructor(
readonly status: number,
readonly details: unknown
) {
super('Failed to fetch Webflow items')
this.name = 'WebflowFetchError'
}
}
export const POST = withRouteHandler(async (request: NextRequest) => {
try {
const requestId = generateRequestId()
@@ -61,32 +132,24 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch(
`https://api.webflow.com/v2/collections/${collectionId}/items?limit=100`,
{
headers: {
Authorization: `Bearer ${accessToken}`,
accept: 'application/json',
},
let items: WebflowItem[]
try {
items = await fetchAllItems(accessToken, collectionId)
} catch (error) {
if (error instanceof WebflowFetchError) {
logger.error('Failed to fetch Webflow items', {
status: error.status,
error: error.details,
collectionId,
})
return NextResponse.json(
{ error: 'Failed to fetch Webflow items', details: error.details },
{ status: error.status }
)
}
)
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch Webflow items', {
status: response.status,
error: errorData,
collectionId,
})
return NextResponse.json(
{ error: 'Failed to fetch Webflow items', details: errorData },
{ status: response.status }
)
throw error
}
const data = (await response.json()) as { items?: WebflowItem[] }
const items = data.items || []
let formattedItems = items.map((item) => {
const fieldData = item.fieldData || {}
const name = fieldData.name || fieldData.title || fieldData.slug || item.id
apps/sim/app/api/tools/zoom/meetings/route.ts
+62 -17
View File
@@ -11,6 +11,24 @@ const logger = createLogger('ZoomMeetingsAPI')
export const dynamic = 'force-dynamic'
/**
* Zoom `GET /v2/users/me/meetings` returns `next_page_token`, which is passed
* back as `?next_page_token=` until it comes back as an empty string. `page_size`
* max is 300. Bounded by `MAX_ZOOM_PAGES` so a runaway response can't loop forever.
*/
const ZOOM_PAGE_SIZE = 300
const MAX_ZOOM_PAGES = 50
interface ZoomMeeting {
id: number
topic: string
}
interface ZoomMeetingsPage {
meetings?: ZoomMeeting[]
next_page_token?: string
}
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
try {
@@ -42,30 +60,57 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
)
}
const response = await fetch(
'https://api.zoom.us/v2/users/me/meetings?page_size=300&type=scheduled',
{
const allMeetings: ZoomMeeting[] = []
let nextPageToken = ''
for (let page = 0; page < MAX_ZOOM_PAGES; page++) {
const url = new URL('https://api.zoom.us/v2/users/me/meetings')
url.searchParams.set('page_size', String(ZOOM_PAGE_SIZE))
url.searchParams.set('type', 'scheduled')
if (nextPageToken) {
url.searchParams.set('next_page_token', nextPageToken)
}
const response = await fetch(url.toString(), {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
}
)
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch Zoom meetings', {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ error: 'Failed to fetch Zoom meetings', details: errorData },
{ status: response.status }
)
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
logger.error('Failed to fetch Zoom meetings', {
status: response.status,
error: errorData,
})
return NextResponse.json(
{ error: 'Failed to fetch Zoom meetings', details: errorData },
{ status: response.status }
)
}
const data = (await response.json()) as ZoomMeetingsPage
if (Array.isArray(data.meetings)) {
allMeetings.push(...data.meetings)
}
nextPageToken = data.next_page_token?.trim() || ''
if (!nextPageToken) {
break
}
if (page === MAX_ZOOM_PAGES - 1) {
logger.warn(
'Zoom meetings pagination hit MAX_ZOOM_PAGES cap; meeting list may be incomplete',
{
maxPages: MAX_ZOOM_PAGES,
}
)
}
}
const data = await response.json()
const meetings = (data.meetings || []).map((meeting: { id: number; topic: string }) => ({
const meetings = allMeetings.map((meeting) => ({
id: String(meeting.id),
name: meeting.topic,
}))
apps/sim/app/api/v1/admin/organizations/[id]/members/[memberId]/route.ts
+8 -3
View File
@@ -35,6 +35,7 @@ import {
adminV1UpdateOrganizationMemberContract,
} from '@/lib/api/contracts/v1/admin'
import { parseRequest } from '@/lib/api/server'
import { getOrgMemberLedgerByUser } from '@/lib/billing/core/organization'
import { removeUserFromOrganization } from '@/lib/billing/organizations/membership'
import { isBillingEnabled } from '@/lib/core/config/feature-flags'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
@@ -86,7 +87,6 @@ export const GET = withRouteHandler(
userEmail: user.email,
currentPeriodCost: userStats.currentPeriodCost,
currentUsageLimit: userStats.currentUsageLimit,
lastActive: userStats.lastActive,
billingBlocked: userStats.billingBlocked,
})
.from(member)
@@ -99,6 +99,10 @@ export const GET = withRouteHandler(
return notFoundResponse('Member')
}
// currentPeriodCost is only a baseline; add this member's attributed
// usage_log for the org's period so admin shows real current usage.
const ledgerByUser = await getOrgMemberLedgerByUser(organizationId)
const data: AdminMemberDetail = {
id: memberData.id,
userId: memberData.userId,
@@ -107,9 +111,10 @@ export const GET = withRouteHandler(
createdAt: memberData.createdAt.toISOString(),
userName: memberData.userName,
userEmail: memberData.userEmail,
currentPeriodCost: memberData.currentPeriodCost ?? '0',
currentPeriodCost: (
Number(memberData.currentPeriodCost ?? 0) + (ledgerByUser.get(memberData.userId) ?? 0)
).toString(),
currentUsageLimit: memberData.currentUsageLimit,
lastActive: memberData.lastActive?.toISOString() ?? null,
billingBlocked: memberData.billingBlocked ?? false,
}
apps/sim/app/api/v1/admin/organizations/[id]/members/route.ts
+9 -3
View File
@@ -37,6 +37,7 @@ import {
adminV1ListOrganizationMembersContract,
} from '@/lib/api/contracts/v1/admin'
import { parseRequest } from '@/lib/api/server'
import { getOrgMemberLedgerByUser } from '@/lib/billing/core/organization'
import { addUserToOrganization } from '@/lib/billing/organizations/membership'
import { isBillingEnabled } from '@/lib/core/config/feature-flags'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
@@ -96,7 +97,6 @@ export const GET = withRouteHandler(
userEmail: user.email,
currentPeriodCost: userStats.currentPeriodCost,
currentUsageLimit: userStats.currentUsageLimit,
lastActive: userStats.lastActive,
billingBlocked: userStats.billingBlocked,
})
.from(member)
@@ -109,6 +109,11 @@ export const GET = withRouteHandler(
])
const total = countResult[0].count
// currentPeriodCost is only a baseline; add each member's attributed
// usage_log for the org's period so admin shows real current usage.
const usageByUser = await getOrgMemberLedgerByUser(organizationId)
const data: AdminMemberDetail[] = membersData.map((m) => ({
id: m.id,
userId: m.userId,
@@ -117,9 +122,10 @@ export const GET = withRouteHandler(
createdAt: m.createdAt.toISOString(),
userName: m.userName,
userEmail: m.userEmail,
currentPeriodCost: m.currentPeriodCost ?? '0',
currentPeriodCost: (
Number(m.currentPeriodCost ?? 0) + (usageByUser.get(m.userId) ?? 0)
).toString(),
currentUsageLimit: m.currentUsageLimit,
lastActive: m.lastActive?.toISOString() ?? null,
billingBlocked: m.billingBlocked ?? false,
}))
apps/sim/app/api/v1/admin/organizations/[id]/seats/route.ts
-10
View File
@@ -51,16 +51,6 @@ export const GET = withRouteHandler(
subscriptionPlan: analytics.subscriptionPlan,
canAddSeats: analytics.canAddSeats,
utilizationRate: analytics.utilizationRate,
activeMembers: analytics.activeMembers,
inactiveMembers: analytics.inactiveMembers,
memberActivity: analytics.memberActivity.map((m) => ({
userId: m.userId,
userName: m.userName,
userEmail: m.userEmail,
role: m.role,
joinedAt: m.joinedAt.toISOString(),
lastActive: m.lastActive?.toISOString() ?? null,
})),
}
logger.info(`Admin API: Retrieved seat analytics for organization ${organizationId}`)
apps/sim/app/api/v1/admin/types.ts
+1 -25
View File
@@ -543,7 +543,6 @@ export interface AdminMemberDetail extends AdminMember {
// Billing/usage info from userStats
currentPeriodCost: string
currentUsageLimit: string | null
lastActive: string | null
billingBlocked: boolean
}
@@ -574,28 +573,15 @@ interface AdminUserBilling {
userEmail: string
stripeCustomerId: string | null
// Usage stats
totalManualExecutions: number
totalApiCalls: number
totalWebhookTriggers: number
totalScheduledExecutions: number
totalChatExecutions: number
totalMcpExecutions: number
totalA2aExecutions: number
totalTokensUsed: number
totalCost: string
currentUsageLimit: string | null
currentPeriodCost: string
lastPeriodCost: string | null
billedOverageThisPeriod: string
storageUsedBytes: number
lastActive: string | null
billingBlocked: boolean
// Copilot usage
totalCopilotCost: string
// Copilot usage (active per-period baselines)
currentPeriodCopilotCost: string
lastPeriodCopilotCost: string | null
totalCopilotTokens: number
totalCopilotCalls: number
}
export interface AdminUserBillingWithSubscription extends AdminUserBilling {
@@ -643,16 +629,6 @@ export interface AdminSeatAnalytics {
subscriptionPlan: string
canAddSeats: boolean
utilizationRate: number
activeMembers: number
inactiveMembers: number
memberActivity: Array<{
userId: string
userName: string
userEmail: string
role: string
joinedAt: string
lastActive: string | null
}>
}
export interface AdminDeploymentVersion {
apps/sim/app/api/v1/admin/users/[id]/billing/route.ts
+7 -14
View File
@@ -29,6 +29,7 @@ import {
} from '@/lib/api/contracts/v1/admin'
import { parseRequest } from '@/lib/api/server'
import { getHighestPrioritySubscription } from '@/lib/billing/core/subscription'
import { getUserUsageData } from '@/lib/billing/core/usage'
import { isOrgScopedSubscription } from '@/lib/billing/subscriptions/utils'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { withAdminAuthParams } from '@/app/api/v1/admin/middleware'
@@ -78,6 +79,11 @@ export const GET = withRouteHandler(
const [stats] = await db.select().from(userStats).where(eq(userStats.userId, userId)).limit(1)
// currentPeriodCost is now only a baseline; canonical current-period usage
// (baseline + attributed usage_log, refresh-adjusted) comes from the same
// helper users see, so admin reflects real usage instead of a stale 0.
const usage = await getUserUsageData(userId)
const memberOrgs = await db
.select({
organizationId: member.organizationId,
@@ -107,27 +113,14 @@ export const GET = withRouteHandler(
userName: userData.name,
userEmail: userData.email,
stripeCustomerId: userData.stripeCustomerId,
totalManualExecutions: stats?.totalManualExecutions ?? 0,
totalApiCalls: stats?.totalApiCalls ?? 0,
totalWebhookTriggers: stats?.totalWebhookTriggers ?? 0,
totalScheduledExecutions: stats?.totalScheduledExecutions ?? 0,
totalChatExecutions: stats?.totalChatExecutions ?? 0,
totalMcpExecutions: stats?.totalMcpExecutions ?? 0,
totalA2aExecutions: stats?.totalA2aExecutions ?? 0,
totalTokensUsed: stats?.totalTokensUsed ?? 0,
totalCost: stats?.totalCost ?? '0',
currentUsageLimit: stats?.currentUsageLimit ?? null,
currentPeriodCost: stats?.currentPeriodCost ?? '0',
currentPeriodCost: usage.currentUsage.toString(),
lastPeriodCost: stats?.lastPeriodCost ?? null,
billedOverageThisPeriod: stats?.billedOverageThisPeriod ?? '0',
storageUsedBytes: stats?.storageUsedBytes ?? 0,
lastActive: stats?.lastActive?.toISOString() ?? null,
billingBlocked: stats?.billingBlocked ?? false,
totalCopilotCost: stats?.totalCopilotCost ?? '0',
currentPeriodCopilotCost: stats?.currentPeriodCopilotCost ?? '0',
lastPeriodCopilotCost: stats?.lastPeriodCopilotCost ?? null,
totalCopilotTokens: stats?.totalCopilotTokens ?? 0,
totalCopilotCalls: stats?.totalCopilotCalls ?? 0,
subscriptions: subscriptions.map(toAdminSubscription),
organizationMemberships: memberOrgs.map((m) => ({
organizationId: m.organizationId,
apps/sim/app/api/v1/copilot/chat/route.test.ts
+26
View File
@@ -0,0 +1,26 @@
/**
* Tests for the deprecated v1 copilot chat API route
*
* @vitest-environment node
*/
import { NextRequest } from 'next/server'
import { describe, expect, it } from 'vitest'
import { POST } from '@/app/api/v1/copilot/chat/route'
const URL = 'http://localhost:3000/api/v1/copilot/chat'
describe('Deprecated v1 copilot chat route', () => {
it('POST returns 410 with a success:false error body', async () => {
const request = new NextRequest(URL, {
method: 'POST',
headers: { 'Content-Type': 'application/json', 'x-api-key': 'sk-test' },
body: JSON.stringify({ message: 'hello' }),
})
const response = await POST(request)
expect(response.status).toBe(410)
const body = (await response.json()) as { success?: boolean; error?: string }
expect(body.success).toBe(false)
expect(body.error).toContain('deprecated')
})
})
apps/sim/app/api/v1/copilot/chat/route.ts
+12 -145
View File
@@ -1,151 +1,18 @@
import { createLogger } from '@sim/logger'
import { toError } from '@sim/utils/errors'
import { generateId } from '@sim/utils/id'
import { type NextRequest, NextResponse } from 'next/server'
import { v1CopilotChatContract } from '@/lib/api/contracts/v1/copilot'
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { runHeadlessCopilotLifecycle } from '@/lib/copilot/request/lifecycle/headless'
import { NextResponse } from 'next/server'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { getWorkflowById, resolveWorkflowIdForUser } from '@/lib/workflows/utils'
import { authenticateRequest } from '@/app/api/v1/middleware'
export const maxDuration = 3600
const logger = createLogger('CopilotHeadlessAPI')
const DEFAULT_COPILOT_MODEL = 'claude-opus-4-6'
/**
* POST /api/v1/copilot/chat
* Headless copilot endpoint for server-side orchestration.
*
* workflowId is optional - if not provided:
* - If workflowName is provided, finds that workflow
* - If exactly one workflow is available, uses that workflow as context
* - Otherwise requires workflowId or workflowName to disambiguate
* Deprecated: the v1 headless copilot chat API has been removed. The endpoint
* returns 410 Gone for all callers.
*/
export const POST = withRouteHandler(async (req: NextRequest) => {
let messageId: string | undefined
const authorized = await authenticateRequest(req, 'copilot-chat')
if (authorized instanceof NextResponse) {
return authorized
}
const { userId, rateLimit } = authorized
const auth = {
authenticated: true as const,
userId,
keyType: rateLimit.keyType,
workspaceId: rateLimit.workspaceId,
}
try {
const parsedRequest = await parseRequest(
v1CopilotChatContract,
req,
{},
{
validationErrorResponse: (error) =>
NextResponse.json(
{
success: false,
error: getValidationErrorMessage(error, 'Invalid request'),
details: error.issues,
},
{ status: 400 }
),
invalidJsonResponse: () =>
NextResponse.json({ success: false, error: 'Invalid request' }, { status: 400 }),
}
)
if (!parsedRequest.success) return parsedRequest.response
const parsed = parsedRequest.data.body
const selectedModel = parsed.model || DEFAULT_COPILOT_MODEL
// Resolve workflow ID
const resolved = await resolveWorkflowIdForUser(
auth.userId,
parsed.workflowId,
parsed.workflowName,
auth.keyType === 'workspace' ? auth.workspaceId : undefined
)
if (resolved.status !== 'resolved') {
return NextResponse.json(
{
success: false,
error: resolved.message,
},
{ status: 400 }
)
}
if (auth.keyType === 'workspace' && auth.workspaceId) {
const workflow = await getWorkflowById(resolved.workflowId)
if (!workflow?.workspaceId || workflow.workspaceId !== auth.workspaceId) {
return NextResponse.json(
{ success: false, error: 'API key is not authorized for this workspace' },
{ status: 403 }
)
}
}
// Transform mode to transport mode (same as client API)
// build and agent both map to 'agent' on the backend
const effectiveMode = parsed.mode === 'agent' ? 'build' : parsed.mode
const transportMode = effectiveMode === 'build' ? 'agent' : effectiveMode
// Always generate a chatId - required for artifacts system to work with subagents
const chatId = parsed.chatId || generateId()
messageId = generateId()
logger.info(
messageId
? `Received headless copilot chat start request [messageId:${messageId}]`
: 'Received headless copilot chat start request',
{
workflowId: resolved.workflowId,
workflowName: parsed.workflowName,
chatId,
mode: transportMode,
autoExecuteTools: parsed.autoExecuteTools,
timeout: parsed.timeout,
}
)
const requestPayload = {
message: parsed.message,
workflowId: resolved.workflowId,
userId: auth.userId,
model: selectedModel,
mode: transportMode,
messageId,
chatId,
}
const result = await runHeadlessCopilotLifecycle(requestPayload, {
userId: auth.userId,
workflowId: resolved.workflowId,
chatId,
goRoute: '/api/mcp',
autoExecuteTools: parsed.autoExecuteTools,
timeout: parsed.timeout,
interactive: false,
})
return NextResponse.json({
success: result.success,
content: result.content,
toolCalls: result.toolCalls,
chatId: result.chatId || chatId,
error: result.error,
})
} catch (error) {
logger.error(
messageId
? `Headless copilot request failed [messageId:${messageId}]`
: 'Headless copilot request failed',
{
error: toError(error).message,
}
)
return NextResponse.json({ success: false, error: 'Internal server error' }, { status: 500 })
}
})
export const POST = withRouteHandler(async () =>
NextResponse.json(
{
success: false,
error: 'The v1 copilot chat API has been deprecated and is no longer available.',
},
{ status: 410, headers: { 'Cache-Control': 'no-store' } }
)
)
apps/sim/app/api/v1/logs/[id]/route.ts
+11 -3
View File
@@ -7,6 +7,7 @@ import { type NextRequest, NextResponse } from 'next/server'
import { v1GetLogContract } from '@/lib/api/contracts/v1/logs'
import { parseRequest } from '@/lib/api/server'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { materializeExecutionData } from '@/lib/logs/execution/trace-store'
import { createApiResponse, getUserLimits } from '@/app/api/v1/logs/meta'
import {
checkRateLimit,
@@ -50,7 +51,7 @@ export const GET = withRouteHandler(
endedAt: workflowExecutionLogs.endedAt,
totalDurationMs: workflowExecutionLogs.totalDurationMs,
executionData: workflowExecutionLogs.executionData,
cost: workflowExecutionLogs.cost,
costTotal: workflowExecutionLogs.costTotal,
files: workflowExecutionLogs.files,
createdAt: workflowExecutionLogs.createdAt,
workflowName: workflow.name,
@@ -101,8 +102,15 @@ export const GET = withRouteHandler(
totalDurationMs: log.totalDurationMs,
files: log.files || undefined,
workflow: workflowSummary,
executionData: log.executionData as any,
cost: log.cost as any,
executionData: (await materializeExecutionData(
log.executionData as Record<string, unknown> | null,
{
workspaceId: log.workspaceId,
workflowId: log.workflowId,
executionId: log.executionId,
}
)) as any,
cost: log.costTotal != null ? { total: Number(log.costTotal) } : null,
createdAt: log.createdAt.toISOString(),
}
apps/sim/app/api/v1/logs/executions/[executionId]/route.ts
+3 -1
View File
@@ -70,7 +70,9 @@ export const GET = withRouteHandler(
startedAt: workflowLog.startedAt.toISOString(),
endedAt: workflowLog.endedAt?.toISOString(),
totalDurationMs: workflowLog.totalDurationMs,
cost: workflowLog.cost || null,
// Sourced from the cost_total projection of the usage_log ledger
// (the deprecated cost jsonb column was dropped).
cost: workflowLog.costTotal != null ? { total: Number(workflowLog.costTotal) } : null,
},
}
apps/sim/app/api/v1/logs/filters.ts
+6 -5
View File
@@ -84,18 +84,19 @@ export function buildLogFilters(filters: LogFilters): SQL<unknown> {
conditions.push(lte(workflowExecutionLogs.totalDurationMs, filters.maxDurationMs))
}
// Cost filters
// Cost filters — indexed projection of the usage_log ledger (dollars).
if (filters.minCost !== undefined) {
conditions.push(sql`(${workflowExecutionLogs.cost}->>'total')::numeric >= ${filters.minCost}`)
conditions.push(sql`${workflowExecutionLogs.costTotal} >= ${filters.minCost}`)
}
if (filters.maxCost !== undefined) {
conditions.push(sql`(${workflowExecutionLogs.cost}->>'total')::numeric <= ${filters.maxCost}`)
conditions.push(sql`${workflowExecutionLogs.costTotal} <= ${filters.maxCost}`)
}
// Model filter
// Model filter — uses the models_used projection (includes zero-cost/BYOK
// models, which the usage_log ledger drops), preserving prior behavior.
if (filters.model) {
conditions.push(sql`${workflowExecutionLogs.cost}->>'models' ? ${filters.model}`)
conditions.push(sql`${workflowExecutionLogs.modelsUsed} @> ARRAY[${filters.model}]::text[]`)
}
// Combine all conditions with AND
apps/sim/app/api/v1/logs/route.ts
+38 -18
View File
@@ -6,7 +6,9 @@ import { and, eq, sql } from 'drizzle-orm'
import { type NextRequest, NextResponse } from 'next/server'
import { v1ListLogsContract } from '@/lib/api/contracts/v1/logs'
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
import { MATERIALIZE_CONCURRENCY, mapWithConcurrency } from '@/lib/core/utils/concurrency'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { materializeExecutionData } from '@/lib/logs/execution/trace-store'
import { buildLogFilters, getOrderBy } from '@/app/api/v1/logs/filters'
import { createApiResponse, getUserLimits } from '@/app/api/v1/logs/meta'
import {
@@ -103,6 +105,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
.select({
id: workflowExecutionLogs.id,
workflowId: workflowExecutionLogs.workflowId,
workspaceId: workflowExecutionLogs.workspaceId,
executionId: workflowExecutionLogs.executionId,
deploymentVersionId: workflowExecutionLogs.deploymentVersionId,
level: workflowExecutionLogs.level,
@@ -110,7 +113,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
startedAt: workflowExecutionLogs.startedAt,
endedAt: workflowExecutionLogs.endedAt,
totalDurationMs: workflowExecutionLogs.totalDurationMs,
cost: workflowExecutionLogs.cost,
costTotal: workflowExecutionLogs.costTotal,
files: workflowExecutionLogs.files,
executionData: params.details === 'full' ? workflowExecutionLogs.executionData : sql`null`,
workflowName: workflow.name,
@@ -144,7 +147,12 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
})
}
const formattedLogs = data.map((log) => {
// Only materialize externalized execution data when the response actually
// needs it (details=full + finalOutput/traceSpans requested).
const needsMaterialize =
params.details === 'full' && (params.includeFinalOutput || params.includeTraceSpans)
const buildBase = (log: (typeof data)[number]) => {
const result: any = {
id: log.id,
workflowId: log.workflowId,
@@ -155,7 +163,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
startedAt: log.startedAt.toISOString(),
endedAt: log.endedAt?.toISOString() || null,
totalDurationMs: log.totalDurationMs,
cost: log.cost ? { total: (log.cost as any).total } : null,
cost: log.costTotal != null ? { total: Number(log.costTotal) } : null,
files: log.files || null,
}
@@ -166,24 +174,36 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
description: log.workflowDescription,
deleted: !log.workflowName,
}
if (log.cost) {
result.cost = log.cost
}
if (log.executionData) {
const execData = log.executionData as any
if (params.includeFinalOutput && execData.finalOutput) {
result.finalOutput = execData.finalOutput
}
if (params.includeTraceSpans && execData.traceSpans) {
result.traceSpans = execData.traceSpans
}
}
}
return result
})
}
// Only run the bounded-concurrency materialization when the response actually
// needs object-storage reads; otherwise a plain synchronous map avoids the
// per-row worker/promise overhead.
const formattedLogs = needsMaterialize
? await mapWithConcurrency(data, MATERIALIZE_CONCURRENCY, async (log) => {
const result = buildBase(log)
if (log.executionData) {
const execData = (await materializeExecutionData(
log.executionData as Record<string, unknown> | null,
{
workspaceId: log.workspaceId,
workflowId: log.workflowId,
executionId: log.executionId,
}
)) as any
if (params.includeFinalOutput && execData.finalOutput) {
result.finalOutput = execData.finalOutput
}
if (params.includeTraceSpans && execData.traceSpans) {
result.traceSpans = execData.traceSpans
}
}
return result
})
: data.map(buildBase)
const limits = await getUserLimits(userId)
apps/sim/app/api/webhooks/route.ts
+27 -14
View File
@@ -27,7 +27,10 @@ import {
} from '@/lib/webhooks/provider-subscriptions'
import { getProviderHandler } from '@/lib/webhooks/providers'
import { mergeNonUserFields } from '@/lib/webhooks/utils'
import { syncWebhooksForCredentialSet } from '@/lib/webhooks/utils.server'
import {
findConflictingWebhookPathOwner,
syncWebhooksForCredentialSet,
} from '@/lib/webhooks/utils.server'
import { extractCredentialSetId, isCredentialSetValue } from '@/executor/constants'
const logger = createLogger('WebhooksAPI')
@@ -330,21 +333,31 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
}
}
if (!targetWebhookId) {
const existingByPath = await db
.select({ id: webhook.id, workflowId: webhook.workflowId })
const conflictingOwner = await findConflictingWebhookPathOwner({
path: finalPath,
workflowId,
})
if (conflictingOwner) {
logger.warn(`[${requestId}] Webhook path conflict: ${finalPath}`)
return NextResponse.json(
{ error: 'Webhook path already exists.', code: 'PATH_EXISTS' },
{ status: 409 }
)
}
const ownExisting = await db
.select({ id: webhook.id })
.from(webhook)
.where(and(eq(webhook.path, finalPath), isNull(webhook.archivedAt)))
.limit(1)
if (existingByPath.length > 0) {
// If a webhook with the same path exists but belongs to a different workflow, return an error
if (existingByPath[0].workflowId !== workflowId) {
logger.warn(`[${requestId}] Webhook path conflict: ${finalPath}`)
return NextResponse.json(
{ error: 'Webhook path already exists.', code: 'PATH_EXISTS' },
{ status: 409 }
.where(
and(
eq(webhook.path, finalPath),
eq(webhook.workflowId, workflowId),
isNull(webhook.archivedAt)
)
}
targetWebhookId = existingByPath[0].id
)
.limit(1)
if (ownExisting.length > 0) {
targetWebhookId = ownExisting[0].id
}
}
apps/sim/app/api/workflows/[id]/executions/[executionId]/route.ts
+15 -6
View File
@@ -9,6 +9,7 @@ import {
} from '@/lib/api/contracts/workflows'
import { parseRequest } from '@/lib/api/server'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { materializeExecutionData } from '@/lib/logs/execution/trace-store'
import { validateWorkflowAccess } from '@/app/api/workflows/middleware'
import type { PausePoint } from '@/executor/types'
@@ -117,6 +118,7 @@ export const GET = withRouteHandler(
.select({
executionId: workflowExecutionLogs.executionId,
workflowId: workflowExecutionLogs.workflowId,
workspaceId: workflowExecutionLogs.workspaceId,
status: workflowExecutionLogs.status,
level: workflowExecutionLogs.level,
trigger: workflowExecutionLogs.trigger,
@@ -124,7 +126,7 @@ export const GET = withRouteHandler(
endedAt: workflowExecutionLogs.endedAt,
totalDurationMs: workflowExecutionLogs.totalDurationMs,
executionData: workflowExecutionLogs.executionData,
cost: workflowExecutionLogs.cost,
costTotal: workflowExecutionLogs.costTotal,
})
.from(workflowExecutionLogs)
.where(
@@ -177,13 +179,20 @@ export const GET = withRouteHandler(
}
}
const cost = logRow.cost
? { total: Number((logRow.cost as { total?: number }).total ?? 0) }
: null
const cost = logRow.costTotal != null ? { total: Number(logRow.costTotal) } : null
const error = status === 'failed' ? extractError(logRow.executionData) : null
// Heavy execution data may live in object storage; resolve the pointer
// before reading error / finalOutput / traceSpans (no-op for inline rows).
const executionData = (await materializeExecutionData(
logRow.executionData as Record<string, unknown> | null,
{
workspaceId: logRow.workspaceId,
workflowId: logRow.workflowId,
executionId: logRow.executionId,
}
)) as ExecutionDataShape | undefined
const executionData = logRow.executionData as ExecutionDataShape | undefined
const error = status === 'failed' ? extractError(executionData) : null
const finalOutput =
includeOutput && status === 'completed' && executionData
apps/sim/app/workspace/[workspaceId]/components/resource/components/resource-options-bar/resource-options-bar.tsx
+75 -15
View File
@@ -1,4 +1,4 @@
import { memo, type ReactNode } from 'react'
import { memo, type ReactNode, useState } from 'react'
import * as PopoverPrimitive from '@radix-ui/react-popover'
import {
ArrowDown,
@@ -14,12 +14,15 @@ import {
Search,
X,
} from '@/components/emcn'
import { POPOVER_ANIMATION_CLASSES } from '@/components/emcn/components/chip-date-picker/chip-date-picker'
import { cn } from '@/lib/core/utils/cn'
const SEARCH_ICON = (
<Search className='pointer-events-none size-[14px] shrink-0 text-[var(--text-icon)]' />
)
const RESOURCE_MENU_EDGE_OFFSET = 6
type SortDirection = 'asc' | 'desc'
export interface ColumnOption {
@@ -73,6 +76,10 @@ interface ResourceOptionsBarProps {
filterActive?: boolean
filterTags?: FilterTag[]
extras?: ReactNode
/** Right-aligned slot. Unlike `extras` (which sits with the left controls),
* `trailing` is pushed to the far right via `justify-between` — used for the
* table's run/stop control opposite the left-aligned filter/sort. */
trailing?: ReactNode
}
export const ResourceOptionsBar = memo(function ResourceOptionsBar({
@@ -83,9 +90,24 @@ export const ResourceOptionsBar = memo(function ResourceOptionsBar({
filterActive,
filterTags,
extras,
trailing,
}: ResourceOptionsBarProps) {
/**
* Coordinates the Filter popover and Sort menu as a single menu bar: clicking
* one while the other is open switches to it in a single click. Functional
* updates make the close→open ordering race-proof, so whichever menu the click
* targets wins regardless of which `onOpenChange` fires first.
*/
const [openMenu, setOpenMenu] = useState<'filter' | 'sort' | null>(null)
const hasContent =
search || sort || filter || onFilterToggle || extras || (filterTags && filterTags.length > 0)
search ||
sort ||
filter ||
onFilterToggle ||
extras ||
trailing ||
(filterTags && filterTags.length > 0)
if (!hasContent) return null
return (
@@ -123,26 +145,52 @@ export const ResourceOptionsBar = memo(function ResourceOptionsBar({
Filter
</Button>
) : filter ? (
<PopoverPrimitive.Root>
<PopoverPrimitive.Trigger asChild>
<Button variant='subtle' className='px-2 py-1 text-caption'>
<ListFilter className='mr-1.5 size-[14px] text-[var(--text-icon)]' />
Filter
</Button>
</PopoverPrimitive.Trigger>
<PopoverPrimitive.Root
open={openMenu === 'filter'}
onOpenChange={(open) =>
setOpenMenu((current) => (open ? 'filter' : current === 'filter' ? null : current))
}
>
<PopoverPrimitive.Anchor asChild>
<div className='flex items-center gap-1.5'>
<PopoverPrimitive.Trigger asChild>
<Button variant='subtle' className='px-2 py-1 text-caption'>
<ListFilter className='mr-1.5 size-[14px] text-[var(--text-icon)]' />
Filter
</Button>
</PopoverPrimitive.Trigger>
{sort && (
<SortDropdown
config={sort}
open={openMenu === 'sort'}
onOpenChange={(open) =>
setOpenMenu((current) =>
open ? 'sort' : current === 'sort' ? null : current
)
}
/>
)}
</div>
</PopoverPrimitive.Anchor>
<PopoverPrimitive.Portal>
<PopoverPrimitive.Content
align='start'
align='end'
alignOffset={RESOURCE_MENU_EDGE_OFFSET}
collisionPadding={6}
sideOffset={6}
className='z-50 w-fit rounded-lg border border-[var(--border)] bg-[var(--bg)] shadow-sm'
className={cn(
POPOVER_ANIMATION_CLASSES,
'z-50 w-fit origin-[--radix-popover-content-transform-origin] rounded-xl border border-[var(--border)] bg-[var(--bg)] shadow-sm'
)}
>
{filter}
</PopoverPrimitive.Content>
</PopoverPrimitive.Portal>
</PopoverPrimitive.Root>
) : null}
{sort && <SortDropdown config={sort} />}
{sort && (onFilterToggle || !filter) && <SortDropdown config={sort} />}
</div>
{trailing && <div className='flex shrink-0 items-center gap-1.5'>{trailing}</div>}
</div>
</div>
)
@@ -200,11 +248,19 @@ const SearchSection = memo(function SearchSection({ search }: { search: SearchCo
)
})
const SortDropdown = memo(function SortDropdown({ config }: { config: SortConfig }) {
interface SortDropdownProps {
config: SortConfig
/** Controlled open state — omit for standalone (uncontrolled) usage. */
open?: boolean
/** Controlled open-change handler, paired with {@link SortDropdownProps.open}. */
onOpenChange?: (open: boolean) => void
}
const SortDropdown = memo(function SortDropdown({ config, open, onOpenChange }: SortDropdownProps) {
const { options, active, onSort, onClear } = config
return (
<DropdownMenu>
<DropdownMenu modal={false} open={open} onOpenChange={onOpenChange}>
<DropdownMenuTrigger asChild>
<Button
variant='subtle'
@@ -222,7 +278,11 @@ const SortDropdown = memo(function SortDropdown({ config }: { config: SortConfig
Sort
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align='start'>
<DropdownMenuContent
align='end'
alignOffset={RESOURCE_MENU_EDGE_OFFSET}
className='max-h-[var(--radix-dropdown-menu-content-available-height,400px)]'
>
{options.map((option) => {
const isActive = active?.column === option.id
const Icon = option.icon
apps/sim/app/workspace/[workspaceId]/knowledge/[id]/base.tsx
+237 -174
View File
@@ -1,6 +1,6 @@
'use client'
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
import { type ReactNode, useCallback, useEffect, useMemo, useRef, useState } from 'react'
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { generateId } from '@sim/utils/id'
@@ -11,11 +11,10 @@ import { usePostHog } from 'posthog-js/react'
import {
Badge,
Button,
Combobox,
type ComboboxOption,
DatePicker,
ChipDatePicker,
ChipDropdown,
type ChipDropdownOption,
Input,
Label,
Loader,
Modal,
ModalBody,
@@ -95,6 +94,14 @@ const DOCUMENT_COLUMNS: ResourceColumn[] = [
{ id: 'tags', header: 'Tags' },
]
const STATUS_FILTER_OPTIONS: ChipDropdownOption[] = [
{ value: 'all', label: 'All' },
{ value: 'enabled', label: 'Enabled' },
{ value: 'disabled', label: 'Disabled' },
]
const FILTER_SECTION_LABEL_CLASS = 'text-[var(--text-muted)] text-small'
interface KnowledgeBaseProps {
id: string
knowledgeBaseName?: string
@@ -228,7 +235,7 @@ export function KnowledgeBase({
const [searchQuery, setSearchQuery] = useState('')
const [showTagsModal, setShowTagsModal] = useState(false)
const [enabledFilter, setEnabledFilter] = useState<string[]>([])
const [enabledFilter, setEnabledFilter] = useState<'all' | 'enabled' | 'disabled'>('all')
const [tagFilterEntries, setTagFilterEntries] = useState<
{
id: string
@@ -255,17 +262,6 @@ export function KnowledgeBase({
[tagFilterEntries]
)
const enabledFilterParam = useMemo<'all' | 'enabled' | 'disabled'>(() => {
if (enabledFilter.length === 1) return enabledFilter[0] as 'enabled' | 'disabled'
return 'all'
}, [enabledFilter])
const enabledDisplayLabel = useMemo(() => {
if (enabledFilter.length === 0) return 'All'
if (enabledFilter.length === 1) return enabledFilter[0] === 'enabled' ? 'Enabled' : 'Disabled'
return '2 selected'
}, [enabledFilter])
const handleSearchChange = useCallback((newQuery: string) => {
setSearchQuery(newQuery)
setCurrentPage(1)
@@ -356,7 +352,7 @@ export function KnowledgeBase({
if (hasSyncingConnectorsRef.current) return 5000
return false
},
enabledFilter: enabledFilterParam,
enabledFilter: enabledFilter,
tagFilters: activeTagFilters.length > 0 ? activeTagFilters : undefined,
})
@@ -611,7 +607,7 @@ export function KnowledgeBase({
knowledgeBaseId: id,
operation: 'enable',
selectAll: true,
enabledFilter: enabledFilterParam,
enabledFilter: enabledFilter,
},
{
onSuccess: (result) => {
@@ -658,7 +654,7 @@ export function KnowledgeBase({
knowledgeBaseId: id,
operation: 'disable',
selectAll: true,
enabledFilter: enabledFilterParam,
enabledFilter: enabledFilter,
},
{
onSuccess: (result) => {
@@ -707,7 +703,7 @@ export function KnowledgeBase({
knowledgeBaseId: id,
operation: 'delete',
selectAll: true,
enabledFilter: enabledFilterParam,
enabledFilter: enabledFilter,
},
{
onSuccess: (result) => {
@@ -747,12 +743,12 @@ export function KnowledgeBase({
const selectedDocumentsList = documents.filter((doc) => selectedDocuments.has(doc.id))
const enabledCount = isSelectAllMode
? enabledFilterParam === 'disabled'
? enabledFilter === 'disabled'
? 0
: pagination.total
: selectedDocumentsList.filter((doc) => doc.enabled).length
const disabledCount = isSelectAllMode
? enabledFilterParam === 'enabled'
? enabledFilter === 'enabled'
? 0
: pagination.total
: selectedDocumentsList.filter((doc) => !doc.enabled).length
@@ -878,45 +874,40 @@ export function KnowledgeBase({
const filterContent = useMemo(
() => (
<div className='flex w-[240px] flex-col gap-3 p-3'>
<div className='flex flex-col gap-1.5'>
<span className='font-medium text-[var(--text-secondary)] text-caption'>Status</span>
<Combobox
options={[
{ value: 'enabled', label: 'Enabled' },
{ value: 'disabled', label: 'Disabled' },
]}
multiSelect
multiSelectValues={enabledFilter}
onMultiSelectChange={(values) => {
setEnabledFilter(values)
<AutoWidthPanel>
<div className='flex flex-col gap-2'>
<div className='flex h-5 items-center justify-between'>
<span className={FILTER_SECTION_LABEL_CLASS}>Status</span>
{enabledFilter !== 'all' && (
<Button
variant='ghost'
onClick={() => {
setEnabledFilter('all')
setCurrentPage(1)
setSelectedDocuments(new Set())
setIsSelectAllMode(false)
}}
className='-mr-1 h-auto px-1 py-0.5 text-[var(--text-muted)] text-xs hover-hover:text-[var(--text-secondary)]'
>
Clear
</Button>
)}
</div>
<ChipDropdown
options={STATUS_FILTER_OPTIONS}
value={enabledFilter}
onChange={(value) => {
if (value !== 'all' && value !== 'enabled' && value !== 'disabled') return
setEnabledFilter(value)
setCurrentPage(1)
setSelectedDocuments(new Set())
setIsSelectAllMode(false)
}}
overlayContent={
<span className='truncate text-[var(--text-primary)]'>{enabledDisplayLabel}</span>
}
showAllOption
allOptionLabel='All'
size='sm'
className='h-[32px] w-full rounded-md'
align='start'
fullWidth
flush
/>
</div>
{enabledFilter.length > 0 && (
<Button
variant='ghost'
onClick={() => {
setEnabledFilter([])
setCurrentPage(1)
setSelectedDocuments(new Set())
setIsSelectAllMode(false)
}}
className='h-[32px] w-full text-[var(--text-secondary)] text-caption'
>
Clear status filter
</Button>
)}
<TagFilterSection
tagDefinitions={tagDefinitions}
entries={tagFilterEntries}
@@ -927,9 +918,9 @@ export function KnowledgeBase({
setIsSelectAllMode(false)
}}
/>
</div>
</AutoWidthPanel>
),
[enabledFilter, enabledDisplayLabel, tagDefinitions, tagFilterEntries]
[enabledFilter, tagDefinitions, tagFilterEntries]
)
const connectorBadges =
@@ -975,15 +966,12 @@ export function KnowledgeBase({
const filterTags: FilterTag[] = useMemo(
() => [
...(enabledFilter.length > 0
...(enabledFilter !== 'all'
? [
{
label:
enabledFilter.length === 1
? `Status: ${enabledFilter[0] === 'enabled' ? 'Enabled' : 'Disabled'}`
: 'Status: 2 selected',
label: `Status: ${enabledFilter === 'enabled' ? 'Enabled' : 'Disabled'}`,
onRemove: () => {
setEnabledFilter([])
setEnabledFilter('all')
setCurrentPage(1)
setSelectedDocuments(new Set())
setIsSelectAllMode(false)
@@ -1129,7 +1117,7 @@ export function KnowledgeBase({
const emptyMessage = searchQuery
? 'No documents found'
: enabledFilter.length > 0 || activeTagFilters.length > 0
: enabledFilter !== 'all' || activeTagFilters.length > 0
? 'Nothing matches your filter'
: undefined
@@ -1441,6 +1429,18 @@ export function KnowledgeBase({
)
}
/**
* Sizes the filter popover to its content with pure CSS `max-content` (clamped to
* `[280, 420]`). Because the padding box is part of `max-content`, the `p-3`
* inset is preserved on every edge — there is no separate measured/animated outer
* layer that can disagree by a few pixels and clip the right padding. The width
* still adapts to the active filters; it just resizes instantly rather than
* animating.
*/
function AutoWidthPanel({ children }: { children: ReactNode }) {
return <div className='flex w-max min-w-[280px] max-w-[420px] flex-col p-3'>{children}</div>
}
interface TagFilterEntry {
id: string
tagName: string
@@ -1467,13 +1467,97 @@ interface TagFilterSectionProps {
onChange: (entries: TagFilterEntry[]) => void
}
interface TagFilterValueControlProps {
entry: TagFilterEntry
onChange: (patch: Partial<TagFilterEntry>) => void
}
/**
* Tag filter section rendered inside the combined filter popover
* Renders the value input for a knowledge base tag filter row.
*/
function TagFilterValueControl({ entry, onChange }: TagFilterValueControlProps) {
const isBetween = entry.operator === 'between'
if (entry.fieldType === 'date') {
if (isBetween) {
return (
<div className='grid grid-cols-[1fr_auto_1fr] items-center gap-2'>
<ChipDatePicker
value={entry.value || undefined}
onChange={(value) => onChange({ value })}
placeholder='From'
fullWidth
flush
/>
<span className='flex-shrink-0 text-[var(--text-muted)] text-xs'>to</span>
<ChipDatePicker
value={entry.valueTo || undefined}
onChange={(value) => onChange({ valueTo: value })}
placeholder='To'
fullWidth
flush
/>
</div>
)
}
return (
<ChipDatePicker
value={entry.value || undefined}
onChange={(value) => onChange({ value })}
placeholder='Select date'
fullWidth
flush
/>
)
}
if (isBetween) {
return (
<div className='grid grid-cols-[1fr_auto_1fr] items-center gap-2'>
<Input
variant='chip'
size={1}
value={entry.value}
onChange={(event) => onChange({ value: event.target.value })}
placeholder='From'
/>
<span className='flex-shrink-0 text-[var(--text-muted)] text-xs'>to</span>
<Input
variant='chip'
size={1}
value={entry.valueTo}
onChange={(event) => onChange({ valueTo: event.target.value })}
placeholder='To'
/>
</div>
)
}
return (
<Input
variant='chip'
size={1}
value={entry.value}
onChange={(event) => onChange({ value: event.target.value })}
placeholder={
entry.fieldType === 'boolean'
? 'true or false'
: entry.fieldType === 'number'
? 'Enter number'
: 'Enter value'
}
/>
)
}
/**
* Tag filter section rendered inside the combined filter popover.
*/
function TagFilterSection({ tagDefinitions, entries, onChange }: TagFilterSectionProps) {
const activeCount = entries.filter((f) => f.tagSlot && f.value.trim()).length
const tagOptions: ComboboxOption[] = tagDefinitions.map((t) => ({
const tagOptions: ChipDropdownOption[] = tagDefinitions.map((t) => ({
value: t.displayName,
label: t.displayName,
}))
@@ -1483,6 +1567,21 @@ function TagFilterSection({ tagDefinitions, entries, onChange }: TagFilterSectio
[entries]
)
const scrollRef = useRef<HTMLDivElement>(null)
const prevCountRef = useRef(filtersToShow.length)
useEffect(() => {
if (filtersToShow.length > prevCountRef.current) {
const el = scrollRef.current
if (el) {
requestAnimationFrame(() => {
el.scrollTo({ top: el.scrollHeight, behavior: 'smooth' })
})
}
}
prevCountRef.current = filtersToShow.length
}, [filtersToShow.length])
const updateEntry = (id: string, patch: Partial<TagFilterEntry>) => {
const existing = filtersToShow.find((e) => e.id === id)
if (!existing) return
@@ -1516,130 +1615,94 @@ function TagFilterSection({ tagDefinitions, entries, onChange }: TagFilterSectio
if (tagDefinitions.length === 0) return null
return (
<div className='border-[var(--border-1)] border-t'>
<div className='flex items-center justify-between px-3 py-2'>
<span className='font-medium text-[var(--text-secondary)] text-caption'>
Filter by tags
</span>
<div className='flex items-center gap-1'>
{activeCount > 0 && (
<Button
variant='ghost'
className='h-auto px-1.5 py-0.5 text-[var(--text-muted)] text-xs'
onClick={() => onChange([])}
>
Clear all
</Button>
)}
<Button variant='ghost' className='h-auto p-0' onClick={addFilter}>
<Plus className='size-3.5' />
<div className='mt-3 border-[var(--border-1)] border-t pt-3'>
<div className='flex h-5 items-center justify-between'>
<span className={FILTER_SECTION_LABEL_CLASS}>Filter by tags</span>
{activeCount > 0 && (
<Button
variant='ghost'
className='-mr-1 h-auto px-1 py-0.5 text-[var(--text-muted)] text-xs hover-hover:text-[var(--text-secondary)]'
onClick={() => onChange([])}
>
Clear all
</Button>
</div>
)}
</div>
<div className='flex max-h-[320px] flex-col gap-2 overflow-y-auto px-3 pb-3'>
{filtersToShow.map((entry) => {
<div
ref={scrollRef}
className='mt-2 flex max-h-[300px] flex-col gap-2 overflow-y-auto overflow-x-hidden'
>
{filtersToShow.map((entry, index) => {
const operators = getOperatorsForFieldType(entry.fieldType)
const operatorOptions: ComboboxOption[] = operators.map((op) => ({
const operatorOptions: ChipDropdownOption[] = operators.map((op) => ({
value: op.value,
label: op.label,
}))
const isBetween = entry.operator === 'between'
return (
<div
key={entry.id}
className='flex flex-col gap-1.5 rounded-md border border-[var(--border-1)] p-2'
>
<div className='flex items-center justify-between'>
<Label className='text-[var(--text-muted)] text-xs'>Tag</Label>
<Button
variant='ghost'
className='size-5 p-0 text-[var(--text-muted)] hover-hover:text-[var(--text-error)]'
onClick={() => removeFilter(entry.id)}
>
<X className='size-3' />
</Button>
</div>
<Combobox
options={tagOptions}
value={entry.tagName}
onChange={(v) => handleTagChange(entry.id, v)}
placeholder='Select tag'
/>
{entry.tagSlot && (
<>
<Label className='text-[var(--text-muted)] text-xs'>Operator</Label>
<Combobox
options={operatorOptions}
value={entry.operator}
onChange={(v) => updateEntry(entry.id, { operator: v, valueTo: '' })}
placeholder='Select operator'
<div key={entry.id} className='flex flex-col gap-2'>
{index > 0 && (
<div className='flex items-center gap-2'>
<span className='shrink-0 text-[var(--text-muted)] text-xs leading-none'>
and
</span>
<div className='h-px flex-1 bg-[var(--border-1)]' />
</div>
)}
<div className='flex items-start gap-2'>
<div className='flex min-w-0 flex-1 flex-wrap items-center gap-2'>
<ChipDropdown
options={tagOptions}
value={entry.tagName}
onChange={(value) => handleTagChange(entry.id, value)}
placeholder='Select tag'
align='start'
matchTriggerWidth={false}
contentClassName='max-h-[240px] overflow-y-auto'
className='max-w-[150px]'
flush
/>
<Label className='text-[var(--text-muted)] text-xs'>Value</Label>
{entry.fieldType === 'date' ? (
isBetween ? (
<div className='flex items-center gap-1.5'>
<DatePicker
size='sm'
value={entry.value || undefined}
onChange={(v) => updateEntry(entry.id, { value: v })}
placeholder='From'
/>
<span className='flex-shrink-0 text-[var(--text-muted)] text-xs'>to</span>
<DatePicker
size='sm'
value={entry.valueTo || undefined}
onChange={(v) => updateEntry(entry.id, { valueTo: v })}
placeholder='To'
/>
</div>
) : (
<DatePicker
size='sm'
value={entry.value || undefined}
onChange={(v) => updateEntry(entry.id, { value: v })}
placeholder='Select date'
/>
)
) : isBetween ? (
<div className='flex items-center gap-1.5'>
<Input
value={entry.value}
onChange={(e) => updateEntry(entry.id, { value: e.target.value })}
placeholder='From'
className='h-[28px] text-caption'
/>
<span className='flex-shrink-0 text-[var(--text-muted)] text-xs'>to</span>
<Input
value={entry.valueTo}
onChange={(e) => updateEntry(entry.id, { valueTo: e.target.value })}
placeholder='To'
className='h-[28px] text-caption'
/>
</div>
) : (
<Input
value={entry.value}
onChange={(e) => updateEntry(entry.id, { value: e.target.value })}
placeholder={
entry.fieldType === 'boolean'
? 'true or false'
: entry.fieldType === 'number'
? 'Enter number'
: 'Enter value'
}
className='h-[28px] text-caption'
{entry.tagSlot && (
<ChipDropdown
options={operatorOptions}
value={entry.operator}
onChange={(value) => updateEntry(entry.id, { operator: value, valueTo: '' })}
placeholder='Operator'
align='start'
matchTriggerWidth={false}
flush
/>
)}
</>
</div>
<Button
variant='ghost'
className='relative size-[30px] shrink-0 p-0 text-[var(--text-muted)] before:absolute before:inset-[-5px] before:content-[""] hover-hover:bg-[var(--surface-active)] hover-hover:text-[var(--text-error)]'
onClick={() => removeFilter(entry.id)}
aria-label='Remove tag filter'
>
<X className='size-[14px]' />
</Button>
</div>
{entry.tagSlot && (
<TagFilterValueControl
entry={entry}
onChange={(patch) => updateEntry(entry.id, patch)}
/>
)}
</div>
)
})}
</div>
<Button
variant='ghost'
onClick={addFilter}
className='mt-2 h-[30px] w-full justify-start gap-2 px-2 text-[var(--text-secondary)] text-caption hover-hover:text-[var(--text-primary)]'
>
<Plus className='size-[14px]' />
Add filter
</Button>
</div>
)
}
apps/sim/app/workspace/[workspaceId]/knowledge/knowledge.tsx
+79 -98
View File
@@ -3,8 +3,8 @@
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
import { createLogger } from '@sim/logger'
import { useParams, useRouter } from 'next/navigation'
import type { ComboboxOption } from '@/components/emcn'
import { Combobox, Tooltip } from '@/components/emcn'
import type { ChipDropdownOption, ChipMultiSelectOption } from '@/components/emcn'
import { Button, ChipDropdown, ChipMultiSelect, Tooltip } from '@/components/emcn'
import { Database } from '@/components/emcn/icons'
import type { KnowledgeBaseData } from '@/lib/knowledge/types'
import type {
@@ -58,6 +58,20 @@ const COLUMNS: ResourceColumn[] = [
const DATABASE_ICON = <Database className='size-[14px]' />
const CONNECTOR_FILTER_OPTIONS: ChipDropdownOption[] = [
{ value: 'all', label: 'All' },
{ value: 'connected', label: 'With connectors' },
{ value: 'unconnected', label: 'Without connectors' },
]
const CONTENT_FILTER_OPTIONS: ChipDropdownOption[] = [
{ value: 'all', label: 'All' },
{ value: 'has-docs', label: 'Has documents' },
{ value: 'empty', label: 'Empty' },
]
const FILTER_SECTION_LABEL_CLASS = 'text-[var(--text-muted)] text-small'
function connectorCell(connectorTypes?: string[]): ResourceCell {
if (!connectorTypes || connectorTypes.length === 0) {
return { label: EMPTY_CELL_PLACEHOLDER }
@@ -409,28 +423,7 @@ export function Knowledge() {
[activeSort]
)
const connectorDisplayLabel = useMemo(() => {
if (connectorFilter.length === 0) return 'All'
if (connectorFilter.length === 1)
return connectorFilter[0] === 'connected' ? 'With connectors' : 'Without connectors'
return `${connectorFilter.length} selected`
}, [connectorFilter])
const contentDisplayLabel = useMemo(() => {
if (contentFilter.length === 0) return 'All'
if (contentFilter.length === 1)
return contentFilter[0] === 'has-docs' ? 'Has documents' : 'Empty'
return `${contentFilter.length} selected`
}, [contentFilter])
const ownerDisplayLabel = useMemo(() => {
if (ownerFilter.length === 0) return 'All'
if (ownerFilter.length === 1)
return members?.find((m) => m.userId === ownerFilter[0])?.name ?? '1 member'
return `${ownerFilter.length} members`
}, [ownerFilter, members])
const memberOptions: ComboboxOption[] = useMemo(
const memberOptions: ChipMultiSelectOption[] = useMemo(
() =>
(members ?? []).map((m) => ({
value: m.userId,
@@ -451,95 +444,83 @@ export function Knowledge() {
[members]
)
const hasActiveFilters =
connectorFilter.length > 0 || contentFilter.length > 0 || ownerFilter.length > 0
const filterContent = useMemo(
() => (
<div className='flex w-[240px] flex-col gap-3 p-3'>
<div className='flex flex-col gap-1.5'>
<span className='font-medium text-[var(--text-secondary)] text-caption'>Connectors</span>
<Combobox
options={[
{ value: 'connected', label: 'With connectors' },
{ value: 'unconnected', label: 'Without connectors' },
]}
multiSelect
multiSelectValues={connectorFilter}
onMultiSelectChange={setConnectorFilter}
overlayContent={
<span className='truncate text-[var(--text-primary)]'>{connectorDisplayLabel}</span>
}
showAllOption
allOptionLabel='All'
size='sm'
className='h-[32px] w-full rounded-md'
<div className='flex w-[260px] flex-col gap-3 p-3'>
<div className='flex flex-col gap-2'>
<div className='flex h-5 items-center justify-between'>
<span className={FILTER_SECTION_LABEL_CLASS}>Connectors</span>
{connectorFilter.length > 0 && (
<Button
variant='ghost'
onClick={() => setConnectorFilter([])}
className='-mr-1 h-auto px-1 py-0.5 text-[var(--text-muted)] text-xs hover-hover:text-[var(--text-secondary)]'
>
Clear
</Button>
)}
</div>
<ChipDropdown
options={CONNECTOR_FILTER_OPTIONS}
value={connectorFilter[0] ?? 'all'}
onChange={(value) => setConnectorFilter(value === 'all' ? [] : [value])}
align='start'
fullWidth
flush
/>
</div>
<div className='flex flex-col gap-1.5'>
<span className='font-medium text-[var(--text-secondary)] text-caption'>Content</span>
<Combobox
options={[
{ value: 'has-docs', label: 'Has documents' },
{ value: 'empty', label: 'Empty' },
]}
multiSelect
multiSelectValues={contentFilter}
onMultiSelectChange={setContentFilter}
overlayContent={
<span className='truncate text-[var(--text-primary)]'>{contentDisplayLabel}</span>
}
showAllOption
allOptionLabel='All'
size='sm'
className='h-[32px] w-full rounded-md'
<div className='flex flex-col gap-2'>
<div className='flex h-5 items-center justify-between'>
<span className={FILTER_SECTION_LABEL_CLASS}>Content</span>
{contentFilter.length > 0 && (
<Button
variant='ghost'
onClick={() => setContentFilter([])}
className='-mr-1 h-auto px-1 py-0.5 text-[var(--text-muted)] text-xs hover-hover:text-[var(--text-secondary)]'
>
Clear
</Button>
)}
</div>
<ChipDropdown
options={CONTENT_FILTER_OPTIONS}
value={contentFilter[0] ?? 'all'}
onChange={(value) => setContentFilter(value === 'all' ? [] : [value])}
align='start'
fullWidth
flush
/>
</div>
{memberOptions.length > 0 && (
<div className='flex flex-col gap-1.5'>
<span className='font-medium text-[var(--text-secondary)] text-caption'>Owner</span>
<Combobox
<div className='flex flex-col gap-2'>
<div className='flex h-5 items-center justify-between'>
<span className={FILTER_SECTION_LABEL_CLASS}>Owner</span>
{ownerFilter.length > 0 && (
<Button
variant='ghost'
onClick={() => setOwnerFilter([])}
className='-mr-1 h-auto px-1 py-0.5 text-[var(--text-muted)] text-xs hover-hover:text-[var(--text-secondary)]'
>
Clear
</Button>
)}
</div>
<ChipMultiSelect
options={memberOptions}
multiSelect
multiSelectValues={ownerFilter}
onMultiSelectChange={setOwnerFilter}
overlayContent={
<span className='truncate text-[var(--text-primary)]'>{ownerDisplayLabel}</span>
}
values={ownerFilter}
onChange={setOwnerFilter}
allLabel='All'
searchable
searchPlaceholder='Search members...'
showAllOption
allOptionLabel='All'
size='sm'
className='h-[32px] w-full rounded-md'
align='start'
fullWidth
flush
/>
</div>
)}
{hasActiveFilters && (
<button
type='button'
onClick={() => {
setConnectorFilter([])
setContentFilter([])
setOwnerFilter([])
}}
className='flex h-[32px] w-full items-center justify-center rounded-md text-[var(--text-secondary)] text-caption transition-colors hover-hover:bg-[var(--surface-active)]'
>
Clear all filters
</button>
)}
</div>
),
[
connectorFilter,
contentFilter,
ownerFilter,
memberOptions,
connectorDisplayLabel,
contentDisplayLabel,
ownerDisplayLabel,
hasActiveFilters,
]
[connectorFilter, contentFilter, ownerFilter, memberOptions]
)
const filterTags: FilterTag[] = useMemo(() => {
apps/sim/app/workspace/[workspaceId]/logs/components/log-details/components/trace-view/trace-view.tsx
+13 -4
View File
@@ -55,6 +55,13 @@ const MIN_BAR_PCT = 0.5
interface TraceViewProps {
traceSpans: TraceSpan[]
/**
* Authoritative, multiplier-inclusive run cost (dollars) from the persisted
* execution log. When provided it drives the header credit chip so the Trace
* tab and the Overview cost breakdown can never show different totals. Falls
* back to the root span's own cost only when absent (e.g. live previews).
*/
runCostDollars?: number
}
interface FlatSpanEntry {
@@ -706,8 +713,10 @@ const TraceDetailPane = memo(function TraceDetailPane({ span }: { span: TraceSpa
if (cacheRead) metaEntries.push({ label: 'Cache read', value: cacheRead })
if (cacheWrite) metaEntries.push({ label: 'Cache write', value: cacheWrite })
if (reasoning) metaEntries.push({ label: 'Reasoning tokens', value: reasoning })
const costTotal = formatCostAmount(span.cost?.total)
if (costTotal) metaEntries.push({ label: 'Cost', value: costTotal })
// Per-span cost is intentionally not shown: cost lives only in the usage_log
// ledger (the authoritative, multiplier-inclusive run total drives the header
// chip). Persisted spans are cost-stripped, so a per-span row would render on
// live runs but vanish on reload — show one consistent total instead.
if (span.errorType) metaEntries.push({ label: 'Error type', value: span.errorType })
if (span.iterationIndex !== undefined)
metaEntries.push({ label: 'Iteration', value: String(span.iterationIndex + 1) })
@@ -808,7 +817,7 @@ const TraceDetailPane = memo(function TraceDetailPane({ span }: { span: TraceSpa
* in a way that mirrors the executor's internal structure so investigators can
* follow block-by-block and segment-by-segment what happened and why.
*/
export const TraceView = memo(function TraceView({ traceSpans }: TraceViewProps) {
export const TraceView = memo(function TraceView({ traceSpans, runCostDollars }: TraceViewProps) {
const treeRef = useRef<HTMLDivElement>(null)
const [searchQuery, setSearchQuery] = useState('')
const [treePaneWidth, setTreePaneWidth] = useState(DEFAULT_TREE_PANE_WIDTH)
@@ -1017,7 +1026,7 @@ export const TraceView = memo(function TraceView({ traceSpans }: TraceViewProps)
{blockCount} {blockCount === 1 ? 'span' : 'spans'}
</span>
{(() => {
const rootCost = formatCostAmount(normalizedSpans[0]?.cost?.total)
const rootCost = formatCostAmount(runCostDollars ?? normalizedSpans[0]?.cost?.total)
return rootCost ? (
<span className='flex-shrink-0 font-medium text-[var(--text-tertiary)] text-caption tabular-nums'>
{rootCost}
apps/sim/app/workspace/[workspaceId]/logs/components/log-details/log-details.tsx
+85 -54
View File
@@ -25,6 +25,7 @@ import {
} from '@/components/emcn'
import type { WorkflowLogRow } from '@/lib/api/contracts/logs'
import { BASE_EXECUTION_CHARGE } from '@/lib/billing/constants'
import { apportionCredits, dollarsToCredits } from '@/lib/billing/credits/conversion'
import { cn } from '@/lib/core/utils/cn'
import { handleKeyboardActivation } from '@/lib/core/utils/keyboard'
import { filterHiddenOutputKeys } from '@/lib/logs/execution/trace-spans/trace-spans'
@@ -50,6 +51,16 @@ import { formatCost } from '@/providers/utils'
import { useLogDetailsUIStore } from '@/stores/logs/store'
import { MAX_LOG_DETAILS_WIDTH_RATIO, MIN_LOG_DETAILS_WIDTH } from '@/stores/logs/utils'
/**
* Renders an already-apportioned integer credit value. `dollars` is only used
* to distinguish a genuine zero ("0 credits") from a sub-credit charge that
* rounded down to zero ("<1 credit"); the credit figure itself is authoritative.
*/
function creditLabel(credits: number, dollars: number): string {
if (credits <= 0) return dollars > 0 ? '<1 credit' : '0 credits'
return `${credits.toLocaleString()} ${credits === 1 ? 'credit' : 'credits'}`
}
export const WorkflowOutputSection = memo(
function WorkflowOutputSection({ output }: { output: Record<string, unknown> }) {
const contentRef = useRef<HTMLDivElement>(null)
@@ -327,6 +338,54 @@ export function LogDetailsContent({ log, onActiveTabChange }: LogDetailsContentP
return { input: raw } as Record<string, unknown>
}, [log.executionData])
// Cost breakdown, sourced solely from the usage_log ledger (single source of
// truth). Line items (Base Run / per-model / per-integration) get integer
// credits apportioned with a single round at the total so rows always
// reconcile (never round-then-sum, which drifts). Pre-ledger runs that only
// have the cost_total projection show the total alone — no itemization, no
// parallel jsonb reconstruction.
const costBreakdown = useMemo((): {
rows: Array<{ key: string; label: string; credits: number; dollars: number }>
totalCredits: number
totalDollars: number
tokens: { input: number; output: number }
} | null => {
const ledger = log.costLedger
if (ledger && ledger.items.length > 0) {
const credits = apportionCredits(
ledger.items.map((item, i) => ({ key: String(i), dollars: item.cost }))
)
const rows = ledger.items.map((item, i) => ({
key: String(i),
label:
item.category === 'fixed' && item.description === 'execution_fee'
? 'Base Run'
: item.description,
credits: credits[String(i)] ?? 0,
dollars: item.cost,
}))
return {
rows,
totalCredits: dollarsToCredits(ledger.total),
totalDollars: ledger.total,
tokens: {
input: ledger.items.reduce((s, it) => s + (it.inputTokens ?? 0), 0),
output: ledger.items.reduce((s, it) => s + (it.outputTokens ?? 0), 0),
},
}
}
// Total-only (pre-ledger runs with just the cost_total projection).
const total = log.cost?.total
if (total == null) return null
return {
rows: [],
totalCredits: dollarsToCredits(total),
totalDollars: total,
tokens: { input: 0, output: 0 },
}
}, [log.costLedger, log.cost])
const formattedTimestamp = formatDate(log.createdAt)
const logStatus = getDisplayStatus(log.status)
@@ -530,67 +589,39 @@ export function LogDetailsContent({ log, onActiveTabChange }: LogDetailsContentP
{log.files && log.files.length > 0 && <FileCards files={log.files} isExecutionFile />}
{/* Cost Breakdown */}
{hasCostInfo && (
{hasCostInfo && costBreakdown && (
<div className='divide-y divide-[var(--border)] overflow-hidden rounded-md border border-[var(--border)] bg-[var(--surface-2)] dark:bg-transparent'>
<div className='flex h-10 items-center justify-between px-3 transition-colors hover-hover:bg-[var(--surface-2)]'>
<span className='font-medium text-[var(--text-tertiary)] text-caption'>
Base Run
</span>
<span className='font-medium text-[var(--text-secondary)] text-caption tabular-nums'>
{formatCost(BASE_EXECUTION_CHARGE)}
</span>
</div>
<div className='flex h-10 items-center justify-between px-3 transition-colors hover-hover:bg-[var(--surface-2)]'>
<span className='font-medium text-[var(--text-tertiary)] text-caption'>
Model Input
</span>
<span className='font-medium text-[var(--text-secondary)] text-caption tabular-nums'>
{formatCost(log.cost?.input || 0)}
</span>
</div>
<div className='flex h-10 items-center justify-between px-3 transition-colors hover-hover:bg-[var(--surface-2)]'>
<span className='font-medium text-[var(--text-tertiary)] text-caption'>
Model Output
</span>
<span className='font-medium text-[var(--text-secondary)] text-caption tabular-nums'>
{formatCost(log.cost?.output || 0)}
</span>
</div>
{(() => {
const models = (log.cost as Record<string, unknown>)?.models as
| Record<string, { toolCost?: number }>
| undefined
const totalToolCost = models
? Object.values(models).reduce((sum, m) => sum + (m?.toolCost || 0), 0)
: 0
return totalToolCost > 0 ? (
<div className='flex h-10 items-center justify-between px-3 transition-colors hover-hover:bg-[var(--surface-2)]'>
<span className='font-medium text-[var(--text-tertiary)] text-caption'>
Tool Usage
</span>
<span className='font-medium text-[var(--text-secondary)] text-caption tabular-nums'>
{formatCost(totalToolCost)}
</span>
</div>
) : null
})()}
{costBreakdown.rows.map((row) => (
<div
key={row.key}
className='flex h-10 items-center justify-between px-3 transition-colors hover-hover:bg-[var(--surface-2)]'
>
<span className='min-w-0 truncate font-medium text-[var(--text-tertiary)] text-caption'>
{row.label}
</span>
<span className='flex-shrink-0 font-medium text-[var(--text-secondary)] text-caption tabular-nums'>
{creditLabel(row.credits, row.dollars)}
</span>
</div>
))}
<div className='flex h-10 items-center justify-between px-3 transition-colors hover-hover:bg-[var(--surface-2)]'>
<span className='font-medium text-[var(--text-secondary)] text-caption'>
Total
</span>
<span className='font-semibold text-[var(--text-primary)] text-caption tabular-nums'>
{formatCost(log.cost?.total || 0)}
</span>
</div>
<div className='flex h-10 items-center justify-between px-3 transition-colors hover-hover:bg-[var(--surface-2)]'>
<span className='font-medium text-[var(--text-tertiary)] text-caption'>
Tokens
</span>
<span className='font-medium text-[var(--text-secondary)] text-caption tabular-nums'>
{log.cost?.tokens?.input || log.cost?.tokens?.prompt || 0} in ·{' '}
{log.cost?.tokens?.output || log.cost?.tokens?.completion || 0} out
{creditLabel(costBreakdown.totalCredits, costBreakdown.totalDollars)}
</span>
</div>
{(costBreakdown.tokens.input > 0 || costBreakdown.tokens.output > 0) && (
<div className='flex h-10 items-center justify-between px-3 transition-colors hover-hover:bg-[var(--surface-2)]'>
<span className='font-medium text-[var(--text-tertiary)] text-caption'>
Tokens
</span>
<span className='font-medium text-[var(--text-secondary)] text-caption tabular-nums'>
{costBreakdown.tokens.input} in · {costBreakdown.tokens.output} out
</span>
</div>
)}
<div className='px-3 py-2'>
<p className='font-medium text-[var(--text-tertiary)] text-xs'>
Total includes a {formatCost(BASE_EXECUTION_CHARGE)} base charge plus model and
@@ -609,7 +640,7 @@ export function LogDetailsContent({ log, onActiveTabChange }: LogDetailsContentP
className='mt-3 min-h-0 flex-1 overflow-hidden focus-visible:outline-none'
>
{traceSpans?.length ? (
<TraceView traceSpans={traceSpans} />
<TraceView traceSpans={traceSpans} runCostDollars={log.cost?.total} />
) : log.executionData ? (
<div className='flex h-full items-center justify-center px-4 text-center'>
<span className='font-medium text-[var(--text-tertiary)] text-sm'>

Some files were not shown because too many files have changed in this diff Show More