Files
sim/scripts
Waleed 05e8c7cd71 refactor(connectors): split client metadata from server runtime (#5076)
* refactor(connectors): split client metadata from server runtime + cover node:net in client bundle

The browser build broke with `Cannot find module 'node:net'`. Server-only
SSRF code in `input-validation.server.ts` (`dns/promises`, and since PR #5060
`undici` → `node:net`/`node:tls`) is statically reachable from the client
bundle via the tool/connector registries, which the workflow editor imports
for metadata. Node networking builtins have no browser shim, so Turbopack
cannot compile them for the client.

Two changes:

1. Split each connector's client-safe declarative metadata into a sibling
   `meta.ts` (`<name>ConnectorMeta`), mirroring the `XBlockMeta` /
   `BLOCK_META_REGISTRY` pattern. `connectors/registry.ts` is now the
   client-safe `CONNECTOR_META_REGISTRY` (+ `getConnectorMeta` /
   `getAllConnectorMeta`); the full registry with runtime fns moves to
   `connectors/registry.server.ts`. Client components consume the meta
   registry; the sync engine and knowledge API routes consume the server
   registry. This removes connectors from the client's server-only graph.
   Connector metadata is byte-for-byte identical before/after; runtime fns
   are untouched.

2. Extend the existing #4899 `turbopack.resolveAlias` browser stub — which
   already mapped `dns`/`dns/promises` to an empty module for the browser —
   to also cover `net`/`tls` (+ `node:` variants), since `undici` now pulls
   those in. The remaining tool/provider definitions still reach
   `input-validation.server` server-side; the browser-only stub keeps those
   Node builtins out of the client bundle while the real modules stay on the
   server, so SSRF validation and IP pinning are unaffected.

Connector authoring/validation skills updated to teach the meta.ts split.

* fix(icons): use Square logo glyph only, drop wordmark

* fix(connectors): share Discord max-messages default across meta and runtime

Discord defined DEFAULT_MAX_MESSAGES separately in meta.ts (config placeholder)
and discord.ts (sync behavior), which could drift. Export it from meta.ts and
import it in the runtime, matching the single-source pattern used by the other
connectors (e.g. gmail, intercom).

* refactor(tools): route grafana/agiloft egress server-side, drop SSRF browser shim

Move the server-only SSRF-pinned fetch out of the grafana (update_dashboard,
update_alert_rule) and agiloft (11 record/search tools) definitions and into
internal API routes, the same pattern the rest of the server-side tools (and
agiloft's own attach/retrieve) already use. The tool definitions are now purely
declarative (request → internal route), so they no longer import
`input-validation.server` and the tools registry is fully client-safe.

With connectors (meta split) and these tools no longer reaching server-only code
from the client bundle, the browser no longer pulls in `dns`/`net`/`tls`:

- Add `import 'server-only'` to `input-validation.server.ts` so any future client
  import fails loudly at build time instead of silently bloating the bundle.
- Remove the `turbopack.resolveAlias` browser stub and delete
  `empty-node-fallback.browser.ts` — the root cause is fixed, the shim is gone.

Behavior is unchanged: each route runs the exact merge/validation/fetch logic the
tool ran before (every header, param branch, JSON-parse guard, error string, and
SSRF pinning preserved); only the location of execution moved from the client-
bundled definition to a server route.

* fix(connectors): move onedrive tagDefinitions into meta; drop server-only guard

- onedrive's tagDefinitions lived in the runtime file, so the client meta
  registry returned undefined for it and the add-connector tag opt-out section
  stopped rendering for onedrive. Move it into meta.ts like the other connectors
  so client and server see identical metadata (verified across all 50).
- Remove the 'server-only' import from input-validation.server.ts: the meta/route
  split already keeps it out of the client bundle, and blocks/tools registries
  don't use the guard either.

* fix(grafana): surface upstream error when the prefetch GET fails

Check response.ok on the existing-resource GET in both update routes and return
the upstream status/body, matching how the tool framework surfaced GET errors
before the move to internal routes (the framework checks response.ok before
transformResponse). Without this, a failed prefetch produced a generic
'Failed to fetch existing ...' message and dropped Grafana's error detail.

* fix(grafana): reject invalid panels JSON instead of silently ignoring it

Grafana's dashboard API treats panels as a required array and returns 400 on
invalid JSON; this route already errors on every other JSON param. Return
'Invalid JSON for panels parameter' instead of swallowing the parse error and
proceeding with a misleading success.

* fix(grafana): trim dashboard/alert-rule UID in route URLs (carry over #5082)

PR #5082 added .trim() on dashboardUid/alertRuleUid in the original tool URL
builders. Those tools now build their URLs in the internal routes, so apply the
same trim there to preserve that behavior.

* fix(grafana): route update_folder egress server-side (carry over #5082)

#5082 added a grafana update_folder tool that does SSRF-pinned fetch in its
postProcess, re-introducing the client-bundle leak. Convert it to the internal
API route pattern like the other update tools so the def is declarative and
input-validation.server stays out of the client bundle.

* fix(grafana): surface route failures in transformResponse instead of masking them

The grafana update tools' transformResponse hardcoded success: true and dropped
the route's error, so an upstream/validation failure (HTTP 200 with
{ success: false, error }) was reported to the workflow as a success. Forward
data.success and data.error (matching the agiloft tools) so failures propagate
as before the move to internal routes.
2026-06-15 19:53:37 -07:00
..

Integration documentation generator

generate-docs.ts compiles the per-service integration pages under apps/docs/content/docs/en/integrations/ from the block/tool/trigger registry in apps/sim. The ontology it encodes: everything is a block, and an integration is one block that has Actions and, optionally, a Trigger.

Golden rule: the generated .mdx files are derived artifacts, not the source of truth. Do not hand-edit them — your changes are overwritten on the next run. The only editable region is the MANUAL-CONTENT block (see below). To change what a page says, edit the TypeScript in apps/sim and regenerate.

Where an integration lives canonically

For a service like Gmail, three TS sources define it:

Source What it is What it feeds in the page
apps/sim/blocks/blocks/<service>.ts The block: type, name, category (tools for integrations), bgColor, config sub-blocks, tools.access (which actions it exposes), an optional triggers capability, outputs Header / BlockInfoCard, Usage Instructions, and which actions + trigger appear
apps/sim/tools/<service>/*.ts Each action's params + outputs Every ### <action> → #### Input / #### Output under ## Actions
apps/sim/triggers/<provider>/ The trigger's config fields + outputs The ## Triggers section
apps/sim/components/icons.tsx The brand glyph The page icon

The block references actions by id in tools.access; the generator looks each one up in apps/sim/tools/.

What the generator does

Run with cd apps/sim && bun run generate-docs (or bun run scripts/generate-docs.ts from the repo root). One pass (generateAllBlockDocs):

  1. Copies icons apps/sim/components/icons.tsx → apps/docs/components/icons.tsx and builds apps/docs/components/ui/icon-mapping.ts.
  2. Block pass — for each integration block (category: 'tools', plus the memory / knowledge / table exceptions), writes integrations/<service>.mdx: BlockInfoCard + Usage Instructions + ## Actions.
  3. Trigger pass (generateAllTriggerDocs) — reads apps/sim/triggers/<provider>/ and appends a ## Triggers section to that service's page, or writes a standalone page for trigger-only services.
  4. Writes integrations/meta.json and regenerates the landing page's integrations.json.

Hand-written pages it never touches

Core block pages (blocks/*), the native trigger pages (triggers/{start,schedule,webhook,rss,table}), the integrations overview (integrations/index.mdx), and the service-account pages are fully hand-written. The generator skips them via HANDWRITTEN_INTEGRATION_DOCS, HANDWRITTEN_TRIGGER_DOCS, and SKIP_TRIGGER_PROVIDERS. Add a page name to those sets if you hand-author a page the generator would otherwise produce.

Manual content (the one editable region)

Each generated page may carry hand-written prose inside marker comments. The generator preserves anything between the markers and overwrites everything else, so this survives every regeneration:

{/* MANUAL-CONTENT-START:intro */}
[AgentMail](https://agentmail.to/) is an API-first email platform…
{/* MANUAL-CONTENT-END */}

Supported section names: intro (after the BlockInfoCard — the most common), usage, configuration, outputs, notes. The merge is by marker name (extractManualContent + mergeWithManualContent), so a section is re-inserted at the matching spot in the freshly generated structure.

If you move the output folder, reseed manual content from the old location first — the generator only preserves markers it finds in the existing output file, so a fresh folder starts with none.

Practical: to change…

  • An action's params/outputs, a trigger, or to add a service → edit apps/sim/{blocks,tools,triggers} and re-run the generator.
  • A page's prose intro → edit its MANUAL-CONTENT:intro block directly; it survives regen.
  • The overview / service-account / core-block / native-trigger pages → hand-edit freely.

Gotchas

  • Never hand-edit apps/docs/components/icons.tsx — step 1 overwrites it from the sim app. Components that need an icon the sim app lacks should define it locally or use lucide-react (see components/workflow-preview/block-icons.tsx).
  • The generator is the source of truth for integrations/ and its meta.json; manual edits there are transient.

CI

The generator runs in CI on pushes to the main branch and commits the regenerated docs back. Keep block/tool/trigger metadata accurate in apps/sim and the docs follow.