Waleed
8b93e43037
* improvement(integrations): validate BigQuery/Forms/PageSpeed + regenerate integration docs - BigQuery: mark null-defaulted outputs optional (get_table type/numRows/numBytes/creationTime/lastModifiedTime/location, list_datasets location, list_tables type, query totalBytesProcessed) - Google Forms: add response pagination (pageToken + filter params, nextPageToken output), fix pageSize visibility, advanced-mode pagination subBlocks + filter wandConfig - PageSpeed: add a 7th BlockMeta template (competitor benchmark) - Regenerate integration docs; add manual intro sections to new datagma/dropcontact/enrow/icypeas/leadmagic pages * fix(docs-gen): preserve apostrophes in tool descriptions when generating docs The doc generator extracted tool descriptions with a character class that excluded both quote types (['"]([^'"]...)['"]), so a double-quoted description containing an apostrophe (e.g. "Find someone's email") was truncated at the apostrophe — the generated docs/catalog showed stubs like "Find someone". Anchor extraction on the actual opening quote (single/double/backtick), matching the existing extractDescription helper, in both buildToolDescriptionMap and extractToolInfo. Regenerated docs restore full descriptions across all affected integrations (Apollo, Ahrefs, LeadMagic, Findymail, OpenAI, Slack, etc.). * fix(docs-gen): resolve tools defined in a sibling file + scope params per tool The doc generator located a tool's definition only by filename convention (decompress.ts / index.ts), so file_decompress — which lives in compress.ts alongside file_compress — fell back to index.ts and rendered an empty Input table. It also read the params block from the first tool in a multi-tool file, so every tool in such a file inherited the first tool's inputs/outputs. - getToolInfo: when no candidate file declares the exact tool ID, scan the whole tool-prefix directory for the file that does. - extractToolInfo: read the params block scoped to the specific tool, falling back to the full file for tools that inherit params via spread. Regenerated docs eliminate ~50 empty/incorrect input tables across integrations (clickhouse, rb2b, reddit, file, etc.); param-less OAuth-only tools correctly keep an empty input table.
Integration documentation generator
generate-docs.ts compiles the per-service integration pages under
apps/docs/content/docs/en/integrations/ from the block/tool/trigger registry in
apps/sim. The ontology it encodes: everything is a block, and an integration is one
block that has Actions and, optionally, a Trigger.
Golden rule: the generated
.mdxfiles are derived artifacts, not the source of truth. Do not hand-edit them — your changes are overwritten on the next run. The only editable region is theMANUAL-CONTENTblock (see below). To change what a page says, edit the TypeScript inapps/simand regenerate.
Where an integration lives canonically
For a service like Gmail, three TS sources define it:
| Source | What it is | What it feeds in the page |
|---|---|---|
apps/sim/blocks/blocks/<service>.ts |
The block: type, name, category (tools for integrations), bgColor, config sub-blocks, tools.access (which actions it exposes), an optional triggers capability, outputs |
Header / BlockInfoCard, Usage Instructions, and which actions + trigger appear |
apps/sim/tools/<service>/*.ts |
Each action's params + outputs | Every ### <action> → #### Input / #### Output under ## Actions |
apps/sim/triggers/<provider>/ |
The trigger's config fields + outputs | The ## Triggers section |
apps/sim/components/icons.tsx |
The brand glyph | The page icon |
The block references actions by id in tools.access; the generator looks each one up in
apps/sim/tools/.
What the generator does
Run with cd apps/sim && bun run generate-docs (or bun run scripts/generate-docs.ts
from the repo root). One pass (generateAllBlockDocs):
- Copies icons
apps/sim/components/icons.tsx→apps/docs/components/icons.tsxand buildsapps/docs/components/ui/icon-mapping.ts. - Block pass — for each integration block (
category: 'tools', plus thememory/knowledge/tableexceptions), writesintegrations/<service>.mdx:BlockInfoCard+ Usage Instructions +## Actions. - Trigger pass (
generateAllTriggerDocs) — readsapps/sim/triggers/<provider>/and appends a## Triggerssection to that service's page, or writes a standalone page for trigger-only services. - Writes
integrations/meta.jsonand regenerates the landing page'sintegrations.json.
Hand-written pages it never touches
Core block pages (blocks/*), the native trigger pages (triggers/{start,schedule,webhook,rss,table}),
the integrations overview (integrations/index.mdx), and the service-account pages are
fully hand-written. The generator skips them via HANDWRITTEN_INTEGRATION_DOCS,
HANDWRITTEN_TRIGGER_DOCS, and SKIP_TRIGGER_PROVIDERS. Add a page name to those sets if
you hand-author a page the generator would otherwise produce.
Manual content (the one editable region)
Each generated page may carry hand-written prose inside marker comments. The generator preserves anything between the markers and overwrites everything else, so this survives every regeneration:
{/* MANUAL-CONTENT-START:intro */}
[AgentMail](https://agentmail.to/) is an API-first email platform…
{/* MANUAL-CONTENT-END */}
Supported section names: intro (after the BlockInfoCard — the most common),
usage, configuration, outputs, notes. The merge is by marker name
(extractManualContent + mergeWithManualContent), so a section is re-inserted at the
matching spot in the freshly generated structure.
If you move the output folder, reseed manual content from the old location first — the generator only preserves markers it finds in the existing output file, so a fresh folder starts with none.
Practical: to change…
- An action's params/outputs, a trigger, or to add a service → edit
apps/sim/{blocks,tools,triggers}and re-run the generator. - A page's prose intro → edit its
MANUAL-CONTENT:introblock directly; it survives regen. - The overview / service-account / core-block / native-trigger pages → hand-edit freely.
Gotchas
- Never hand-edit
apps/docs/components/icons.tsx— step 1 overwrites it from the sim app. Components that need an icon the sim app lacks should define it locally or uselucide-react(seecomponents/workflow-preview/block-icons.tsx). - The generator is the source of truth for
integrations/and itsmeta.json; manual edits there are transient.
CI
The generator runs in CI on pushes to the main branch and commits the regenerated docs
back. Keep block/tool/trigger metadata accurate in apps/sim and the docs follow.