Waleed
1465e94574
* feat(tinyfish): add TinyFish web agent, search, and fetch integration Adds the TinyFish integration: eight tools across the Agent, Search, and Fetch APIs, a block wiring them, the brand icon, and hosted-key support. Agent runs are metered on the step count TinyFish reports. Search and Fetch are free, so their hosted key costs nothing to run. The async run and its run read/cancel/list companions carry no hosting config — their charge accrues after the request returns and cannot be metered — so they always require the caller's own key. * fix(tinyfish): address review findings - Add tinyfish to PROVIDER_SECTIONS. The sectioned BYOK renderer drops any provider missing from a section, so the key field never rendered. Export PROVIDERS/PROVIDER_SECTIONS and assert they agree, so the next provider to miss a section fails CI instead of vanishing. - Replace the `any` payload params with raw snake_case wire types. This caught two real gaps: `status` could reach the output undefined, and `error` could be null where ToolResponse.error is `string | undefined`. - Reject an already-parsed array output schema, which a `json` block input can produce, and name malformed schema JSON instead of leaking a SyntaxError. - Count Fetch rate-limit usage from the submitted URLs rather than the returned arrays, so a URL in neither array cannot undercount. - Make "List runs" a literal canvas clause; a blank goal filter lists everything. - Regenerate the docs manifest for the new integration page. * chore(byok): audit that every hosted provider is fully wired Adds check:byok-providers. A hosted tool names its provider once in hosting.byokProviderId, but the id must also reach the zod enum, the settings PROVIDERS row, and a PROVIDER_SECTIONS section. Only the union is compiler- enforced; the rest fail silently, which is how TinyFish shipped with no settings row in the first place. Also flags drift between the two BYOKProviderId declarations. Correct the Search and Fetch rate-limit rationale: TinyFish documents both ceilings per API key, not per account, so the numbered key pool does raise the total. Records why a free product is still worth hosting — the endpoints require a key, so hosting is what removes the signup.
Integration documentation generator
generate-docs.ts compiles the per-service integration pages under
apps/docs/content/docs/en/integrations/ from the block/tool/trigger registry in
apps/sim. The ontology it encodes: everything is a block, and an integration is one
block that has Actions and, optionally, a Trigger.
Golden rule: the generated
.mdxfiles are derived artifacts, not the source of truth. Do not hand-edit them — your changes are overwritten on the next run. The only editable region is theMANUAL-CONTENTblock (see below). To change what a page says, edit the TypeScript inapps/simand regenerate.
Where an integration lives canonically
For a service like Gmail, three TS sources define it:
| Source | What it is | What it feeds in the page |
|---|---|---|
apps/sim/blocks/blocks/<service>.ts |
The block: type, name, category (tools for integrations), bgColor, config sub-blocks, tools.access (which actions it exposes), an optional triggers capability, outputs |
Header / BlockInfoCard, Usage Instructions, and which actions + trigger appear |
apps/sim/tools/<service>/*.ts |
Each action's params + outputs | Every ### <action> → #### Input / #### Output under ## Actions |
apps/sim/triggers/<provider>/ |
The trigger's config fields + outputs | The ## Triggers section |
apps/sim/components/icons.tsx |
The brand glyph | The page icon |
The block references actions by id in tools.access; the generator looks each one up in
apps/sim/tools/.
What the generator does
Run with cd apps/sim && bun run generate-docs (or bun run scripts/generate-docs.ts
from the repo root). One pass (generateAllBlockDocs):
- Copies icons
apps/sim/components/icons.tsx→apps/docs/components/icons.tsxand buildsapps/docs/components/ui/icon-mapping.ts. - Block pass — for each integration block (
category: 'tools', plus thememory/knowledge/tableexceptions), writesintegrations/<service>.mdx:BlockInfoCard+ Usage Instructions +## Actions. - Trigger pass (
generateAllTriggerDocs) — readsapps/sim/triggers/<provider>/and appends a## Triggerssection to that service's page, or writes a standalone page for trigger-only services. - Writes
integrations/meta.jsonand regenerates the landing page'sintegrations.json.
Hand-written pages it never touches
Core block pages (blocks/*), the native trigger pages (triggers/{start,schedule,webhook,rss,table}),
the integrations overview (integrations/index.mdx), and the service-account pages are
fully hand-written. The generator skips them via HANDWRITTEN_INTEGRATION_DOCS,
HANDWRITTEN_TRIGGER_DOCS, and SKIP_TRIGGER_PROVIDERS. Add a page name to those sets if
you hand-author a page the generator would otherwise produce.
Manual content (the one editable region)
Each generated page may carry hand-written prose inside marker comments. The generator preserves anything between the markers and overwrites everything else, so this survives every regeneration:
{/* MANUAL-CONTENT-START:intro */}
[AgentMail](https://agentmail.to/) is an API-first email platform…
{/* MANUAL-CONTENT-END */}
Supported section names: intro (after the BlockInfoCard — the most common),
usage, configuration, outputs, notes. The merge is by marker name
(extractManualContent + mergeWithManualContent), so a section is re-inserted at the
matching spot in the freshly generated structure.
If you move the output folder, reseed manual content from the old location first — the generator only preserves markers it finds in the existing output file, so a fresh folder starts with none.
Practical: to change…
- An action's params/outputs, a trigger, or to add a service → edit
apps/sim/{blocks,tools,triggers}and re-run the generator. - A page's prose intro → edit its
MANUAL-CONTENT:introblock directly; it survives regen. - The overview / service-account / core-block / native-trigger pages → hand-edit freely.
Gotchas
- Never hand-edit
apps/docs/components/icons.tsx— step 1 overwrites it from the sim app. Components that need an icon the sim app lacks should define it locally or use@sim/emcn/icons(seecomponents/workflow-preview/block-icons.tsx). - The generator is the source of truth for
integrations/and itsmeta.json; manual edits there are transient.
CI
The generator runs in CI on pushes to the main branch and commits the regenerated docs
back. Keep block/tool/trigger metadata accurate in apps/sim and the docs follow.