mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-21 21:15:56 +08:00
Waleed
303986f45f
* feat(snowflake): credential-based auth, object pickers, and 9 new operations
Replace the per-block host + PAT fields with a Snowflake service-account
credential, move the credential picker to the top of the block, back the
object fields with metadata-only pickers, and add nine operations.
- credential: snowflake-service-account token service account (account host +
programmatic access token), verified against the SQL API with the same
headers the tools use
- selectors: database, schema, table, warehouse, execution role, file format
and procedure pickers behind one /api/tools/snowflake/objects route
- new operations: unload_data, list_databases, list_schemas, list_tables,
alter_warehouse, resume_task, suspend_task, list_query_history,
list_copy_history
* fix(snowflake): migrate renamed subblock IDs and authenticate before parsing
- add SUBBLOCK_ID_MIGRATIONS entries so the renamed object fields map onto
their pickers and the removed host/apiKey values are parked
- authenticate the caller before contract validation in the selector route,
per the API route convention
* fix(snowflake): close unload-query breakouts, drop parked secrets, correct docs
- assertBalancedQuery now skips // line comments, $$ dollar quoting and rejects
ambiguous nested block comments; each hid a paren that let an injected
OVERWRITE = TRUE escape the derived table
- always emit OVERWRITE so an injected duplicate is rejected by Snowflake
rather than silently replacing staged files
- _removed_ migration targets now drop the stored value instead of parking it
under a dead key, where export scrubbing (which walks the block config) would
never clear it
- 403 falls back to the shared invalid-credentials message, which names the
network policy and SQL API causes Snowflake does not distinguish in the body
- correct the network-policy-by-user-type claim: only SERVICE_AGENT is exempt
- correct MAX_FILE_SIZE and errorOnly tool descriptions to match the fixed code
* fix(snowflake): stop untouched switches emitting clauses; retarget migration
- an untouched switch serializes as null, and advanced mode emits every
advanced subblock, so alter_warehouse silently sent AUTO_RESUME = FALSE and
permanently disabled auto-resume on the warehouse; normalize optional
booleans to undefined in tools.config.params
- point the subblock migration at the advanced text members: a migrated block
has no credential, so a picker cannot hydrate a stored name, and legacy
fileFormat values were qualified while the picker lists bare names
- add the missing json-object wand type and scope the SQL wand prompt, which
promised bindings that unload_data does not accept
* fix(migrations): sweep already-parked subblock values; align picker 403
- an earlier version of this migration renamed retired fields into _removed_*
keys instead of deleting them, so deployed workflows still hold those values;
they match no oldId, so a dedicated sweep clears them for every block type
- the picker now treats a Snowflake 403 like a 401: it means a network policy
or a disabled SQL API, which the credential validator already reports as a
credential problem rather than a bad request
* fix(wand): add json-array generation type for array-contract fields
The json-object reinforcement tells the model the response must start with {
and end with }, which fights any field whose contract is an array. Snowflake's
rows, matchColumns and procedureArguments all ask for arrays, so they were
being steered toward an object that the JSON parse would then reject.
Adds a sibling json-array type that strips fences the same way but reinforces
brackets, and points the three array fields at it. bindings and filters are
genuine objects and stay on json-object.
* fix(snowflake): unload a table, not an inline query
The COPY INTO grammar places the source immediately before its copy options, so
an inlined query sits one parenthesis from being able to rewrite them. Guarding
that means matching Snowflake's tokenizer exactly, and three successive versions
of the guard were each defeated: // line comments, $$ dollar quoting, and a bare
carriage return, which the scanner did not treat as a line terminator but
Snowflake does. Each fix was a guess at a lexer the public docs do not specify.
Removes the inline-query source instead of guessing a fourth time. A table name
goes through qualifiedIdentifier, which is provably safe. Exporting a query
result now means materializing it first — a view, or CREATE TABLE AS SELECT via
Execute SQL — which the tool description, the block skill and the docs all say.
Also from the final audit:
- optionalBoolean accepts the string forms a direct tool call delivers, matching
the other boolean readers on this block, and its TSDoc no longer states the
serializer rule backwards
- the five JSON editors declare language: 'json', so invalid JSON is caught
inline instead of at execution
- bound the RESULT_SCAN read in SQL, not only by rows_per_resultset
- pin every migration target to a live subblock id, for all blocks
Integration documentation generator
generate-docs.ts compiles the per-service integration pages under
apps/docs/content/docs/en/integrations/ from the block/tool/trigger registry in
apps/sim. The ontology it encodes: everything is a block, and an integration is one
block that has Actions and, optionally, a Trigger.
Golden rule: the generated
.mdxfiles are derived artifacts, not the source of truth. Do not hand-edit them — your changes are overwritten on the next run. The only editable region is theMANUAL-CONTENTblock (see below). To change what a page says, edit the TypeScript inapps/simand regenerate.
Where an integration lives canonically
For a service like Gmail, three TS sources define it:
| Source | What it is | What it feeds in the page |
|---|---|---|
apps/sim/blocks/blocks/<service>.ts |
The block: type, name, category (tools for integrations), bgColor, config sub-blocks, tools.access (which actions it exposes), an optional triggers capability, outputs |
Header / BlockInfoCard, Usage Instructions, and which actions + trigger appear |
apps/sim/tools/<service>/*.ts |
Each action's params + outputs | Every ### <action> → #### Input / #### Output under ## Actions |
apps/sim/triggers/<provider>/ |
The trigger's config fields + outputs | The ## Triggers section |
apps/sim/components/icons.tsx |
The brand glyph | The page icon |
The block references actions by id in tools.access; the generator looks each one up in
apps/sim/tools/.
What the generator does
Run with cd apps/sim && bun run generate-docs (or bun run scripts/generate-docs.ts
from the repo root). One pass (generateAllBlockDocs):
- Copies icons
apps/sim/components/icons.tsx→apps/docs/components/icons.tsxand buildsapps/docs/components/ui/icon-mapping.ts. - Block pass — for each integration block (
category: 'tools', plus thememory/knowledge/tableexceptions), writesintegrations/<service>.mdx:BlockInfoCard+ Usage Instructions +## Actions. - Trigger pass (
generateAllTriggerDocs) — readsapps/sim/triggers/<provider>/and appends a## Triggerssection to that service's page, or writes a standalone page for trigger-only services. - Writes
integrations/meta.jsonand regenerates the landing page'sintegrations.json.
Hand-written pages it never touches
Core block pages (blocks/*), the native trigger pages (triggers/{start,schedule,webhook,rss,table}),
the integrations overview (integrations/index.mdx), and the service-account pages are
fully hand-written. The generator skips them via HANDWRITTEN_INTEGRATION_DOCS,
HANDWRITTEN_TRIGGER_DOCS, and SKIP_TRIGGER_PROVIDERS. Add a page name to those sets if
you hand-author a page the generator would otherwise produce.
Manual content (the one editable region)
Each generated page may carry hand-written prose inside marker comments. The generator preserves anything between the markers and overwrites everything else, so this survives every regeneration:
{/* MANUAL-CONTENT-START:intro */}
[AgentMail](https://agentmail.to/) is an API-first email platform…
{/* MANUAL-CONTENT-END */}
Supported section names: intro (after the BlockInfoCard — the most common),
usage, configuration, outputs, notes. The merge is by marker name
(extractManualContent + mergeWithManualContent), so a section is re-inserted at the
matching spot in the freshly generated structure.
If you move the output folder, reseed manual content from the old location first — the generator only preserves markers it finds in the existing output file, so a fresh folder starts with none.
Practical: to change…
- An action's params/outputs, a trigger, or to add a service → edit
apps/sim/{blocks,tools,triggers}and re-run the generator. - A page's prose intro → edit its
MANUAL-CONTENT:introblock directly; it survives regen. - The overview / service-account / core-block / native-trigger pages → hand-edit freely.
Gotchas
- Never hand-edit
apps/docs/components/icons.tsx— step 1 overwrites it from the sim app. Components that need an icon the sim app lacks should define it locally or use@sim/emcn/icons(seecomponents/workflow-preview/block-icons.tsx). - The generator is the source of truth for
integrations/and itsmeta.json; manual edits there are transient.
CI
The generator runs in CI on pushes to the main branch and commits the regenerated docs
back. Keep block/tool/trigger metadata accurate in apps/sim and the docs follow.