mirror of
https://github.com/simstudioai/sim.git
synced 2026-10-31 03:47:12 +08:00
Vikhyath MondretiandClaude Fable 5
5112b0711d
* improvement(db): stop declaring retired usage columns ahead of their drop Contract-prep for the legacy usage-column drop. Removes the drizzle declarations for 19 retired user_stats counters, organization.departed_member_usage, and workflow_execution_logs.cost, so generated SELECT lists stop referencing them, plus the last incidental references: the cycle-close departed_member_usage resets, the v1 admin exposure of departedMemberUsage/currentPeriodCopilotCost (both serving dead values since the ledger cutover), the testing schema mirror, and a lock-order fixture. The DROP COLUMN migration ships separately once this is the deployed production app; contract-pending markers in schema.ts track it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(db): reserve the drop migration as a journaled no-op for schema parity CI requires drizzle-kit generate to be a no-op on every PR, so the declaration removal cannot ship without its migration. Committing the generated 0309 with every DROP commented out (a SELECT 1 no-op) keeps the parity gate and future generates clean while deferring the destructive half until this change is part of an already-deployed production release; the file documents the follow-up procedure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * improvement(db): keep declarations, route reads through live-column maps Restructures the contract prep per review: the no-op 0309 migration is gone and schema.ts keeps the deprecated declarations (with the contract-pending markers on the columns), so the follow-up PR ships the declaration removal and the real generated DROP migration together. What makes that safe is that no deployed query names the doomed columns anymore: every argless select()/returning()/relational read of user_stats, organization, and workflow_execution_logs (29 sites) now goes through omit(getTableColumns(...)) live-column maps, following the workspaceFileColumns precedent. scripts/check-pending-drop-tables.ts enforces this — it derives the guarded tables from the contract-pending markers themselves (picking up workspace_files' pending size drop too) and fails CI on any argless read, so the invariant holds between the two PRs and retires itself when the markers are deleted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(db): backfill residual cost_total projections before the cost drop The 0220 procedure projected every then-existing legacy cost json into cost_total, but a transition-window writer added 23 rows (all 2026-05-30, verified on the prod replica) carrying a numeric json total with no projection. Script migration 0009 re-runs 0220's exact candidate filter and projection in bounded batches at deploy time, so the pending cost DROP abandons nothing cost_total should hold. The contract PR that drops the column must deregister the script in the same change — it reads the column; the contract-pending marker says so. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): validate live-column builders and inline aliases in the pending-drop audit Hardens check-pending-drop-tables per review: the audit now derives each guarded table's doomed columns from schema.ts (@deprecated tags and column-level contract-pending markers), so a sanctioned omit(getTableColumns(t), [...]) or destructure-omit must name every doomed column away — an unrelated or incomplete key list is reported with the columns it misses, and schema.ts itself is no longer skipped, so the <table>Columns helpers fail the audit if a later deprecation is not added to their omit lists. Inline alias(t, ...) expressions passed directly to from()/getTableColumns()/insert/update/delete now resolve to their canonical table instead of escaping the checks. The testing mirror's *Columns aliases now omit the same doomed keys as the real helpers (and gain the missing proPeriodCostSnapshotAt entry), so tests cannot keep passing against doomed-column references the app no longer serves. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): derive pending-drop columns from the AST, and align the mock schema The doomed-column derivation matched `@deprecated` with a line regex, so a column documented with a multiline TSDoc block was never marked doomed and the live-column helpers were free to omit nothing for it. Derivation now runs off the parsed schema: guarded tables are pgTable calls whose enclosed comments carry a drop-flavored contract-pending marker, and doomed columns come from each property's own leading TSDoc block, so single-line and multiline forms read alike. Verified by mutation: making a deprecated column's TSDoc multiline and dropping it from the helper's omit list now fails with the column named, where the previous version reported clean. Also fills the seven live columns the testing mirror was missing (user_stats.limit_notifications, four organization settings columns, workflow_execution_logs.cost_total/models_used) so mock-derived selections have the same shape as production ones. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): resolve namespace and renamed imports in the pending-drop audit Table references were matched by bare identifier only, so an argless read reached through `import * as schema` — a live pattern here, e.g. `db.insert(schema.userStats)` in lib/auth/anonymous.ts — or through a renamed import was invisible to the audit. Resolution now runs over the file's schema-module import bindings (named, renamed, and namespace) as well as alias() bindings, at every position the audit inspects. Verified by probe: argless select() via `schema.userStats`, via a renamed import, and an argless .returning() on `db.insert(schema.userStats)` are all reported now, where the previous version reported none of the three. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): close the pending-drop resolver under variable assignment A table parked in an ordinary binding (`const t = userStats`) escaped the audit, since only alias() results were re-bound. Binding collection now resolves any declarator whose right side names a table — the table itself, a namespace member, an earlier binding, or an alias() call — and repeats to a fixpoint so declaration order does not matter. Verified by probe: argless reads through a direct assignment, a namespace-member assignment, and a chained binding are all reported now, where the previous version reported none; the repo stays clean, including a file where a local array shadows a pending table's name. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): match relational reads by table name, not by a literal query chain An argless findFirst/findMany reached through a binding that holds db.query escaped the audit, because the check required the receiver to be a literal `<x>.query` member. Drizzle's relational API is keyed by table name however the root is reached, so the check now keys on the table-named property itself, closing the whole class rather than the one aliasing form. Verified by probe: an aliased root (`const q = db.query`) is reported now where it previously was not, `tx.query` still is, a read that passes `columns` still is not, and the repo stays clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Integration documentation generator
generate-docs.ts compiles the per-service integration pages under
apps/docs/content/docs/en/integrations/ from the block/tool/trigger registry in
apps/sim. The ontology it encodes: everything is a block, and an integration is one
block that has Actions and, optionally, a Trigger.
Golden rule: the generated
.mdxfiles are derived artifacts, not the source of truth. Do not hand-edit them — your changes are overwritten on the next run. The only editable region is theMANUAL-CONTENTblock (see below). To change what a page says, edit the TypeScript inapps/simand regenerate.
Where an integration lives canonically
For a service like Gmail, three TS sources define it:
| Source | What it is | What it feeds in the page |
|---|---|---|
apps/sim/blocks/blocks/<service>.ts |
The block: type, name, category (tools for integrations), bgColor, config sub-blocks, tools.access (which actions it exposes), an optional triggers capability, outputs |
Header / BlockInfoCard, Usage Instructions, and which actions + trigger appear |
apps/sim/tools/<service>/*.ts |
Each action's params + outputs | Every ### <action> → #### Input / #### Output under ## Actions |
apps/sim/triggers/<provider>/ |
The trigger's config fields + outputs | The ## Triggers section |
apps/sim/components/icons.tsx |
The brand glyph | The page icon |
The block references actions by id in tools.access; the generator looks each one up in
apps/sim/tools/.
What the generator does
Run with cd apps/sim && bun run generate-docs (or bun run scripts/generate-docs.ts
from the repo root). One pass (generateAllBlockDocs):
- Copies icons
apps/sim/components/icons.tsx→apps/docs/components/icons.tsxand buildsapps/docs/components/ui/icon-mapping.ts. - Block pass — for each integration block (
category: 'tools', plus thememory/knowledge/tableexceptions), writesintegrations/<service>.mdx:BlockInfoCard+ Usage Instructions +## Actions. - Trigger pass (
generateAllTriggerDocs) — readsapps/sim/triggers/<provider>/and appends a## Triggerssection to that service's page, or writes a standalone page for trigger-only services. - Writes
integrations/meta.jsonand regenerates the landing page'sintegrations.json.
Hand-written pages it never touches
Core block pages (blocks/*), the native trigger pages (triggers/{start,schedule,webhook,rss,table}),
the integrations overview (integrations/index.mdx), and the service-account pages are
fully hand-written. The generator skips them via HANDWRITTEN_INTEGRATION_DOCS,
HANDWRITTEN_TRIGGER_DOCS, and SKIP_TRIGGER_PROVIDERS. Add a page name to those sets if
you hand-author a page the generator would otherwise produce.
Manual content (the one editable region)
Each generated page may carry hand-written prose inside marker comments. The generator preserves anything between the markers and overwrites everything else, so this survives every regeneration:
{/* MANUAL-CONTENT-START:intro */}
[AgentMail](https://agentmail.to/) is an API-first email platform…
{/* MANUAL-CONTENT-END */}
Supported section names: intro (after the BlockInfoCard — the most common),
usage, configuration, outputs, notes. The merge is by marker name
(extractManualContent + mergeWithManualContent), so a section is re-inserted at the
matching spot in the freshly generated structure.
If you move the output folder, reseed manual content from the old location first — the generator only preserves markers it finds in the existing output file, so a fresh folder starts with none.
Practical: to change…
- An action's params/outputs, a trigger, or to add a service → edit
apps/sim/{blocks,tools,triggers}and re-run the generator. - A page's prose intro → edit its
MANUAL-CONTENT:introblock directly; it survives regen. - The overview / service-account / core-block / native-trigger pages → hand-edit freely.
Gotchas
- Never hand-edit
apps/docs/components/icons.tsx— step 1 overwrites it from the sim app. Components that need an icon the sim app lacks should define it locally or use@sim/emcn/icons(seecomponents/workflow-preview/block-icons.tsx). - The generator is the source of truth for
integrations/and itsmeta.json; manual edits there are transient.
CI
The generator runs in CI on pushes to the main branch and commits the regenerated docs
back. Keep block/tool/trigger metadata accurate in apps/sim and the docs follow.