opensource/sim
3
0
Fork 0
mirror of https://github.com/simstudioai/sim.git synced 2026-09-24 15:45:35 +08:00
Code Issues Packages Projects Releases Wiki Activity

perf(prefetch): stop calling our own API over the wire during server render (#6657)

Browse Source 
* perf(prefetch): read the data layer instead of calling our own API over the wire

Four server-render prefetches went out over HTTP to our own routes. With
INTERNAL_API_BASE_URL unset in prod, getInternalApiBaseUrl() falls back to the
public base URL, so each was RSC -> public HTTPS -> load balancer -> back into
the app, awaited inside the render with a second round of auth.

- /home fetched the workflow folder list that the workspace layout had already
  fetched, under the identical query key. Since getQueryClient() builds a new
  client per call on the server, the two never deduped: same data, twice a
  request, once directly and once over the wire. Dropped; the layout's entry
  already hydrates it.
- /home cached raw route JSON under workspaceFilesKeys.list, while
  files/prefetch.ts seeds that same key from listWorkspaceFilesWithShares. The
  contract declares the date fields z.coerce.date(), so consumers hold Dates —
  a file record's type depended on which page the viewer landed on. Now reads
  the same function files/prefetch.ts does.
- tables and knowledge folder reads now call listFoldersForWorkspace, matching
  the sidebar prefetch.

These reads carry no authorization of their own, so each surface proves the
viewer through getWorkspaceHostContextForViewer first and caches nothing when
it fails, leaving the client fetch to reach the route for the real 403. Both it
and getSession are cache()d and already resolved by the layout, so the proof
costs no extra queries.

Left on the wire, deliberately: the tables and knowledge lists, whose cached
shape is the serialized wire shape, and pinned items and members, which have no
exported data-layer function.

* improvement(prefetch): skip the viewer proof when there is no session

Passing an empty-string userId ran a real permission query that could only
return null. Take an optional userId instead and skip straight to the
unauthorized path, matching how the home prefetch is called.

* perf(prefetch): finish removing self-HTTP prefetches and delete the legacy helper

Converts the last four server-render prefetches that called our own API over
HTTP, and deletes prefetch-internal-fetch.ts now that nothing imports it.

- knowledge bases: runs the route's own listInternalKnowledgeBases use case
  with a principal from the same internalSessionAuth policy the route declares,
  then projects through the same presenter and contract. Not a bypass of the
  application boundary — the same path, called in-process.
- tables: extracts the route's list projection into lib/table/wire.ts as
  toTableListItem, which the route and the prefetch now both call. This matters
  because listTablesContract's response schema is a passthrough z.custom, so a
  client fetch caches the route's JSON verbatim. Seeding listTables() directly
  would have put Date objects and the server-only metadata field under a key the
  hook never sees them on.
- pinned items: extracts the route's inline query into lib/pinned-items/queries.ts
  as listPinnedItemsForUser, which the route now calls too.
- workspace members: getWorkspaceMemberProfiles already existed; the prefetch
  calls it directly.

normalizeColumn moves from app/api/table/utils.ts to lib/table/wire.ts with ten
importers repointed. That also removes a pre-existing lib/* -> app/api/* boundary
violation in lib/table/import-runner.ts. No response shape changes: the v1/v2
edits are import-path moves only.

Every converted read proves the viewer first and caches nothing when that fails,
so an unauthorized viewer's client fetch still reaches the route for the real
403. Authorization equivalence was checked by unfolding both paths to
checkWorkspaceAccess rather than assumed.

* improvement(prefetch): collapse the duplicated folder prefetch and unify the call shape

- Extract prefetchResourceFolders. The same eight-line folder prefetch was
  written three times, varying only by resourceType, with the key, stale time
  and mapper kept in sync by hand.
- Adopting it removes the conditional spread from the tables and knowledge
  prefetches. Tables can now early-return, matching prefetchFilesBrowser:
  prefetchResourceListChrome already self-guards on the same cached host
  context, so a null context meant the function did nothing either way.
- Take userId as string | undefined everywhere and guard inside, so every
  prefetch module has one calling convention rather than two.
- Export toWireTimestamp and use it for the create-table response's own copy of
  the same idiom, and drop a cast that the extraction made dead: the parameter
  is already TableDefinition, whose schema is TableSchema.
- Read params and the session concurrently on the tables and knowledge pages,
  matching the files page, and drop TSDoc that restated each prefetch's own.

* fix(prefetch): keep the tables list on its route and cut the executor edge

Reading listTables from a page prefetch put the executable tool registry into
the Tables page server graph — ~4,700 modules, which check:tool-registry-boundary
rejects. lib/table/service reaches workflow-columns by several independent
paths (directly, and through jobs/service and rows/service), so severing one
edge is not enough; untangling that belongs in its own change.

- The tables list goes back through GET /api/table, with the reason recorded so
  the next person does not repeat the attempt. Folders and chrome on that page
  stay on the data layer.
- stripGroupDeps moves to its own leaf module. It is a pure projection over a
  WorkflowGroup, but living beside the group runtime meant every importer of
  lib/table/service paid for the executor to get it.

Net effect on the Tables page graph: 2,186 modules to 1,742.

* perf(prefetch): finish the migration, delete the legacy helper, ratchet page graphs

Answers the question the previous commit left open: the tables list did not have
to stay on HTTP. lib/table/service reached the executor through
jobs/service -> rows/service -> workflow-columns, for one symbol.
pendingDeleteMask is a delete-visibility SQL clause with no executor
involvement, so it moves to its own leaf and that chain is cut. The tables
prefetch now reads the data layer like every other one, and
prefetch-internal-fetch.ts is deleted: nothing in the app calls its own API over
HTTP during a server render any more.

stripGroupDeps likewise moves to a leaf rather than being re-exported through
workflow-columns, so its importers no longer pull the executor to get a pure
projection.

React Query mechanism fixes, all found by audit:
- settings/[section] fired two prefetches without awaiting them. Only a settled
  query is dehydrated, so those were shipped mid-flight; a rejection hydrated
  into an error state retryOnMount: false never retries, leaving the panel
  broken for the session. Awaited now, and the pending-dehydration opt-in is
  removed since nothing streams.
- The viewer profile was prefetched by both the layout and the settings page.
  Separate server QueryClients mean that was a real second read per request.
- prefetchSubscriptionData was dead, and hand-rolled an unannotated raw fetch.
- retry is scoped to the browser. Query core defaults it to 0 on the server;
  stating one value for both opted awaited prefetches into a retry backoff. The
  gcTime default is dropped entirely — 5 minutes is already the browser default,
  and setting it explicitly overrode the server's Infinity, leaving a live timer
  and payload per request.

check:tool-registry-boundary now also ratchets per-page module counts against a
committed baseline, attributing a regression to the import that caused it via a
dominator tree. It caught a +444 regression in this branch by hand; it would
have caught it in CI. Its import regex also missed bare side-effect imports,
so `import '@/tools/registry'` could have slipped past it entirely.

Prefetch guidance added to .claude/rules/sim-queries.md.

* fix(prefetch): correct the extracted module's db imports and stale rationale

Audit findings from the migration.

- pending-delete-mask imported its schema tables from @sim/db rather than
  @sim/db/schema, which the module it came from was careful to split. The
  global test mocks are bound per-entrypoint and only the schema mock exports
  tables, so every suite that reaches pendingDeleteMask would have failed on a
  missing mock export. Restored to the original convention, and the same split
  applied to the new pinned-items queries module before it grows a test.
- The settings prefetch and page justified awaiting with a mechanism this
  branch removed — pending queries being shipped with their promise. Only a
  settled query is dehydrated now, so an unawaited prefetch is dropped from the
  payload entirely. Same conclusion, correct reason, and no longer contradicting
  the rule this branch added.
- Removed the doc block left orphaned above validateSchema when stripGroupDeps
  moved out of workflow-columns.

Skill projections regenerated after trimming the boundary skill.

* chore(table): drop a section separator comment

Separators like these are non-TSDoc decoration that CLAUDE.md already rules
out. This is the only one in a file this branch touches; the rest of the repo
is swept separately.

* chore: remove section separator comments

CLAUDE.md already rules these out ("No ==== separators. No non-TSDoc
comments"), but 546 of them had accumulated across 48 files. They decorate
rather than explain, and they drift: a separator says "Validation" while the
code beneath it moved elsewhere, as one in workflow-columns already had.

Pure deletion — no source line was touched, and lines inside template
literals were skipped so nothing in a generated string changed.
This commit is contained in:
Waleed
2026-08-13 00:12:45 -07:00
committed by GitHub
parent e8d278b6e4
commit c49751b32e
102 changed files with 1462 additions and 1166 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/check-tool-registry-boundary.baseline.json
+382
View File
@@ -0,0 +1,382 @@
{
"generatedFrom": "app/workspace page/layout module graphs",
"tolerance": {
"modules": 25,
"percent": 2
},
"entries": {
"app/workspace/[workspaceId]/chat/[chatId]/layout.tsx": {
"modules": 5,
"gateways": {}
},
"app/workspace/[workspaceId]/chat/[chatId]/page.tsx": {
"modules": 2890,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/home/home.tsx": 1328,
"apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/mothership-view.tsx": 971,
"apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/index.ts": 836,
"apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-content/index.ts": 833,
"apps/sim/triggers/registry.ts": 446,
"apps/sim/blocks/registry.ts": 301,
"apps/sim/lib/auth/index.ts": 297,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx": 295
}
},
"app/workspace/[workspaceId]/files/[fileId]/page.tsx": {
"modules": 1909,
"gateways": {
"apps/sim/triggers/registry.ts": 446,
"apps/sim/blocks/registry.ts": 328,
"apps/sim/lib/auth/index.ts": 300,
"apps/sim/app/workspace/[workspaceId]/files/files.tsx": 275,
"apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/index.ts": 143,
"apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/file-viewer.tsx": 129,
"apps/sim/lib/api/contracts/index.ts": 107,
"apps/sim/lib/webhooks/providers/index.ts": 99
}
},
"app/workspace/[workspaceId]/files/[fileId]/view/page.tsx": {
"modules": 57,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/files/[fileId]/view/file-viewer.tsx": 56,
"apps/sim/hooks/queries/workspace-files.ts": 53
}
},
"app/workspace/[workspaceId]/files/page.tsx": {
"modules": 1909,
"gateways": {
"apps/sim/triggers/registry.ts": 446,
"apps/sim/blocks/registry.ts": 328,
"apps/sim/lib/auth/index.ts": 300,
"apps/sim/app/workspace/[workspaceId]/files/files.tsx": 277,
"apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/index.ts": 143,
"apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/file-viewer.tsx": 129,
"apps/sim/lib/api/contracts/index.ts": 107,
"apps/sim/lib/webhooks/providers/index.ts": 99
}
},
"app/workspace/[workspaceId]/home/layout.tsx": {
"modules": 6,
"gateways": {}
},
"app/workspace/[workspaceId]/home/page.tsx": {
"modules": 2890,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/home/home.tsx": 1328,
"apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/mothership-view.tsx": 971,
"apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/index.ts": 836,
"apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-content/index.ts": 833,
"apps/sim/triggers/registry.ts": 446,
"apps/sim/blocks/registry.ts": 301,
"apps/sim/lib/auth/index.ts": 297,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx": 295
}
},
"app/workspace/[workspaceId]/integrations/[block]/page.tsx": {
"modules": 1268,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/integrations/[block]/integration-block-detail.tsx": 1243,
"apps/sim/blocks/registry.ts": 925,
"apps/sim/triggers/index.ts": 482,
"apps/sim/lib/api/contracts/index.ts": 128,
"apps/sim/stores/workflows/registry/store.ts": 82,
"apps/sim/lib/api/contracts/tools/index.ts": 60,
"apps/sim/hooks/queries/deployments.ts": 59,
"apps/sim/lib/workflows/comparison/compare.ts": 56
}
},
"app/workspace/[workspaceId]/integrations/connected/[credentialId]/page.tsx": {
"modules": 1254,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/integrations/connected/[credentialId]/connected-credential-detail.tsx": 1253,
"apps/sim/triggers/registry.ts": 481,
"apps/sim/blocks/registry.ts": 331,
"apps/sim/lib/api/contracts/index.ts": 134,
"apps/sim/stores/workflows/registry/store.ts": 62,
"apps/sim/lib/api/contracts/tools/index.ts": 60,
"apps/sim/hooks/queries/deployments.ts": 59,
"apps/sim/lib/workflows/comparison/compare.ts": 56
}
},
"app/workspace/[workspaceId]/integrations/page.tsx": {
"modules": 1253,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/integrations/integrations.tsx": 979,
"apps/sim/blocks/registry.ts": 926,
"apps/sim/triggers/index.ts": 482,
"apps/sim/lib/api/contracts/index.ts": 130,
"apps/sim/stores/workflows/registry/store.ts": 83,
"apps/sim/lib/api/contracts/tools/index.ts": 60,
"apps/sim/hooks/queries/deployments.ts": 59,
"apps/sim/lib/workflows/comparison/compare.ts": 56
}
},
"app/workspace/[workspaceId]/knowledge/[id]/[documentId]/page.tsx": {
"modules": 1460,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/knowledge/[id]/[documentId]/document.tsx": 1183,
"apps/sim/triggers/registry.ts": 481,
"apps/sim/blocks/registry.ts": 322,
"apps/sim/blocks/registry-maps.ts": 319,
"apps/sim/lib/api/contracts/index.ts": 119,
"apps/sim/lib/api/contracts/tools/index.ts": 60,
"apps/sim/connectors/registry.ts": 53,
"apps/sim/app/workspace/[workspaceId]/components/index.ts": 52
}
},
"app/workspace/[workspaceId]/knowledge/[id]/page.tsx": {
"modules": 1461,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/knowledge/[id]/base.tsx": 1184,
"apps/sim/triggers/registry.ts": 481,
"apps/sim/blocks/registry.ts": 322,
"apps/sim/blocks/registry-maps.ts": 319,
"apps/sim/lib/api/contracts/index.ts": 119,
"apps/sim/lib/api/contracts/tools/index.ts": 60,
"apps/sim/connectors/registry.ts": 53,
"apps/sim/app/workspace/[workspaceId]/components/index.ts": 52
}
},
"app/workspace/[workspaceId]/knowledge/page.tsx": {
"modules": 2091,
"gateways": {
"apps/sim/triggers/registry.ts": 446,
"apps/sim/blocks/registry.ts": 317,
"apps/sim/app/workspace/[workspaceId]/knowledge/prefetch.ts": 249,
"apps/sim/lib/knowledge/application/knowledge-bases.ts": 198,
"apps/sim/app/workspace/[workspaceId]/knowledge/knowledge.tsx": 168,
"apps/sim/lib/auth/index.ts": 158,
"apps/sim/lib/knowledge/orchestration/index.ts": 121,
"apps/sim/lib/knowledge/orchestration/connectors.ts": 116
}
},
"app/workspace/[workspaceId]/layout.tsx": {
"modules": 1957,
"gateways": {
"apps/sim/triggers/registry.ts": 446,
"apps/sim/blocks/registry.ts": 316,
"apps/sim/app/workspace/[workspaceId]/components/workspace-chrome/index.ts": 255,
"apps/sim/app/workspace/[workspaceId]/w/components/sidebar/sidebar.tsx": 247,
"apps/sim/lib/auth/index.ts": 180,
"apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/index.ts": 151,
"apps/sim/lib/api/contracts/index.ts": 109,
"apps/sim/lib/webhooks/providers/index.ts": 99
}
},
"app/workspace/[workspaceId]/logs/page.tsx": {
"modules": 1696,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/logs/logs.tsx": 1421,
"apps/sim/triggers/registry.ts": 481,
"apps/sim/app/workspace/[workspaceId]/logs/components/index.ts": 418,
"apps/sim/app/workspace/[workspaceId]/logs/components/log-details/components/execution-snapshot/index.ts": 366,
"apps/sim/app/workspace/[workspaceId]/w/components/preview/components/preview-editor/index.ts": 322,
"apps/sim/blocks/registry.ts": 318,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/index.ts": 286,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/sub-block.tsx": 256
}
},
"app/workspace/[workspaceId]/page.tsx": {
"modules": 5,
"gateways": {}
},
"app/workspace/[workspaceId]/settings/[section]/layout.tsx": {
"modules": 4,
"gateways": {}
},
"app/workspace/[workspaceId]/settings/[section]/page.tsx": {
"modules": 1977,
"gateways": {
"apps/sim/triggers/registry.ts": 446,
"apps/sim/app/workspace/[workspaceId]/settings/[section]/settings.tsx": 409,
"apps/sim/blocks/registry.ts": 320,
"apps/sim/lib/auth/index.ts": 282,
"apps/sim/lib/api/contracts/index.ts": 106,
"apps/sim/lib/webhooks/providers/index.ts": 99,
"apps/sim/lib/api/contracts/tools/index.ts": 59,
"apps/sim/lib/workflows/lifecycle.ts": 48
}
},
"app/workspace/[workspaceId]/settings/billing/credit-usage/layout.tsx": {
"modules": 4,
"gateways": {}
},
"app/workspace/[workspaceId]/settings/billing/credit-usage/page.tsx": {
"modules": 1573,
"gateways": {
"apps/sim/lib/auth/index.ts": 1445,
"apps/sim/triggers/index.ts": 447,
"apps/sim/blocks/registry.ts": 330,
"apps/sim/blocks/registry-maps.ts": 327,
"apps/sim/lib/api/contracts/index.ts": 122,
"apps/sim/lib/webhooks/providers/index.ts": 99,
"apps/sim/stores/workflows/registry/store.ts": 71,
"apps/sim/lib/api/contracts/tools/index.ts": 60
}
},
"app/workspace/[workspaceId]/settings/layout.tsx": {
"modules": 3,
"gateways": {}
},
"app/workspace/[workspaceId]/settings/page.tsx": {
"modules": 1,
"gateways": {}
},
"app/workspace/[workspaceId]/settings/secrets/[credentialId]/page.tsx": {
"modules": 1283,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/settings/secrets/[credentialId]/secret-detail.tsx": 1282,
"apps/sim/app/workspace/[workspaceId]/components/credential-detail/index.ts": 993,
"apps/sim/components/permissions/index.ts": 980,
"apps/sim/components/permissions/add-people-modal.tsx": 971,
"apps/sim/app/workspace/[workspaceId]/providers/workspace-permissions-provider.tsx": 969,
"apps/sim/triggers/registry.ts": 481,
"apps/sim/blocks/registry.ts": 333,
"apps/sim/blocks/registry-maps.ts": 330
}
},
"app/workspace/[workspaceId]/skills/[skillId]/page.tsx": {
"modules": 1374,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/skills/[skillId]/skill-detail.tsx": 1373,
"apps/sim/triggers/registry.ts": 481,
"apps/sim/blocks/registry.ts": 332,
"apps/sim/blocks/registry-maps.ts": 330,
"apps/sim/lib/api/contracts/index.ts": 126,
"apps/sim/app/workspace/[workspaceId]/skills/components/skill-fields/index.ts": 89,
"apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-field.tsx": 86,
"apps/sim/lib/api/contracts/tools/index.ts": 60
}
},
"app/workspace/[workspaceId]/skills/new/page.tsx": {
"modules": 1372,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/skills/new/skill-create.tsx": 1371,
"apps/sim/triggers/registry.ts": 481,
"apps/sim/blocks/registry.ts": 332,
"apps/sim/blocks/registry-maps.ts": 330,
"apps/sim/lib/api/contracts/index.ts": 126,
"apps/sim/app/workspace/[workspaceId]/skills/components/skill-fields/index.ts": 89,
"apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-field.tsx": 86,
"apps/sim/lib/api/contracts/tools/index.ts": 60
}
},
"app/workspace/[workspaceId]/skills/page.tsx": {
"modules": 1236,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/skills/skills.tsx": 962,
"apps/sim/app/workspace/[workspaceId]/integrations/components/showcase-with-explore/index.ts": 950,
"apps/sim/blocks/registry.ts": 938,
"apps/sim/blocks/registry-maps.ts": 936,
"apps/sim/triggers/index.ts": 482,
"apps/sim/lib/api/contracts/index.ts": 135,
"apps/sim/stores/workflows/registry/store.ts": 84,
"apps/sim/hooks/queries/deployments.ts": 60
}
},
"app/workspace/[workspaceId]/tables/[tableId]/page.tsx": {
"modules": 2186,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/tables/[tableId]/table.tsx": 574,
"apps/sim/triggers/registry.ts": 446,
"apps/sim/app/workspace/[workspaceId]/w/components/preview/index.ts": 328,
"apps/sim/lib/auth/index.ts": 302,
"apps/sim/blocks/registry.ts": 301,
"apps/sim/app/workspace/[workspaceId]/w/components/preview/components/preview-editor/index.ts": 286,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/index.ts": 259,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/sub-block.tsx": 230
}
},
"app/workspace/[workspaceId]/tables/page.tsx": {
"modules": 1767,
"gateways": {
"apps/sim/triggers/registry.ts": 446,
"apps/sim/blocks/registry.ts": 327,
"apps/sim/lib/auth/index.ts": 296,
"apps/sim/app/workspace/[workspaceId]/tables/tables.tsx": 121,
"apps/sim/lib/api/contracts/index.ts": 111,
"apps/sim/lib/webhooks/providers/index.ts": 99,
"apps/sim/lib/api/contracts/tools/index.ts": 60,
"apps/sim/app/workspace/[workspaceId]/components/index.ts": 50
}
},
"app/workspace/[workspaceId]/upgrade/page.tsx": {
"modules": 263,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/upgrade/upgrade.tsx": 256,
"apps/sim/app/workspace/[workspaceId]/upgrade/hooks/index.ts": 210,
"apps/sim/lib/billing/client/upgrade.ts": 205,
"apps/sim/hooks/queries/organization.ts": 201,
"apps/sim/hooks/queries/workspace.ts": 192,
"apps/sim/lib/api/contracts/index.ts": 190,
"apps/sim/lib/api/contracts/tools/index.ts": 61,
"apps/sim/lib/api/contracts/v1/index.ts": 38
}
},
"app/workspace/[workspaceId]/w/[workflowId]/layout.tsx": {
"modules": 2145,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/error/index.tsx": 2144,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/index.ts": 540,
"apps/sim/triggers/registry.ts": 481,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/index.ts": 458,
"apps/sim/blocks/registry.ts": 318,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/index.ts": 285,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/index.ts": 141,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/sub-block.tsx": 134
}
},
"app/workspace/[workspaceId]/w/[workflowId]/page.tsx": {
"modules": 2172,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx": 2171,
"apps/sim/triggers/registry.ts": 481,
"apps/sim/blocks/registry.ts": 318,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/index.ts": 305,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/index.ts": 267,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/index.ts": 224,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/index.ts": 140,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/sub-block.tsx": 133
}
},
"app/workspace/[workspaceId]/w/page.tsx": {
"modules": 2145,
"gateways": {
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/index.ts": 904,
"apps/sim/triggers/registry.ts": 481,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/index.ts": 458,
"apps/sim/blocks/registry.ts": 318,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/index.ts": 285,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/index.ts": 141,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/sub-block.tsx": 134,
"apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/error/index.tsx": 133
}
},
"app/workspace/layout.tsx": {
"modules": 1194,
"gateways": {
"apps/sim/app/workspace/providers/socket-provider.tsx": 1184,
"apps/sim/triggers/registry.ts": 481,
"apps/sim/blocks/registry.ts": 333,
"apps/sim/blocks/registry-maps.ts": 330,
"apps/sim/lib/api/contracts/index.ts": 139,
"apps/sim/stores/workflows/registry/store.ts": 63,
"apps/sim/hooks/queries/deployments.ts": 60,
"apps/sim/lib/api/contracts/tools/index.ts": 60
}
},
"app/workspace/page.tsx": {
"modules": 1188,
"gateways": {
"apps/sim/lib/auth/stale-session-recovery.ts": 959,
"apps/sim/triggers/index.ts": 482,
"apps/sim/blocks/registry.ts": 333,
"apps/sim/blocks/registry-maps.ts": 330,
"apps/sim/lib/api/contracts/index.ts": 138,
"apps/sim/stores/workflows/registry/store.ts": 62,
"apps/sim/lib/api/contracts/tools/index.ts": 60,
"apps/sim/hooks/queries/deployments.ts": 56
}
}
}
}
scripts/check-tool-registry-boundary.ts
+388 -31
View File
@@ -1,6 +1,7 @@
#!/usr/bin/env bun
/**
* Fails if a workspace route can reach the executable tool registry.
* Fails if a workspace route can reach the executable tool registry, or if any
* route's module graph grows past its recorded baseline.
*
* `@/tools/registry` is a barrel over 4,300+ tools whose `ToolConfig`s hold
* closures (`request.headers`, `transformResponse`, `directExecution`). Those
@@ -19,11 +20,21 @@
* `formatParameterLabel`. Neither import looks remotely suspicious at the call
* site, which is why this is a lint and not a convention.
*
* The registry is only the loudest instance of the problem. The same walk yields
* each entry's exact module count, so `--check` additionally ratchets those
* counts against `check-tool-registry-boundary.baseline.json` (mirroring
* `check-react-query-patterns.ts`): an entry may not exceed its recorded size by
* more than `max(25 modules, 2%)`. A regression is reported with the dominator
* chain that grew — the import edge every one of the new modules must pass
* through — because a bare number is not actionable.
*
* Usage:
* bun run scripts/check-tool-registry-boundary.ts
* bun run scripts/check-tool-registry-boundary.ts --verbose # print counts
* bun run scripts/check-tool-registry-boundary.ts # registry gate only
* bun run scripts/check-tool-registry-boundary.ts --check # + graph-weight ratchet
* bun run scripts/check-tool-registry-boundary.ts --verbose # print counts
* bun run scripts/check-tool-registry-boundary.ts --update-baseline
*/
import { existsSync, readdirSync, readFileSync, statSync } from 'node:fs'
import { existsSync, readdirSync, readFileSync, statSync, writeFileSync } from 'node:fs'
import { dirname, join, relative, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
@@ -74,8 +85,14 @@ const EXTENSIONS = ['.ts', '.tsx', '.js', '.jsx', '.mjs']
* `REQUIRE_RE` matters for the same reason: this codebase uses lazy
* `require('@/…')` to break import cycles (`tools/params.ts` reaches `@/blocks`
* that way), and those edges are as real as static ones.
*
* In `IMPORT_RE` the `from` clause is optional AND lazily optional (`??`). A
* greedy `?` tries to match the clause before trying to skip it, so a bare
* side-effect import (`import '@/executor'`) was swallowed as the prefix of the
* *next* statement's `from`: that edge was dropped and the next one attributed to
* the wrong importer.
*/
const IMPORT_RE = /(?:^|\n)\s*import\s+(?!type\b)(?:[\s\S]*?from\s*)?['"]([^'"]+)['"]/g
const IMPORT_RE = /(?:^|\n)\s*import\s+(?!type\b)(?:[\s\S]*?from\s*)??['"]([^'"]+)['"]/g
const REEXPORT_RE =
/(?:^|\n)\s*export\s+(?!type\b)(?:\*(?:\s+as\s+[\w$]+)?|\{[\s\S]*?\})\s*from\s*['"]([^'"]+)['"]/g
const DYNAMIC_IMPORT_RE = /\bimport\s*\(\s*['"]([^'"]+)['"]\s*\)/g
@@ -105,41 +122,75 @@ function resolveSpecifier(specifier: string, importer: string): string | null {
return null
}
const NO_DEPS: readonly string[] = []
/**
* Resolved value-import edges out of one file.
*
* Memoized across entries: the 34 entries overlap heavily (every route drags in
* the same shell), so without this each file is re-read and re-scanned once per
* entry that reaches it. The cache is what pays for the dominator analysis added
* below — the whole check got faster, not slower.
*/
const depsCache = new Map<string, readonly string[]>()
function depsOf(file: string): readonly string[] {
const cached = depsCache.get(file)
if (cached) return cached
let source: string
try {
source = readFileSync(file, 'utf8')
} catch {
depsCache.set(file, NO_DEPS)
return NO_DEPS
}
const deps: string[] = []
const seen = new Set<string>()
for (const pattern of [IMPORT_RE, REEXPORT_RE, DYNAMIC_IMPORT_RE, REQUIRE_RE]) {
pattern.lastIndex = 0
let match = pattern.exec(source)
while (match !== null) {
const resolved = resolveSpecifier(match[1], file)
if (resolved && !seen.has(resolved)) {
seen.add(resolved)
deps.push(resolved)
}
match = pattern.exec(source)
}
}
depsCache.set(file, deps)
return deps
}
interface Walk {
entry: string
reachable: Set<string>
importedBy: Map<string, string>
}
/**
* Breadth-first so `importedBy` records the *shortest* path to each module —
* a depth-first parent chain reports whatever winding route the stack happened
* to take, which reads as noise in a failure message.
*/
function walk(entry: string): Walk {
const reachable = new Set<string>()
const reachable = new Set<string>([entry])
const importedBy = new Map<string, string>()
const queue = [entry]
reachable.add(entry)
while (queue.length > 0) {
const file = queue.pop() as string
let source: string
try {
source = readFileSync(file, 'utf8')
} catch {
continue
}
for (const pattern of [IMPORT_RE, REEXPORT_RE, DYNAMIC_IMPORT_RE, REQUIRE_RE]) {
pattern.lastIndex = 0
let match = pattern.exec(source)
while (match !== null) {
const resolved = resolveSpecifier(match[1], file)
if (resolved && !reachable.has(resolved)) {
reachable.add(resolved)
importedBy.set(resolved, file)
queue.push(resolved)
}
match = pattern.exec(source)
}
for (let head = 0; head < queue.length; head++) {
const file = queue[head]
for (const resolved of depsOf(file)) {
if (reachable.has(resolved)) continue
reachable.add(resolved)
importedBy.set(resolved, file)
queue.push(resolved)
}
}
return { reachable, importedBy }
return { entry, reachable, importedBy }
}
/** Walks parent links back to the entry so the offending edge is obvious. */
@@ -153,9 +204,234 @@ function explainChain({ importedBy }: Walk, target: string): string[] {
return chain.reverse()
}
const BASELINE_PATH = join(SCRIPT_DIR, 'check-tool-registry-boundary.baseline.json')
/**
* A graph may grow by `max(TOLERANCE_MODULES, TOLERANCE_PERCENT%)` before failing.
*
* Both halves are needed. A pure percentage lets the 2,186-module workflow route
* absorb 40 modules while pinning the 263-module upgrade route to 5, which would
* fail on an ordinary component addition. A pure absolute number is the same
* trade in reverse. The floor is sized so adding a feature's worth of components
* and hooks is free, while every regression this guard has actually seen — the
* `listTables` prefetch at +444, the registry at +4,700 — is far outside it.
*/
const TOLERANCE_MODULES = 25
const TOLERANCE_PERCENT = 2
/** Smallest dominated subtree worth naming as a gateway in the baseline. */
const GATEWAY_MIN_MODULES = 30
/** Gateways recorded per entry, largest first. */
const GATEWAY_LIMIT = 8
interface BaselineEntry {
modules: number
/** Module → number of modules reachable *only* through it. See `gatewaysFor`. */
gateways: Record<string, number>
}
interface Baseline {
generatedFrom: string
tolerance: { modules: number; percent: number }
entries: Record<string, BaselineEntry>
}
function allowanceFor(baselineModules: number): number {
return Math.max(TOLERANCE_MODULES, Math.ceil((baselineModules * TOLERANCE_PERCENT) / 100))
}
interface Dominators {
/** Immediate dominator of each module; the entry maps to itself. */
idom: Map<string, string>
/** Size of each module's dominator subtree — its exclusive cost to this entry. */
weight: Map<string, number>
}
/**
* Dominator tree of the entry's import graph (Cooper–Harvey–Kennedy).
*
* The point is the weight: a module's dominator-subtree size is exactly how many
* modules would leave the graph if its incoming edge were cut. That turns "this
* page gained 444 modules" into "this page gained 444 modules through
* `lib/table/index.ts`", which names the import to delete.
*
* Computed lazily — only for entries that regress, plus every entry during
* `--update-baseline`.
*/
function dominators({ entry, reachable }: Walk): Dominators {
const succ = new Map<string, string[]>()
for (const file of reachable) {
succ.set(
file,
depsOf(file).filter((dep) => reachable.has(dep))
)
}
// Iterative DFS postorder, then reverse it for the RPO numbering the
// algorithm's `intersect` walks against.
const postorder: string[] = []
const visited = new Set<string>([entry])
const stack: Array<{ node: string; next: number }> = [{ node: entry, next: 0 }]
while (stack.length > 0) {
const frame = stack[stack.length - 1]
const children = succ.get(frame.node) as string[]
if (frame.next < children.length) {
const child = children[frame.next++]
if (!visited.has(child)) {
visited.add(child)
stack.push({ node: child, next: 0 })
}
} else {
postorder.push(frame.node)
stack.pop()
}
}
const rpo = [...postorder].reverse()
const rpoIndex = new Map<string, number>()
rpo.forEach((node, index) => rpoIndex.set(node, index))
const preds = new Map<string, string[]>()
for (const [file, children] of succ) {
if (!rpoIndex.has(file)) continue
for (const child of children) {
if (!rpoIndex.has(child)) continue
const list = preds.get(child)
if (list) list.push(file)
else preds.set(child, [file])
}
}
const idom = new Map<string, string>([[entry, entry]])
const intersect = (a: string, b: string): string => {
let left = a
let right = b
while (left !== right) {
while ((rpoIndex.get(left) as number) > (rpoIndex.get(right) as number))
left = idom.get(left) as string
while ((rpoIndex.get(right) as number) > (rpoIndex.get(left) as number))
right = idom.get(right) as string
}
return left
}
let changed = true
while (changed) {
changed = false
for (let i = 1; i < rpo.length; i++) {
const node = rpo[i]
let candidate: string | null = null
for (const pred of preds.get(node) ?? []) {
if (!idom.has(pred)) continue
candidate = candidate === null ? pred : intersect(candidate, pred)
}
if (candidate !== null && idom.get(node) !== candidate) {
idom.set(node, candidate)
changed = true
}
}
}
// A node's dominator parent always has a smaller RPO index, so folding sizes
// from the deepest index upward completes every subtree in one pass.
const weight = new Map<string, number>()
for (const node of rpo) weight.set(node, 1)
for (let i = rpo.length - 1; i >= 1; i--) {
const node = rpo[i]
const parent = idom.get(node)
if (!parent || parent === node) continue
weight.set(parent, (weight.get(parent) as number) + (weight.get(node) as number))
}
return { idom, weight }
}
/**
* The heaviest gateway modules of an entry, collapsed to one per chain.
*
* In a pass-through chain `a → b → c` every link dominates the same subtree, so
* reporting all three says the same thing three times. `weight[idom] > weight + 1`
* keeps only the topmost link of each chain — the branch point nearest the entry,
* which is the edge a developer can actually delete.
*/
function gatewaysFor(walkResult: Walk, doms: Dominators): Array<[string, number]> {
const gateways: Array<[string, number]> = []
for (const [node, size] of doms.weight) {
if (node === walkResult.entry || size < GATEWAY_MIN_MODULES) continue
const parent = doms.idom.get(node)
if (!parent || parent === node) continue
if (parent !== walkResult.entry && (doms.weight.get(parent) as number) <= size + 1) continue
gateways.push([relative(ROOT, node), size])
}
gateways.sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0]))
return gateways.slice(0, GATEWAY_LIMIT)
}
/** Dominator-tree ancestry of a module: the edges every path from the entry crosses. */
function dominatorChain(walkResult: Walk, doms: Dominators, target: string): string[] {
const chain: string[] = []
let current = target
for (let guard = 0; guard < 10_000; guard++) {
chain.push(relative(ROOT, current))
if (current === walkResult.entry) break
const parent = doms.idom.get(current)
if (!parent || parent === current) break
current = parent
}
return chain.reverse()
}
function loadBaseline(): Baseline | null {
try {
return JSON.parse(readFileSync(BASELINE_PATH, 'utf8')) as Baseline
} catch {
return null
}
}
const RERECORD_COMMAND = 'bun run scripts/check-tool-registry-boundary.ts --update-baseline'
const REBASELINE_HINT = `If the growth is intentional, re-record it: ${RERECORD_COMMAND}`
/**
* Reports a regressed entry by naming the gateways that grew, not just the delta.
*
* Baseline gateways are matched by path, so an edge that is new (absent from the
* baseline) and an edge that got heavier are both surfaced, largest growth first.
*/
function reportRegression(entry: string, walkResult: Walk, before: BaselineEntry, after: number) {
const allowance = allowanceFor(before.modules)
console.error(
`\n❌ ${entry} grew to ${after} modules (baseline ${before.modules}, +${after - before.modules}, allowed +${allowance})`
)
const doms = dominators(walkResult)
const growth = gatewaysFor(walkResult, doms)
.map(([module, size]) => ({ module, size, delta: size - (before.gateways[module] ?? 0) }))
.filter((row) => row.delta > 0)
.sort((a, b) => b.delta - a.delta)
if (growth.length === 0) {
console.error(
' No single import edge accounts for it — the growth is spread across many small modules.'
)
console.error(` Compare with --verbose to see which entries moved. ${REBASELINE_HINT}`)
return
}
for (const row of growth.slice(0, 3)) {
console.error(` +${row.delta} modules reach it only via ${row.module} (${row.size} total):`)
const chain = dominatorChain(walkResult, doms, join(ROOT, row.module))
for (const step of chain) console.error(` ${step}`)
}
console.error(` ${REBASELINE_HINT}`)
}
function main() {
const verbose = process.argv.includes('--verbose')
const check = process.argv.includes('--check')
const update = process.argv.includes('--update-baseline')
const failures: string[] = []
let ratchetFailures = 0
const entryRoot = join(APP, ENTRY_ROOT)
if (!existsSync(entryRoot)) {
@@ -170,9 +446,10 @@ function main() {
process.exit(1)
}
const walked = new Map<string, Walk>()
for (const entry of entries) {
const entryPath = join(APP, entry)
const result = walk(entryPath)
const result = walk(join(APP, entry))
walked.set(entry, result)
if (result.reachable.has(FORBIDDEN)) {
failures.push(entry)
console.error(`\n❌ ${entry} can reach @/tools/registry via:`)
@@ -184,6 +461,83 @@ function main() {
}
}
if (update) {
const baseline: Baseline = {
generatedFrom: `${ENTRY_ROOT} page/layout module graphs`,
tolerance: { modules: TOLERANCE_MODULES, percent: TOLERANCE_PERCENT },
entries: {},
}
for (const entry of entries) {
const result = walked.get(entry) as Walk
baseline.entries[entry] = {
modules: result.reachable.size,
gateways: Object.fromEntries(gatewaysFor(result, dominators(result))),
}
}
writeFileSync(BASELINE_PATH, `${JSON.stringify(baseline, null, 2)}\n`)
console.log(
`✓ Baseline written for ${entries.length} entries: ${relative(ROOT, BASELINE_PATH)}`
)
process.exit(failures.length > 0 ? 1 : 0)
}
if (check) {
const baseline = loadBaseline()
if (!baseline) {
console.error(
`\n❌ Missing ${relative(ROOT, BASELINE_PATH)}. Refusing to pass without a ratchet — ` +
'generate it with --update-baseline.'
)
process.exit(1)
}
const shrunk: string[] = []
const unbaselined: string[] = []
let regressed = 0
for (const entry of entries) {
const result = walked.get(entry) as Walk
const before = baseline.entries[entry]
if (!before) {
unbaselined.push(`${entry} (${result.reachable.size} modules)`)
continue
}
const after = result.reachable.size
if (after > before.modules + allowanceFor(before.modules)) {
regressed++
reportRegression(entry, result, before, after)
} else if (after < before.modules - allowanceFor(before.modules)) {
shrunk.push(`${entry}: ${before.modules} → ${after} (−${before.modules - after})`)
}
}
const removed = Object.keys(baseline.entries).filter((entry) => !entries.includes(entry))
if (shrunk.length > 0) {
console.log(`\nℹ ${shrunk.length} entr(ies) shrank below baseline:`)
for (const line of shrunk) console.log(` ${line}`)
console.log(` Lock the win in, or it can be spent again: ${RERECORD_COMMAND}`)
}
if (unbaselined.length > 0) {
console.log(`\nℹ ${unbaselined.length} new entr(ies) not yet in the baseline (unratcheted):`)
for (const line of unbaselined) console.log(` ${line}`)
console.log(` ${REBASELINE_HINT}`)
}
if (removed.length > 0) {
console.log(`\nℹ ${removed.length} baseline entr(ies) no longer exist: ${removed.join(', ')}`)
}
if (regressed > 0) {
ratchetFailures = regressed
console.error(
`\n${regressed} route(s) exceeded their module-graph baseline by more than max(${TOLERANCE_MODULES}, ${TOLERANCE_PERCENT}%).`
)
console.error(
'Every module in a page graph is parsed and shipped, so this is a real page-weight cost.'
)
}
}
if (failures.length > 0) {
console.error(
`\n${failures.length} route(s) reach the executable tool registry, which adds ~4,700 modules to each.`
@@ -195,10 +549,13 @@ function main() {
'(outputs), or `@/tools/tool-ids` (existence/resolution). Only code that executes a tool'
)
console.error('may import `getTool`. See .agents/skills/tool-registry-boundary/SKILL.md.')
process.exit(1)
}
if (failures.length > 0 || ratchetFailures > 0) process.exit(1)
console.log(`✓ tool registry stays out of ${entries.length} workspace page/layout graphs`)
if (check)
console.log(`✓ ${entries.length} page/layout graphs within their module-count baseline`)
}
main()