perf(tools): read tool metadata instead of the registry on client paths (#6155)

* perf(tools): read tool metadata instead of the registry on client paths

Cuts the last four edges that pulled `@/tools/registry` into the workspace
shell. Every workspace route drops ~4,700 modules:

  route                before   after
  /w (canvas)           6,592   1,908   -71%
  /logs                 6,227   1,543   -75%
  /tables               5,903   1,217   -79%
  /files                5,996   1,310   -78%
  workspace layout      5,751   1,063   -82%

Dev cold compile of the canvas, n=3, cache cleared between runs:

  before   32.3s / 31.4s / 30.1s   RSS 9.0-12.5 GB
  after    22.4s / 22.2s / 21.6s   RSS 7.8-9.2 GB

That lands where the `dev:minimal` escape hatch measured (20.0s / 6.7 GB)
without its downside — `dev:minimal` swaps in curated registries that drop ~250
services, whereas this keeps every tool working.

Rewired:
  - `block-outputs`  -> `getToolOutputsMetadata` (needed `outputs`)
  - `serializer`     -> `getToolParams`          (needed `params`)
  - `validation`     -> `hasToolId`              (needed existence only)
  - `tools/params`   -> `getToolMetadata`        (needed `params`, `oauth`, `name`)

`tools/params.ts` was the stubborn one: `mcp-dynamic-args.tsx` imports only
`formatParameterLabel` from it, so the whole registry rode in behind a string
helper — the same shape as the `mergeToolParameters` edge cut earlier.

Adds a third generated artifact, `tool-ids.ts` (~110 KB). Resolution needs only
the key set, so `@/tools/metadata` and `@/tools/metadata-outputs` both resolve
through it and stay independent of each other, and an existence check costs
~110 KB instead of ~4 MB.

Behaviour preservation was the risk here: `getTool` resolves an unversioned name
onto its newest version, and a plain key lookup would have silently reported 246
versioned tools as missing. `resolveToolId` is reproduced against the id set and
differentially tested — 4,404 probes (every id, every stripped base name, and an
unknown) comparing old vs new resolution and existence: 0 mismatches.

`ToolWithParameters.toolConfig` and `SubBlocksForToolInput.toolConfig` narrow
from `ToolConfig` to `ToolMetadata`. The only external reader is
`tool-input.tsx`, which uses `.name`.

* docs(tools): point the boundary skill at the three metadata modules

The skill still routed `hasToolMetadata` and `getToolIds` to `@/tools/metadata`,
but this PR moved id resolution into `@/tools/tool-ids`. Left as-is it would
send the next caller to the 4 MB module for an existence check that costs
110 KB — the exact mistake the skill exists to prevent.

Also records the two properties a caller can silently get wrong: lookups guard
with `Object.hasOwn` (a bare bracket lookup returns inherited prototype members),
and they resolve unversioned names (246 tools are versioned, and a plain lookup
reports them missing rather than crashing).

* fix(tools): cut the settings-route registry edge and fix serializer test mocks

Two findings from review, both real.

The settings route still reached the registry:

  settings/[section]/page.tsx -> settings.tsx -> (dynamic import)
  ee/access-control/components/access-control.tsx -> group-detail.tsx
  -> tools/utils.ts -> tools/registry.ts

It reads `getTool(id)?.name` — metadata — so it moves to `getToolMetadata`.
The earlier audit missed it because it walked only from the canvas route, and
the edge hides behind a dynamic `import()` that a static walk skips.

Serializer tests mocked the wrong module. `Serializer` now reads params via
`getToolParams` from `@/tools/metadata`, but the tests still only mocked
`@/tools/utils`, so they controlled nothing and passed because the real
generated artifacts happen to agree with the fixtures.

Adds `toolsMetadataMock` to `@sim/testing/mocks`, backed by the same
`mockToolConfigs` as `toolsUtilsMock` so a test mocking both sees one consistent
tool universe, and mocks it in the three serializer suites.

Verified the mock is now load-bearing: pointing it at a sentinel param makes the
three user-only-required validation tests fail, and restoring it returns all 110
serializer tests to green. Before this they passed either way.

* fix(tools): freeze the tool id array handed out by getToolIds

`getToolIds()` returned the module's internal array by reference, so a caller
doing `getToolIds().sort()` would reorder it in place and silently corrupt every
later lookup — the in-place-mutation footgun `.claude/rules/sim-react-performance.md`
calls out.

Frozen rather than copied: the array is consumed in loops, so copying would
allocate on every call. Freezing makes the mutation throw instead of corrupt, and
`[...getToolIds()].sort()` still works. Return type is now `readonly string[]`,
so the mistake is a compile error rather than a runtime surprise.

No caller mutates it today; this is closing the hole, not fixing a live bug.

* test(tools): enforce that the two tool-id resolvers never diverge

`resolveToolId` now exists twice on purpose — `@/tools/utils` resolves against
the live registry (so a tool added before regeneration still resolves at
runtime), `@/tools/tool-ids` against the generated id list (so client code
resolves without importing 4,300 tools). Nothing structurally kept them in step;
a change to versioning logic in one would silently drift from the other.

`tool-metadata:check` now asserts they agree across every id, every stripped
base name, and an unknown — 4,404 probes — and only after the staleness check
passes, so a missing regeneration reports as staleness rather than as drift.
Verified it fails: breaking resolution for `gmail*` exits 1; restoring it passes.

It cannot live in a vitest suite. `vitest.setup.ts` globally mocks
`@/tools/registry` to an empty map, so `getTool` resolves nothing there — a
parity test written as a spec passes or fails for the wrong reason. Both facts
are recorded where the code is.

Both resolvers stay exported. An earlier pass here un-exported the `@/tools/utils`
one as dead; `tools/utils.server.ts` imports it through a multi-line import that
a grep missed, and `tsc` caught it. Its doc now says which resolver a caller
should reach for instead of leaving two identically-named functions unexplained.
This commit is contained in:
Waleed
2026-08-01 11:27:37 -07:00
committed by GitHub
parent d6e08d38d7
commit 452d82a636
23 changed files with 713 additions and 73 deletions
+25 -8
View File
@@ -19,17 +19,20 @@ You keep the 4,300-tool executable registry out of module graphs that don't exec
| you need | import | notes |
| --- | --- | --- |
| whether a tool id exists | `hasToolMetadata` from `@/tools/metadata` | |
| a tool's params | `getToolParams` / `getToolMetadata` from `@/tools/metadata` | |
| a tool's declared outputs | `getToolOutputsMetadata` from `@/tools/metadata-outputs` | separate module on purpose — see below |
| every tool id | `getToolIds` from `@/tools/metadata` | |
| whether a tool id exists | `hasToolId` from `@/tools/tool-ids` | ~110 KB — the cheapest module |
| to resolve an unversioned name | `resolveToolId` from `@/tools/tool-ids` | |
| every tool id | `getToolIds` from `@/tools/tool-ids` | |
| a tool's params | `getToolParams` / `getToolMetadata` from `@/tools/metadata` | ~4 MB |
| a tool's declared outputs | `getToolOutputsMetadata` from `@/tools/metadata-outputs` | ~4 MB, separate on purpose |
| to **execute** a tool | `getTool` from `@/tools/utils`, or `@/tools/utils.server` | server paths only |
Outputs live in their own module because they are roughly two thirds of the generated data and have a single consumer. Importing `@/tools/metadata` must never pull them — do not "helpfully" re-export one from the other.
Three modules, cheapest first. Ids are their own artifact because resolution and existence checks need only the key set; outputs are their own because they are the larger half of the data with a single consumer. `@/tools/metadata` and `@/tools/metadata-outputs` both resolve ids through `@/tools/tool-ids`, which is what keeps them independent of each other — do not "helpfully" re-export one from another, or every caller pays for all three.
All lookups guard with `Object.hasOwn`. `JSON.parse` yields an object with the normal prototype, so a bare bracket lookup returns inherited members: `getToolMetadata('constructor')` returned a *function* typed as tool metadata before that was fixed.
## The generated artifacts
`apps/sim/tools/generated/tool-metadata.ts` and `tool-outputs.ts` are produced by `scripts/sync-tool-metadata.ts`:
`apps/sim/tools/generated/tool-ids.ts`, `tool-metadata.ts` and `tool-outputs.ts` are produced by `scripts/sync-tool-metadata.ts`:
```bash
bun run tool-metadata:generate # after adding/changing a tool
@@ -43,6 +46,20 @@ Three non-obvious properties, each of which was measured and is easy to undo by
- **The data is a JSON string parsed at runtime, not an imported `.json` and not an object literal.** With `resolveJsonModule` (which this repo enables), a `.json` import makes TypeScript infer a literal type for all 4,300+ entries and takes `tsc --noEmit` from **12.6s to 8m07s** — a 38x regression. An ambient `declare module` does *not* short-circuit it, and an object literal costs the same. A single string literal is one cheap token for both the compiler and the bundler, and `JSON.parse` beats evaluating the equivalent literal at runtime. Do not "clean this up" into a `.json` import.
- **The generator refuses to emit function values.** If you add a field to `METADATA_FIELDS` that contains a closure, generation fails loudly rather than shipping executable config to the client. `hosting` and `schemaEnrichment` are excluded for exactly this reason (`hosting.enabled`, `pricing`, and `enrichSchema` are functions) — they are server-only.
- **Empty param entries are stripped.** The registry contains one (`stt_deepgram_v2`), which crashes callers that read `param.type` while iterating.
- **Lookups resolve versions.** `getTool` maps an unversioned name onto the newest version, and 246 tools are versioned. A plain key lookup would silently report them missing — a quiet correctness bug, not a crash. `resolveToolId` reproduces that against the id set and is differentially tested against the original.
## Testing code that reads tool metadata
Mock the module the code under test actually reads. `vi.mock('@/tools/utils', () => toolsUtilsMock)` only controls `getTool`; code that reads `params`/`outputs`/`name` goes through `@/tools/metadata`, so mocking `tools/utils` there is a **no-op that still passes** — because the real generated artifacts happen to agree with the mock fixtures. The test looks green while controlling nothing.
```ts
import { blocksMock, toolsMetadataMock, toolsUtilsMock } from '@sim/testing/mocks'
vi.mock('@/tools/utils', () => toolsUtilsMock) // executable lookup
vi.mock('@/tools/metadata', () => toolsMetadataMock) // params / outputs / name
```
Both are backed by the same `mockToolConfigs`, so mocking both gives one consistent tool universe. If you are unsure whether a mock is load-bearing, change a fixture value to a sentinel and confirm the test fails.
## How to verify an edge actually got cut
@@ -59,9 +76,9 @@ Reference points measured on this repo:
| `tools/registry.ts` reachable | ~4,900 |
| `tools/merge-params.ts` (leaf) | 2 |
| `providers/utils.ts` after cutting its `params` edge | 22 |
| `app/workspace/[workspaceId]/w/page.tsx` (canvas) | 6,591, of which 4,689 are the registry |
| `app/workspace/[workspaceId]/w/page.tsx` (canvas) | 6,592 before, 1,908 after |
The canvas route reaches the registry through **four** redundant edges — `providers/utils` (via `tools/params`), `lib/workflows/blocks/block-outputs`, `lib/workflows/sanitization/validation`, and `serializer/index`. Cutting any one alone moves the module count by ~1. They must all be cut before anything improves; measure the route, not the file you edited.
The canvas route reached the registry through **four** redundant edges — `providers/utils` (via `tools/params`), `lib/workflows/blocks/block-outputs`, `lib/workflows/sanitization/validation`, and `serializer/index`. Cutting any one alone moved the module count by ~1. They all had to go before anything improved; measure the route, not the file you edited.
## When adding a new caller
+25 -8
View File
@@ -18,17 +18,20 @@ You keep the 4,300-tool executable registry out of module graphs that don't exec
| you need | import | notes |
| --- | --- | --- |
| whether a tool id exists | `hasToolMetadata` from `@/tools/metadata` | |
| a tool's params | `getToolParams` / `getToolMetadata` from `@/tools/metadata` | |
| a tool's declared outputs | `getToolOutputsMetadata` from `@/tools/metadata-outputs` | separate module on purpose — see below |
| every tool id | `getToolIds` from `@/tools/metadata` | |
| whether a tool id exists | `hasToolId` from `@/tools/tool-ids` | ~110 KB — the cheapest module |
| to resolve an unversioned name | `resolveToolId` from `@/tools/tool-ids` | |
| every tool id | `getToolIds` from `@/tools/tool-ids` | |
| a tool's params | `getToolParams` / `getToolMetadata` from `@/tools/metadata` | ~4 MB |
| a tool's declared outputs | `getToolOutputsMetadata` from `@/tools/metadata-outputs` | ~4 MB, separate on purpose |
| to **execute** a tool | `getTool` from `@/tools/utils`, or `@/tools/utils.server` | server paths only |
Outputs live in their own module because they are roughly two thirds of the generated data and have a single consumer. Importing `@/tools/metadata` must never pull them — do not "helpfully" re-export one from the other.
Three modules, cheapest first. Ids are their own artifact because resolution and existence checks need only the key set; outputs are their own because they are the larger half of the data with a single consumer. `@/tools/metadata` and `@/tools/metadata-outputs` both resolve ids through `@/tools/tool-ids`, which is what keeps them independent of each other — do not "helpfully" re-export one from another, or every caller pays for all three.
All lookups guard with `Object.hasOwn`. `JSON.parse` yields an object with the normal prototype, so a bare bracket lookup returns inherited members: `getToolMetadata('constructor')` returned a *function* typed as tool metadata before that was fixed.
## The generated artifacts
`apps/sim/tools/generated/tool-metadata.ts` and `tool-outputs.ts` are produced by `scripts/sync-tool-metadata.ts`:
`apps/sim/tools/generated/tool-ids.ts`, `tool-metadata.ts` and `tool-outputs.ts` are produced by `scripts/sync-tool-metadata.ts`:
```bash
bun run tool-metadata:generate # after adding/changing a tool
@@ -42,6 +45,20 @@ Three non-obvious properties, each of which was measured and is easy to undo by
- **The data is a JSON string parsed at runtime, not an imported `.json` and not an object literal.** With `resolveJsonModule` (which this repo enables), a `.json` import makes TypeScript infer a literal type for all 4,300+ entries and takes `tsc --noEmit` from **12.6s to 8m07s** — a 38x regression. An ambient `declare module` does *not* short-circuit it, and an object literal costs the same. A single string literal is one cheap token for both the compiler and the bundler, and `JSON.parse` beats evaluating the equivalent literal at runtime. Do not "clean this up" into a `.json` import.
- **The generator refuses to emit function values.** If you add a field to `METADATA_FIELDS` that contains a closure, generation fails loudly rather than shipping executable config to the client. `hosting` and `schemaEnrichment` are excluded for exactly this reason (`hosting.enabled`, `pricing`, and `enrichSchema` are functions) — they are server-only.
- **Empty param entries are stripped.** The registry contains one (`stt_deepgram_v2`), which crashes callers that read `param.type` while iterating.
- **Lookups resolve versions.** `getTool` maps an unversioned name onto the newest version, and 246 tools are versioned. A plain key lookup would silently report them missing — a quiet correctness bug, not a crash. `resolveToolId` reproduces that against the id set and is differentially tested against the original.
## Testing code that reads tool metadata
Mock the module the code under test actually reads. `vi.mock('@/tools/utils', () => toolsUtilsMock)` only controls `getTool`; code that reads `params`/`outputs`/`name` goes through `@/tools/metadata`, so mocking `tools/utils` there is a **no-op that still passes** — because the real generated artifacts happen to agree with the mock fixtures. The test looks green while controlling nothing.
```ts
import { blocksMock, toolsMetadataMock, toolsUtilsMock } from '@sim/testing/mocks'
vi.mock('@/tools/utils', () => toolsUtilsMock) // executable lookup
vi.mock('@/tools/metadata', () => toolsMetadataMock) // params / outputs / name
```
Both are backed by the same `mockToolConfigs`, so mocking both gives one consistent tool universe. If you are unsure whether a mock is load-bearing, change a fixture value to a sentinel and confirm the test fails.
## How to verify an edge actually got cut
@@ -58,9 +75,9 @@ Reference points measured on this repo:
| `tools/registry.ts` reachable | ~4,900 |
| `tools/merge-params.ts` (leaf) | 2 |
| `providers/utils.ts` after cutting its `params` edge | 22 |
| `app/workspace/[workspaceId]/w/page.tsx` (canvas) | 6,591, of which 4,689 are the registry |
| `app/workspace/[workspaceId]/w/page.tsx` (canvas) | 6,592 before, 1,908 after |
The canvas route reaches the registry through **four** redundant edges — `providers/utils` (via `tools/params`), `lib/workflows/blocks/block-outputs`, `lib/workflows/sanitization/validation`, and `serializer/index`. Cutting any one alone moves the module count by ~1. They must all be cut before anything improves; measure the route, not the file you edited.
The canvas route reached the registry through **four** redundant edges — `providers/utils` (via `tools/params`), `lib/workflows/blocks/block-outputs`, `lib/workflows/sanitization/validation`, and `serializer/index`. Cutting any one alone moved the module count by ~1. They all had to go before anything improved; measure the route, not the file you edited.
## When adding a new caller
+25 -8
View File
@@ -14,17 +14,20 @@ You keep the 4,300-tool executable registry out of module graphs that don't exec
| you need | import | notes |
| --- | --- | --- |
| whether a tool id exists | `hasToolMetadata` from `@/tools/metadata` | |
| a tool's params | `getToolParams` / `getToolMetadata` from `@/tools/metadata` | |
| a tool's declared outputs | `getToolOutputsMetadata` from `@/tools/metadata-outputs` | separate module on purpose — see below |
| every tool id | `getToolIds` from `@/tools/metadata` | |
| whether a tool id exists | `hasToolId` from `@/tools/tool-ids` | ~110 KB — the cheapest module |
| to resolve an unversioned name | `resolveToolId` from `@/tools/tool-ids` | |
| every tool id | `getToolIds` from `@/tools/tool-ids` | |
| a tool's params | `getToolParams` / `getToolMetadata` from `@/tools/metadata` | ~4 MB |
| a tool's declared outputs | `getToolOutputsMetadata` from `@/tools/metadata-outputs` | ~4 MB, separate on purpose |
| to **execute** a tool | `getTool` from `@/tools/utils`, or `@/tools/utils.server` | server paths only |
Outputs live in their own module because they are roughly two thirds of the generated data and have a single consumer. Importing `@/tools/metadata` must never pull them — do not "helpfully" re-export one from the other.
Three modules, cheapest first. Ids are their own artifact because resolution and existence checks need only the key set; outputs are their own because they are the larger half of the data with a single consumer. `@/tools/metadata` and `@/tools/metadata-outputs` both resolve ids through `@/tools/tool-ids`, which is what keeps them independent of each other — do not "helpfully" re-export one from another, or every caller pays for all three.
All lookups guard with `Object.hasOwn`. `JSON.parse` yields an object with the normal prototype, so a bare bracket lookup returns inherited members: `getToolMetadata('constructor')` returned a *function* typed as tool metadata before that was fixed.
## The generated artifacts
`apps/sim/tools/generated/tool-metadata.ts` and `tool-outputs.ts` are produced by `scripts/sync-tool-metadata.ts`:
`apps/sim/tools/generated/tool-ids.ts`, `tool-metadata.ts` and `tool-outputs.ts` are produced by `scripts/sync-tool-metadata.ts`:
```bash
bun run tool-metadata:generate # after adding/changing a tool
@@ -38,6 +41,20 @@ Three non-obvious properties, each of which was measured and is easy to undo by
- **The data is a JSON string parsed at runtime, not an imported `.json` and not an object literal.** With `resolveJsonModule` (which this repo enables), a `.json` import makes TypeScript infer a literal type for all 4,300+ entries and takes `tsc --noEmit` from **12.6s to 8m07s** — a 38x regression. An ambient `declare module` does *not* short-circuit it, and an object literal costs the same. A single string literal is one cheap token for both the compiler and the bundler, and `JSON.parse` beats evaluating the equivalent literal at runtime. Do not "clean this up" into a `.json` import.
- **The generator refuses to emit function values.** If you add a field to `METADATA_FIELDS` that contains a closure, generation fails loudly rather than shipping executable config to the client. `hosting` and `schemaEnrichment` are excluded for exactly this reason (`hosting.enabled`, `pricing`, and `enrichSchema` are functions) — they are server-only.
- **Empty param entries are stripped.** The registry contains one (`stt_deepgram_v2`), which crashes callers that read `param.type` while iterating.
- **Lookups resolve versions.** `getTool` maps an unversioned name onto the newest version, and 246 tools are versioned. A plain key lookup would silently report them missing — a quiet correctness bug, not a crash. `resolveToolId` reproduces that against the id set and is differentially tested against the original.
## Testing code that reads tool metadata
Mock the module the code under test actually reads. `vi.mock('@/tools/utils', () => toolsUtilsMock)` only controls `getTool`; code that reads `params`/`outputs`/`name` goes through `@/tools/metadata`, so mocking `tools/utils` there is a **no-op that still passes** — because the real generated artifacts happen to agree with the mock fixtures. The test looks green while controlling nothing.
```ts
import { blocksMock, toolsMetadataMock, toolsUtilsMock } from '@sim/testing/mocks'
vi.mock('@/tools/utils', () => toolsUtilsMock) // executable lookup
vi.mock('@/tools/metadata', () => toolsMetadataMock) // params / outputs / name
```
Both are backed by the same `mockToolConfigs`, so mocking both gives one consistent tool universe. If you are unsure whether a mock is load-bearing, change a fixture value to a sentinel and confirm the test fails.
## How to verify an edge actually got cut
@@ -54,9 +71,9 @@ Reference points measured on this repo:
| `tools/registry.ts` reachable | ~4,900 |
| `tools/merge-params.ts` (leaf) | 2 |
| `providers/utils.ts` after cutting its `params` edge | 22 |
| `app/workspace/[workspaceId]/w/page.tsx` (canvas) | 6,591, of which 4,689 are the registry |
| `app/workspace/[workspaceId]/w/page.tsx` (canvas) | 6,592 before, 1,908 after |
The canvas route reaches the registry through **four** redundant edges — `providers/utils` (via `tools/params`), `lib/workflows/blocks/block-outputs`, `lib/workflows/sanitization/validation`, and `serializer/index`. Cutting any one alone moves the module count by ~1. They must all be cut before anything improves; measure the route, not the file you edited.
The canvas route reached the registry through **four** redundant edges — `providers/utils` (via `tools/params`), `lib/workflows/blocks/block-outputs`, `lib/workflows/sanitization/validation`, and `serializer/index`. Cutting any one alone moved the module count by ~1. They all had to go before anything improved; measure the route, not the file you edited.
## When adding a new caller
@@ -76,7 +76,7 @@ import {
import type { ProviderId } from '@/providers/types'
import { getAllProviderIds, getProviderFromModel } from '@/providers/utils'
import type { ProviderName } from '@/stores/providers'
import { getTool } from '@/tools/utils'
import { getToolMetadata } from '@/tools/metadata'
const logger = createLogger('AccessControlGroupDetail')
@@ -733,7 +733,7 @@ function BlockToolRow({
const checkboxId = `block-${block.type}`
const toolItems = useMemo<DenylistGridItem[]>(
() => (block.tools?.access ?? []).map((id) => ({ id, label: getTool(id)?.name ?? id })),
() => (block.tools?.access ?? []).map((id) => ({ id, label: getToolMetadata(id)?.name ?? id })),
[block.tools?.access]
)
const isExpandable = toolItems.length > 1
@@ -22,7 +22,7 @@ import {
type OutputCondition,
type OutputFieldDefinition,
} from '@/blocks/types'
import { getTool } from '@/tools/utils'
import { getToolOutputsMetadata } from '@/tools/metadata-outputs'
import { getTrigger, isTriggerValid } from '@/triggers'
const logger = createLogger('BlockOutputs')
@@ -681,13 +681,13 @@ export function getToolOutputs(
const toolId = blockConfig.tools.config.tool(params)
if (!toolId) return {}
const toolConfig = getTool(toolId)
if (!toolConfig?.outputs) return {}
const toolOutputs = getToolOutputsMetadata(toolId)
if (!toolOutputs) return {}
if (includeHidden) {
return toolConfig.outputs
return toolOutputs
}
return Object.fromEntries(
Object.entries(toolConfig.outputs).filter(([_, def]) => !isHiddenFromDisplay(def))
Object.entries(toolOutputs).filter(([_, def]) => !isHiddenFromDisplay(def))
)
} catch (error) {
logger.warn('Failed to get tool outputs', { error })
@@ -4,7 +4,7 @@ import { isRecordLike } from '@sim/utils/object'
import { getBlock } from '@/blocks/registry'
import { isCustomTool, isMcpTool } from '@/executor/constants'
import type { BlockState, WorkflowState } from '@/stores/workflows/workflow/types'
import { getTool } from '@/tools/utils'
import { hasToolId } from '@/tools/tool-ids'
const logger = createLogger('WorkflowValidation')
@@ -305,8 +305,7 @@ export function validateToolReference(
if (!isCustomTool(toolId) && !isMcpTool(toolId)) {
// For built-in tools, verify they exist
const tool = getTool(toolId)
if (!tool) {
if (!hasToolId(toolId)) {
return `Block ${blockName || 'unknown'} (${blockType}): references non-existent tool '${toolId}'`
}
}
@@ -7,7 +7,7 @@
* - a deleted *input* on a live custom block no longer leaks its stale value into
* the child `inputMapping` (Bug 1).
*/
import { toolsUtilsMock } from '@sim/testing/mocks'
import { toolsMetadataMock, toolsUtilsMock } from '@sim/testing/mocks'
import { beforeEach, describe, expect, it, vi } from 'vitest'
// Build the custom-block configs INSIDE the factory (hoisted) so no top-level
@@ -33,6 +33,7 @@ vi.mock('@/blocks', async () => {
return { getBlock, getAllBlocks: () => Object.values(mockBlockConfigs) }
})
vi.mock('@/tools/utils', () => toolsUtilsMock)
vi.mock('@/tools/metadata', () => toolsMetadataMock)
import { extractBlockParams, Serializer } from '@/serializer/index'
+2 -1
View File
@@ -5,12 +5,13 @@
* (collectBlockFieldIssues / extractBlockParams) — the single source of truth
* shared by the serializer's required-field validation and the copilot lint.
*/
import { blocksMock, toolsUtilsMock } from '@sim/testing/mocks'
import { blocksMock, toolsMetadataMock, toolsUtilsMock } from '@sim/testing/mocks'
import { describe, expect, it, vi } from 'vitest'
const { svcConfig } = vi.hoisted(() => ({ svcConfig: { value: null as any } }))
vi.mock('@/tools/utils', () => toolsUtilsMock)
vi.mock('@/tools/metadata', () => toolsMetadataMock)
vi.mock('@/blocks', () => ({
...blocksMock,
getBlock: (type: string) => (type === 'svc' ? svcConfig.value : blocksMock.getBlock(type)),
+2 -1
View File
@@ -18,13 +18,14 @@ import {
createMinimalWorkflowState,
createMissingMetadataWorkflow,
} from '@sim/testing/factories'
import { blocksMock, toolsUtilsMock } from '@sim/testing/mocks'
import { blocksMock, toolsMetadataMock, toolsUtilsMock } from '@sim/testing/mocks'
import { describe, expect, it, vi } from 'vitest'
import { Serializer } from '@/serializer/index'
import type { SerializedWorkflow } from '@/serializer/types'
vi.mock('@/blocks', () => blocksMock)
vi.mock('@/tools/utils', () => toolsUtilsMock)
vi.mock('@/tools/metadata', () => toolsMetadataMock)
describe('Serializer', () => {
describe('serializeWorkflow', () => {
+5 -5
View File
@@ -21,7 +21,7 @@ import type { SubBlockConfig } from '@/blocks/types'
import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types'
import type { BlockState, Loop, Parallel } from '@/stores/workflows/workflow/types'
import { generateLoopBlocks, generateParallelBlocks } from '@/stores/workflows/workflow/utils'
import { getTool } from '@/tools/utils'
import { getToolParams } from '@/tools/metadata'
const logger = createLogger('Serializer')
@@ -637,13 +637,13 @@ export function collectBlockFieldIssues(
// Get the tool configuration to check parameter visibility
const toolAccess = blockConfig.tools?.access
const currentToolId = toolAccess?.length > 0 ? selectToolId(blockConfig, params) : null
const currentTool = currentToolId ? getTool(currentToolId) : null
const currentToolParams = currentToolId ? getToolParams(currentToolId) : undefined
// Validate tool parameters (for blocks with tools).
// Lookup contract: a tool param's value lives under its own paramId in `params`.
// Block subBlocks align via either `id === paramId` or `canonicalParamId === paramId`.
if (currentTool) {
Object.entries(currentTool.params || {}).forEach(([paramId, paramConfig]: [string, any]) => {
if (currentToolParams) {
Object.entries(currentToolParams).forEach(([paramId, paramConfig]: [string, any]) => {
if (paramConfig.required && paramConfig.visibility === 'user-only') {
const matchingConfigs =
blockConfig.subBlocks?.filter(
@@ -699,7 +699,7 @@ export function collectBlockFieldIssues(
}
// Validate required subBlocks not covered by tool params (e.g., blocks with empty tools.access)
const validatedByTool = new Set(currentTool ? Object.keys(currentTool.params || {}) : [])
const validatedByTool = new Set(currentToolParams ? Object.keys(currentToolParams) : [])
blockConfig.subBlocks?.forEach((subBlockConfig: SubBlockConfig) => {
if (validatedByTool.has(subBlockConfig.id)) {
File diff suppressed because one or more lines are too long
+3 -1
View File
@@ -1,4 +1,5 @@
import rawOutputs from '@/tools/generated/tool-outputs'
import { resolveToolId } from '@/tools/tool-ids'
import type { ToolConfig } from '@/tools/types'
/**
@@ -28,5 +29,6 @@ const outputs: Record<string, ToolOutputs> = rawOutputs as Record<string, ToolOu
* `Object.hasOwn` rather than a bare lookup — see `getToolMetadata` for why.
*/
export function getToolOutputsMetadata(toolId: string): ToolOutputs | undefined {
return Object.hasOwn(outputs, toolId) ? outputs[toolId] : undefined
const resolved = resolveToolId(toolId)
return Object.hasOwn(outputs, resolved) ? outputs[resolved] : undefined
}
+38 -3
View File
@@ -2,8 +2,9 @@
* @vitest-environment node
*/
import { describe, expect, it } from 'vitest'
import { getToolIds, getToolMetadata, getToolParams, hasToolMetadata } from '@/tools/metadata'
import { getToolMetadata, getToolParams } from '@/tools/metadata'
import { getToolOutputsMetadata } from '@/tools/metadata-outputs'
import { getToolIds, hasToolId, resolveToolId } from '@/tools/tool-ids'
/**
* Guards the properties the generated artifacts are relied on for. The
@@ -22,7 +23,7 @@ describe('generated tool metadata', () => {
})
it('reports unknown tools as absent without throwing', () => {
expect(hasToolMetadata('definitely_not_a_tool')).toBe(false)
expect(hasToolId('definitely_not_a_tool')).toBe(false)
expect(getToolMetadata('definitely_not_a_tool')).toBeUndefined()
expect(getToolParams('definitely_not_a_tool')).toBeUndefined()
expect(getToolOutputsMetadata('definitely_not_a_tool')).toBeUndefined()
@@ -43,10 +44,44 @@ describe('generated tool metadata', () => {
expect(getToolMetadata(key)).toBeUndefined()
expect(getToolParams(key)).toBeUndefined()
expect(getToolOutputsMetadata(key)).toBeUndefined()
expect(hasToolMetadata(key)).toBe(false)
expect(hasToolId(key)).toBe(false)
}
)
/**
* `getTool` resolves an unversioned name onto the newest version, and callers
* migrated off it depend on that. A plain key lookup would silently report
* versioned tools as missing.
*/
describe('version resolution', () => {
/**
* Only a versioned id whose base name is *not* itself registered exercises
* resolution — where both exist, the base name resolves to itself.
*/
const versionedId = getToolIds().find(
(id) => /_v[2-9]\d*$/.test(id) && !getToolIds().includes(id.replace(/_v\d+$/, ''))
)
it('has at least one versioned tool to exercise', () => {
expect(versionedId).toBeDefined()
})
it('maps an unversioned name onto the newest version', () => {
const baseName = (versionedId as string).replace(/_v\d+$/, '')
expect(getToolIds()).not.toContain(baseName)
expect(resolveToolId(baseName)).toBe(versionedId)
expect(hasToolId(baseName)).toBe(true)
expect(getToolMetadata(baseName)?.id).toBe(getToolMetadata(versionedId as string)?.id)
expect(getToolOutputsMetadata(baseName)).toEqual(
getToolOutputsMetadata(versionedId as string)
)
})
it('returns an unknown name unchanged', () => {
expect(resolveToolId('definitely_not_a_tool')).toBe('definitely_not_a_tool')
})
})
/**
* The registry contains a null param entry (`stt_deepgram_v2`), which crashes
* any consumer that iterates params unguarded. The generator strips those, so
+7 -12
View File
@@ -1,4 +1,5 @@
import rawMetadata from '@/tools/generated/tool-metadata'
import { resolveToolId } from '@/tools/tool-ids'
import type { OAuthConfig, ToolConfig } from '@/tools/types'
/**
@@ -15,7 +16,10 @@ import type { OAuthConfig, ToolConfig } from '@/tools/types'
*
* Outputs live in `@/tools/metadata-outputs`, not here: they are the larger half
* of the data and have a single consumer, so keeping them in a separate module
* means callers that only need params don't pay for them.
* means callers that only need params don't pay for them. Callers that need
* neither should use `@/tools/tool-ids`, which is ~40x smaller again.
*
* Lookups resolve unversioned names the same way `getTool` does.
*/
export interface ToolMetadata {
id: string
@@ -33,16 +37,6 @@ export interface ToolMetadata {
*/
const metadata: Record<string, ToolMetadata> = rawMetadata as Record<string, ToolMetadata>
/** Every registered tool id, including versioned variants. */
export function getToolIds(): string[] {
return Object.keys(metadata)
}
/** Whether `toolId` names a built-in tool. Cheaper than resolving its metadata. */
export function hasToolMetadata(toolId: string): boolean {
return Object.hasOwn(metadata, toolId)
}
/**
* Serializable metadata for a built-in tool, or `undefined` if unknown.
*
@@ -52,7 +46,8 @@ export function hasToolMetadata(toolId: string): boolean {
* metadata.
*/
export function getToolMetadata(toolId: string): ToolMetadata | undefined {
return Object.hasOwn(metadata, toolId) ? metadata[toolId] : undefined
const resolved = resolveToolId(toolId)
return Object.hasOwn(metadata, resolved) ? metadata[resolved] : undefined
}
/** Declared parameters for a built-in tool, or `undefined` if unknown. */
+6 -4
View File
@@ -1,5 +1,6 @@
import { afterAll, describe, expect, it, vi } from 'vitest'
import { mergeToolParameters } from '@/tools/merge-params'
import * as toolMetadata from '@/tools/metadata'
import {
createExecutionToolSchema,
createLLMToolSchema,
@@ -15,7 +16,6 @@ import {
validateToolParameters,
} from '@/tools/params'
import type { HttpMethod, ParameterVisibility } from '@/tools/types'
import * as toolsUtils from '@/tools/utils'
const mockToolConfig = {
id: 'test_tool',
@@ -58,11 +58,13 @@ const mockToolConfig = {
/**
* Spy on the real module namespace instead of vi.mock: under `isolate: false`
* `@/tools/params` may already be cached bound to the real `@/tools/utils`
* `@/tools/params` may already be cached bound to the real `@/tools/metadata`
* module, so patching the shared namespace is the only wiring that always
* applies.
*/
const getToolSpy = vi.spyOn(toolsUtils, 'getTool').mockImplementation(((toolId: string) => {
const getToolSpy = vi.spyOn(toolMetadata, 'getToolMetadata').mockImplementation(((
toolId: string
) => {
if (toolId === 'test_tool') {
return mockToolConfig
}
@@ -76,7 +78,7 @@ const getToolSpy = vi.spyOn(toolsUtils, 'getTool').mockImplementation(((toolId:
}
}
return null
}) as unknown as typeof toolsUtils.getTool)
}) as unknown as typeof toolMetadata.getToolMetadata)
afterAll(() => {
getToolSpy.mockRestore()
+5 -5
View File
@@ -18,6 +18,7 @@ import type {
GenerationType,
} from '@/blocks/types'
import { isNonEmpty } from '@/tools/merge-params'
import { getToolMetadata, type ToolMetadata } from '@/tools/metadata'
import { safeAssign } from '@/tools/safe-assign'
import type {
OAuthConfig,
@@ -25,7 +26,6 @@ import type {
ToolConfig,
ToolParameterItemSchema,
} from '@/tools/types'
import { getTool } from '@/tools/utils'
const logger = createLogger('ToolsParams')
type ToolParamDefinition = ToolConfig['params'][string]
@@ -173,7 +173,7 @@ export interface ToolParameterConfig {
}
export interface ToolWithParameters {
toolConfig: ToolConfig
toolConfig: ToolMetadata
allParameters: ToolParameterConfig[]
userInputParameters: ToolParameterConfig[] // Parameters shown to user
requiredParameters: ToolParameterConfig[] // Must be filled by user or LLM
@@ -301,7 +301,7 @@ export function getToolParametersConfig(
blockConfigOverride?: Pick<ToolInputBlockConfig, 'subBlocks'>
): ToolWithParameters | null {
try {
const toolConfig = getTool(toolId)
const toolConfig = getToolMetadata(toolId)
if (!toolConfig) {
logger.warn(`Tool not found: ${toolId}`)
return null
@@ -971,7 +971,7 @@ const EXCLUDED_SUBBLOCK_TYPES = new Set([
])
export interface SubBlocksForToolInput {
toolConfig: ToolConfig
toolConfig: ToolMetadata
subBlocks: BlockSubBlockConfig[]
oauthConfig?: OAuthConfig
}
@@ -992,7 +992,7 @@ export function getSubBlocksForToolInput(
blockConfigOverride?: Pick<ToolInputBlockConfig, 'subBlocks'>
): SubBlocksForToolInput | null {
try {
const toolConfig = getTool(toolId)
const toolConfig = getToolMetadata(toolId)
if (!toolConfig) {
logger.warn(`Tool not found: ${toolId}`)
return null
+75
View File
@@ -0,0 +1,75 @@
import { stripVersionSuffix } from '@sim/utils/string'
import rawToolIds from '@/tools/generated/tool-ids'
/**
* Tool id resolution, without importing the executable registry.
*
* Resolving a tool name and checking whether one exists need only the registry's
* key set — never a `ToolConfig` — so this module carries just the ids (~100 KB
* versus ~4 MB for params or outputs). `@/tools/metadata` and
* `@/tools/metadata-outputs` both resolve through here, which is what keeps them
* independent of each other.
*
* Semantics mirror `resolveToolId`/`getTool` in `@/tools/utils` exactly,
* including returning the input unchanged when nothing matches. See
* `.agents/skills/tool-registry-boundary/SKILL.md`.
*/
/**
* Frozen because {@link getToolIds} hands it out directly. Returning a copy
* would allocate on every call in the loops that consume it; freezing makes an
* in-place `sort()`/`push()` by a caller throw rather than silently corrupt
* every later lookup.
*/
const toolIds: readonly string[] = Object.freeze(rawToolIds)
const toolIdSet = new Set(toolIds)
/**
* Base name -> newest versioned id, built once.
*
* `@/tools/utils` rebuilds this on every unresolved lookup; the id set is static
* at runtime, so caching it is behaviour-identical and drops the repeated O(n)
* scan.
*/
let latestByBaseName: Map<string, string> | null = null
function getLatestByBaseName(): Map<string, string> {
if (latestByBaseName) return latestByBaseName
const versions = new Map<string, { toolId: string; version: number }>()
for (const toolId of toolIds) {
const baseName = stripVersionSuffix(toolId)
const versionMatch = toolId.match(/_v(\d+)$/)
const version = versionMatch ? Number.parseInt(versionMatch[1], 10) : 1
const previous = versions.get(baseName)
if (!previous || version > previous.version) {
versions.set(baseName, { toolId, version })
}
}
latestByBaseName = new Map()
for (const [baseName, { toolId }] of versions) {
latestByBaseName.set(baseName, toolId)
}
return latestByBaseName
}
/** Every registered tool id, including versioned variants. Frozen — copy before sorting. */
export function getToolIds(): readonly string[] {
return toolIds
}
/**
* Resolves a tool name to its registered id, mapping an unversioned name onto
* the newest version (`notion_search` -> `notion_search_v2`). Returns the input
* unchanged when nothing matches, matching `@/tools/utils`.
*/
export function resolveToolId(toolName: string): string {
if (toolIdSet.has(toolName)) return toolName
return getLatestByBaseName().get(toolName) ?? toolName
}
/** Whether `toolId` names a built-in tool, resolving unversioned names. */
export function hasToolId(toolId: string): boolean {
return toolIdSet.has(resolveToolId(toolId))
}
+7
View File
@@ -58,6 +58,13 @@ export function getLatestVersionTools(
/**
* Resolves a tool name to its actual tool ID in the registry.
* Handles both stripped names (e.g., 'notion_search') and versioned names (e.g., 'notion_search_v2').
*
* Server-side counterpart to `resolveToolId` in `@/tools/tool-ids`. Both exist
* deliberately: this one reads the live registry, so a tool added but not yet
* regenerated stays resolvable; that one resolves against the generated id list
* without pulling 4,300 tools into a client graph. Client code wants that one.
* `tool-metadata:check` asserts the two never diverge.
*
* @param toolName The tool name to resolve (may or may not have version suffix)
* @returns The actual tool ID in the registry, or the original name if not found
*/
+200
View File
@@ -0,0 +1,200 @@
import type { ToolResponse } from '@/tools/types'
/** Params shared by every Zoho tool: the OAuth token and the account's region. */
export interface ZohoBaseParams {
accessToken: string
dataCenter?: string
}
/** Params shared by every Zoho Desk tool — Desk requires an `orgId` header. */
export interface ZohoDeskBaseParams extends ZohoBaseParams {
orgId: string
}
/**
* Pagination block returned by CRM list/search endpoints.
* @see https://www.zoho.com/crm/developer/docs/api/v8/get-records.html
*/
export interface ZohoCrmPageInfo {
page: number | null
perPage: number | null
count: number | null
moreRecords: boolean
}
export const ZOHO_CRM_PAGE_INFO_OUTPUT = {
type: 'object' as const,
description: 'Pagination metadata returned by Zoho CRM',
properties: {
page: { type: 'number' as const, description: 'Current page number', optional: true },
perPage: { type: 'number' as const, description: 'Records requested per page', optional: true },
count: {
type: 'number' as const,
description: 'Number of records in this page',
optional: true,
},
moreRecords: { type: 'boolean' as const, description: 'Whether further pages are available' },
},
}
/** A CRM write result entry, as returned in the `data` array of insert/update/upsert. */
export interface ZohoCrmWriteResult {
id: string | null
code: string | null
status: string | null
message: string | null
}
export const ZOHO_CRM_WRITE_RESULT_OUTPUT = {
type: 'array' as const,
description: 'Per-record results returned by Zoho CRM',
properties: {
id: { type: 'string' as const, description: 'Record ID', optional: true },
code: {
type: 'string' as const,
description: 'Zoho result code (e.g. SUCCESS)',
optional: true,
},
status: { type: 'string' as const, description: 'Result status', optional: true },
message: {
type: 'string' as const,
description: 'Human-readable result message',
optional: true,
},
},
}
export interface ZohoCrmGetRecordsParams extends ZohoBaseParams {
module: string
recordId?: string
fields?: string
page?: string | number
perPage?: string | number
sortBy?: string
sortOrder?: string
}
export interface ZohoCrmGetRecordsResponse extends ToolResponse {
output: {
records: Record<string, unknown>[]
record?: Record<string, unknown> | null
info: ZohoCrmPageInfo
}
}
export interface ZohoCrmCreateRecordsParams extends ZohoBaseParams {
module: string
records: unknown
trigger?: string
}
export interface ZohoCrmWriteResponse extends ToolResponse {
output: {
results: ZohoCrmWriteResult[]
}
}
export interface ZohoCrmUpdateRecordParams extends ZohoBaseParams {
module: string
recordId: string
record: unknown
trigger?: string
}
export interface ZohoCrmUpsertRecordsParams extends ZohoBaseParams {
module: string
records: unknown
duplicateCheckFields?: string
trigger?: string
}
export interface ZohoCrmDeleteRecordParams extends ZohoBaseParams {
module: string
recordId: string
wfTrigger?: string
}
export interface ZohoCrmSearchRecordsParams extends ZohoBaseParams {
module: string
criteria?: string
email?: string
phone?: string
word?: string
fields?: string
page?: string | number
perPage?: string | number
}
export interface ZohoCrmCoqlQueryParams extends ZohoBaseParams {
selectQuery: string
}
export interface ZohoCrmCoqlQueryResponse extends ToolResponse {
output: {
records: Record<string, unknown>[]
info: ZohoCrmPageInfo
}
}
export interface ZohoCrmGetModulesParams extends ZohoBaseParams {}
export interface ZohoCrmGetModulesResponse extends ToolResponse {
output: {
modules: Array<{
apiName: string | null
moduleName: string | null
id: string | null
pluralLabel: string | null
singularLabel: string | null
creatable: boolean | null
editable: boolean | null
deletable: boolean | null
}>
}
}
export interface ZohoCrmGetFieldsParams extends ZohoBaseParams {
module: string
}
export interface ZohoCrmGetFieldsResponse extends ToolResponse {
output: {
fields: Array<{
apiName: string | null
displayLabel: string | null
dataType: string | null
id: string | null
required: boolean | null
readOnly: boolean | null
length: number | null
}>
}
}
export interface ZohoCrmGetUsersParams extends ZohoBaseParams {
type?: string
page?: string | number
perPage?: string | number
}
export interface ZohoCrmGetUsersResponse extends ToolResponse {
output: {
users: Record<string, unknown>[]
info: ZohoCrmPageInfo
}
}
export interface ZohoCrmAddNoteParams extends ZohoBaseParams {
parentId: string
seModule: string
noteTitle?: string
noteContent: string
}
export type ZohoCrmResponse =
| ZohoCrmGetRecordsResponse
| ZohoCrmWriteResponse
| ZohoCrmCoqlQueryResponse
| ZohoCrmGetModulesResponse
| ZohoCrmGetFieldsResponse
| ZohoCrmGetUsersResponse
+192
View File
@@ -0,0 +1,192 @@
import { truncate } from '@sim/utils/string'
/**
* Zoho data centers Sim supports.
*
* Zoho is multi-DC: a user's account lives in exactly one region, and every API
* host is region-scoped. Zoho's guidance is to never hardcode a single region —
* the token response carries an `api_domain` naming the caller's CRM host.
* Sim cannot persist that value (the shared Better Auth `account` table has no
* column for provider-specific token fields), so the region is selected on the
* block instead and resolved to hosts here.
*
* Only regions whose CRM *and* Desk hosts are confirmed in Zoho's own
* documentation are listed. CA and SA are deliberately absent: Zoho's accounts
* docs give `accounts.zohocloud.ca` while the Desk SDK uses `accounts.zoho.ca`,
* and neither source states their CRM API host.
*
* @see https://www.zoho.com/crm/developer/docs/api/v8/multi-dc.html
*/
export const ZOHO_DATA_CENTERS = {
us: { crm: 'https://www.zohoapis.com', desk: 'https://desk.zoho.com' },
eu: { crm: 'https://www.zohoapis.eu', desk: 'https://desk.zoho.eu' },
in: { crm: 'https://www.zohoapis.in', desk: 'https://desk.zoho.in' },
au: { crm: 'https://www.zohoapis.com.au', desk: 'https://desk.zoho.com.au' },
jp: { crm: 'https://www.zohoapis.jp', desk: 'https://desk.zoho.jp' },
} as const
export type ZohoDataCenter = keyof typeof ZOHO_DATA_CENTERS
const DEFAULT_DATA_CENTER: ZohoDataCenter = 'us'
/** Zoho CRM API version pinned across every CRM tool. */
export const ZOHO_CRM_API_VERSION = 'v8'
function resolveDataCenter(dataCenter?: string): ZohoDataCenter {
const normalized = dataCenter?.trim().toLowerCase()
if (normalized && normalized in ZOHO_DATA_CENTERS) {
return normalized as ZohoDataCenter
}
return DEFAULT_DATA_CENTER
}
/**
* Returns the CRM API base URL (including version segment) for a data center.
* Falls back to the US region when unset or unrecognized.
*/
export function getCrmBaseUrl(dataCenter?: string): string {
return `${ZOHO_DATA_CENTERS[resolveDataCenter(dataCenter)].crm}/crm/${ZOHO_CRM_API_VERSION}`
}
/**
* Returns the Desk API base URL (including the `/api/v1` segment) for a data
* center. Falls back to the US region when unset or unrecognized.
*/
export function getDeskBaseUrl(dataCenter?: string): string {
return `${ZOHO_DATA_CENTERS[resolveDataCenter(dataCenter)].desk}/api/v1`
}
/**
* Zoho authenticates with a bespoke scheme rather than `Bearer`.
* @see https://www.zoho.com/crm/developer/docs/api/v8/access-refresh.html
*/
export function buildZohoHeaders(accessToken: string, orgId?: string): Record<string, string> {
const headers: Record<string, string> = {
Authorization: `Zoho-oauthtoken ${accessToken}`,
'Content-Type': 'application/json',
}
if (orgId?.trim()) {
headers.orgId = orgId.trim()
}
return headers
}
/**
* Trims a required identifier and throws when it is missing or whitespace-only,
* so a blank value can never collapse into an empty URL path segment and send a
* malformed request to Zoho.
*/
export function requireZohoId(value: string | undefined, label: string): string {
const trimmed = value?.trim()
if (!trimmed) {
throw new Error(`${label} is required.`)
}
return trimmed
}
/**
* Parses a JSON object supplied either as an object (from the LLM) or as a JSON
* string (from a block input), and rejects anything that is not a plain object.
*/
export function parseJsonObject(
value: unknown,
label: string
): Record<string, unknown> | undefined {
if (value === undefined || value === null || value === '') return undefined
let parsed: unknown = value
if (typeof value === 'string') {
try {
parsed = JSON.parse(value)
} catch {
throw new Error(`${label} must be valid JSON.`)
}
}
if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) {
throw new Error(`${label} must be a JSON object.`)
}
return parsed as Record<string, unknown>
}
/**
* Coerces a numeric-ish param into a positive integer, clamped to `max`.
* Returns undefined when unset or unparseable so callers can omit the param.
*/
export function toPositiveInt(value: unknown, max: number): number | undefined {
if (value === undefined || value === null || value === '') return undefined
const parsed = Number(value)
if (!Number.isFinite(parsed) || parsed < 1) return undefined
return Math.min(Math.floor(parsed), max)
}
/**
* Appends only the defined entries of `params` to a URL's query string, so unset
* optional params never surface as empty values Zoho would reject.
*/
export function buildQuery(params: Record<string, string | number | undefined>): string {
const search = new URLSearchParams()
for (const [key, value] of Object.entries(params)) {
if (value === undefined || value === '') continue
search.set(key, String(value))
}
const query = search.toString()
return query ? `?${query}` : ''
}
/**
* Reads a Zoho response body as JSON, tolerating the empty bodies Zoho returns
* for "no matching records" — CRM answers `204 No Content` on an empty list or
* search, and Desk does the same, so a bare `response.json()` would throw on a
* perfectly successful call.
*/
export async function readZohoJson(response: Response): Promise<any> {
if (response.status === 204) return {}
const text = await response.text()
if (!text.trim()) return {}
try {
return JSON.parse(text)
} catch {
return { message: truncate(text, 500) }
}
}
/**
* Extracts a descriptive message from a Zoho error payload.
*
* CRM returns `{ code, message, status, details }`; Desk returns
* `{ errorCode, message }`. Both are handled, with an HTTP-status fallback.
*/
export function extractZohoErrorMessage(
data: unknown,
status: number,
defaultMessage: string
): string {
const payload = data as
| { message?: unknown; code?: unknown; errorCode?: unknown; data?: unknown }
| undefined
const nested = Array.isArray(payload?.data) ? payload.data[0] : undefined
const record = (nested ?? payload) as
| { message?: unknown; code?: unknown; errorCode?: unknown }
| undefined
if (record && typeof record.message === 'string' && record.message.trim()) {
const code = record.code ?? record.errorCode
const suffix = typeof code === 'string' && code.trim() ? ` [${code}]` : ''
return `Zoho API Error (${status}): ${record.message}${suffix}`
}
switch (status) {
case 400:
return `Zoho API Error (400): Bad Request — the request was malformed or missing required parameters.`
case 401:
return `Zoho API Error (401): Unauthorized — the access token is invalid or expired. Please reconnect your Zoho account.`
case 403:
return `Zoho API Error (403): Forbidden — your Zoho account lacks permission, or the required OAuth scope was not granted.`
case 404:
return `Zoho API Error (404): Not Found — the requested record does not exist or is not visible to you.`
case 429:
return `Zoho API Error (429): Rate limit exceeded — too many API calls. Please retry later.`
default:
return `${defaultMessage} (HTTP ${status})`
}
}
+18
View File
@@ -338,7 +338,25 @@ export const blocksMock = {
/**
* Pre-configured tools/utils mock for use with vi.mock('@/tools/utils', () => toolsUtilsMock).
*
* Only mocks the *executable* lookup. Code that reads a tool's shape now goes
* through `@/tools/metadata`, so mocking this alone leaves such code reading the
* real generated artifacts — see {@link toolsMetadataMock}.
*/
export const toolsUtilsMock = {
getTool: createMockGetTool(),
}
/**
* Pre-configured metadata mock for use with
* `vi.mock('@/tools/metadata', () => toolsMetadataMock)`.
*
* Backed by the same `mockToolConfigs` as {@link toolsUtilsMock}, so a test that
* mocks both sees one consistent tool universe. Mock this wherever the code under
* test reads `params`, `outputs` or `name` — mocking `@/tools/utils` there is a
* no-op that only appears to work because the real artifacts happen to agree.
*/
export const toolsMetadataMock = {
getToolMetadata: createMockGetTool(),
getToolParams: (toolId: string) => createMockGetTool()(toolId)?.params,
}
+1
View File
@@ -33,6 +33,7 @@ export {
createMockGetTool,
mockBlockConfigs,
mockToolConfigs,
toolsMetadataMock,
toolsUtilsMock,
} from './blocks.mock'
// Copilot HTTP mocks (for @/lib/copilot/request/http)
+57 -6
View File
@@ -17,9 +17,14 @@
* a single consumer — keeping it separate means callers that only need `params`
* or an id check don't pay for it:
*
* tools/generated/tool-ids.ts every registered tool id
* tools/generated/tool-metadata.ts id -> { name, description, version, params, oauth }
* tools/generated/tool-outputs.ts id -> outputs
*
* Ids are their own artifact because resolving a possibly-unversioned tool name
* needs only the key set, and an existence check needs nothing more — so those
* callers load ~100 KB instead of ~4 MB.
*
* Each artifact holds its data as one JSON string parsed at runtime — see
* `serialize()` for why an imported `.json` or an object literal is not viable
* at this size.
@@ -32,11 +37,14 @@ import { mkdir, readFile, writeFile } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { tools } from '../apps/sim/tools/registry'
import { hasToolId } from '../apps/sim/tools/tool-ids'
import type { ToolConfig } from '../apps/sim/tools/types'
import { getTool } from '../apps/sim/tools/utils'
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const ROOT = resolve(SCRIPT_DIR, '..')
const GENERATED_DIR = resolve(ROOT, 'apps/sim/tools/generated')
const IDS_PATH = resolve(GENERATED_DIR, 'tool-ids.ts')
const METADATA_PATH = resolve(GENERATED_DIR, 'tool-metadata.ts')
const OUTPUTS_PATH = resolve(GENERATED_DIR, 'tool-outputs.ts')
@@ -141,6 +149,12 @@ function build(registry: ToolRecord) {
}
return {
ids: serializeValue(
Object.keys(metadata),
'toolIds',
'/** Every registered tool id, including versioned variants. */',
'string[]'
),
metadata: serialize(
metadata,
'toolMetadata',
@@ -192,12 +206,16 @@ function toJsStringLiteral(json: string): string {
* generated artifact nothing reads by eye and CI verifies wholesale.
*/
function serialize(entries: Record<string, unknown>, exportName: string, doc: string): string {
const literal = toJsStringLiteral(JSON.stringify(entries))
return serializeValue(entries, exportName, doc, 'Record<string, unknown>')
}
function serializeValue(value: unknown, exportName: string, doc: string, type: string): string {
const literal = toJsStringLiteral(JSON.stringify(value))
return `// Generated by scripts/sync-tool-metadata.ts — do not edit.
// Regenerate with: bun run tool-metadata:generate
${doc}
const ${exportName}: Record<string, unknown> = JSON.parse(
const ${exportName}: ${type} = JSON.parse(
${literal}
)
@@ -205,24 +223,57 @@ export default ${exportName}
`
}
/**
* Asserts the two tool-id resolvers agree.
*
* `@/tools/utils` resolves against the live registry; `@/tools/tool-ids` against
* the generated id list. Both exist deliberately — the registry-backed one keeps
* a newly-added tool resolvable before regeneration, the list-backed one lets
* client code resolve without importing 4,300 tools. Nothing structurally keeps
* the two in step, so it is checked here rather than left to trust.
*
* Runs only after the staleness check passes, since a stale id list would
* otherwise report a divergence that is really just a missing regeneration. It
* cannot live in a vitest suite: `vitest.setup.ts` globally mocks
* `@/tools/registry` to an empty map, so `getTool` resolves nothing there.
*/
function assertResolverParity() {
const ids = Object.keys(tools)
const probes = new Set([...ids, ...ids.map((id) => id.replace(/_v\d+$/, '')), '__not_a_tool__'])
const divergent: string[] = []
for (const probe of probes) {
if (Boolean(getTool(probe)) !== hasToolId(probe)) divergent.push(probe)
}
if (divergent.length > 0) {
throw new Error(
`Tool id resolvers disagree on ${divergent.length} of ${probes.size} names ` +
`(e.g. ${divergent.slice(0, 5).join(', ')}).\n` +
'resolveToolId in apps/sim/tools/utils.ts and apps/sim/tools/tool-ids.ts have drifted.'
)
}
}
async function main() {
const checkOnly = process.argv.includes('--check')
const { metadata, outputs, toolCount } = build(tools as ToolRecord)
const { ids, metadata, outputs, toolCount } = build(tools as ToolRecord)
if (checkOnly) {
const [existingMetadata, existingOutputs] = await Promise.all([
const [existingIds, existingMetadata, existingOutputs] = await Promise.all([
readFile(IDS_PATH, 'utf8').catch(() => null),
readFile(METADATA_PATH, 'utf8').catch(() => null),
readFile(OUTPUTS_PATH, 'utf8').catch(() => null),
])
if (existingMetadata !== metadata || existingOutputs !== outputs) {
if (existingIds !== ids || existingMetadata !== metadata || existingOutputs !== outputs) {
throw new Error('Generated tool metadata is stale. Run: bun run tool-metadata:generate')
}
console.log(`✓ tool metadata in sync (${toolCount} tools)`)
assertResolverParity()
console.log(`✓ tool metadata in sync (${toolCount} tools), resolvers agree`)
return
}
await mkdir(GENERATED_DIR, { recursive: true })
await Promise.all([
writeFile(IDS_PATH, ids, 'utf8'),
writeFile(METADATA_PATH, metadata, 'utf8'),
writeFile(OUTPUTS_PATH, outputs, 'utf8'),
])