mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
improvement(x): align OAuth scopes, add scope descriptions, and set optional fields to advanced mode (#3372)
* improvement(x): align OAuth scopes, add scope descriptions, and set optional fields to advanced mode * improvement(skills): add typed JSON outputs guidance to add-tools, add-block, and add-integration skills * improvement(skills): add final validation steps to add-tools, add-block, and add-integration skills * fix(skills): correct misleading JSON array comment in wandConfig example * feat(skills): add validate-integration skill for auditing tools, blocks, and registry against API docs * improvement(skills): expand validate-integration with full block-tool alignment, OAuth scopes, pagination, and error handling checks
This commit is contained in:
@@ -532,6 +532,41 @@ outputs: {
|
||||
}
|
||||
```
|
||||
|
||||
### Typed JSON Outputs
|
||||
|
||||
When using `type: 'json'` and you know the object shape in advance, **describe the inner fields in the description** so downstream blocks know what properties are available. For well-known, stable objects, use nested output definitions instead:
|
||||
|
||||
```typescript
|
||||
outputs: {
|
||||
// BAD: Opaque json with no info about what's inside
|
||||
plan: { type: 'json', description: 'Zone plan information' },
|
||||
|
||||
// GOOD: Describe the known fields in the description
|
||||
plan: {
|
||||
type: 'json',
|
||||
description: 'Zone plan information (id, name, price, currency, frequency, is_subscribed)',
|
||||
},
|
||||
|
||||
// BEST: Use nested output definition when the shape is stable and well-known
|
||||
plan: {
|
||||
id: { type: 'string', description: 'Plan identifier' },
|
||||
name: { type: 'string', description: 'Plan name' },
|
||||
price: { type: 'number', description: 'Plan price' },
|
||||
currency: { type: 'string', description: 'Price currency' },
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Use the nested pattern when:
|
||||
- The object has a small, stable set of fields (< 10)
|
||||
- Downstream blocks will commonly access specific properties
|
||||
- The API response shape is well-documented and unlikely to change
|
||||
|
||||
Use `type: 'json'` with a descriptive string when:
|
||||
- The object has many fields or a dynamic shape
|
||||
- It represents a list/array of items
|
||||
- The shape varies by operation
|
||||
|
||||
## V2 Block Pattern
|
||||
|
||||
When creating V2 blocks (alongside legacy V1):
|
||||
@@ -695,6 +730,62 @@ Please provide the SVG and I'll convert it to a React component.
|
||||
You can usually find this in the service's brand/press kit page, or copy it from their website.
|
||||
```
|
||||
|
||||
## Advanced Mode for Optional Fields
|
||||
|
||||
Optional fields that are rarely used should be set to `mode: 'advanced'` so they don't clutter the basic UI. This includes:
|
||||
- Pagination tokens
|
||||
- Time range filters (start/end time)
|
||||
- Sort order options
|
||||
- Reply settings
|
||||
- Rarely used IDs (e.g., reply-to tweet ID, quote tweet ID)
|
||||
- Max results / limits
|
||||
|
||||
```typescript
|
||||
{
|
||||
id: 'startTime',
|
||||
title: 'Start Time',
|
||||
type: 'short-input',
|
||||
placeholder: 'ISO 8601 timestamp',
|
||||
condition: { field: 'operation', value: ['search', 'list'] },
|
||||
mode: 'advanced', // Rarely used, hide from basic view
|
||||
}
|
||||
```
|
||||
|
||||
## WandConfig for Complex Inputs
|
||||
|
||||
Use `wandConfig` for fields that are hard to fill out manually, such as timestamps, comma-separated lists, and complex query strings. This gives users an AI-assisted input experience.
|
||||
|
||||
```typescript
|
||||
// Timestamps - use generationType: 'timestamp' to inject current date context
|
||||
{
|
||||
id: 'startTime',
|
||||
title: 'Start Time',
|
||||
type: 'short-input',
|
||||
mode: 'advanced',
|
||||
wandConfig: {
|
||||
enabled: true,
|
||||
prompt: 'Generate an ISO 8601 timestamp based on the user description. Return ONLY the timestamp string.',
|
||||
generationType: 'timestamp',
|
||||
},
|
||||
}
|
||||
|
||||
// Comma-separated lists - simple prompt without generationType
|
||||
{
|
||||
id: 'mediaIds',
|
||||
title: 'Media IDs',
|
||||
type: 'short-input',
|
||||
mode: 'advanced',
|
||||
wandConfig: {
|
||||
enabled: true,
|
||||
prompt: 'Generate a comma-separated list of media IDs. Return ONLY the comma-separated values.',
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
## Naming Convention
|
||||
|
||||
All tool IDs referenced in `tools.access` and returned by `tools.config.tool` MUST use `snake_case` (e.g., `x_create_tweet`, `slack_send_message`). Never use camelCase or PascalCase.
|
||||
|
||||
## Checklist Before Finishing
|
||||
|
||||
- [ ] All subBlocks have `id`, `title` (except switch), and `type`
|
||||
@@ -702,9 +793,24 @@ You can usually find this in the service's brand/press kit page, or copy it from
|
||||
- [ ] DependsOn set for fields that need other values
|
||||
- [ ] Required fields marked correctly (boolean or condition)
|
||||
- [ ] OAuth inputs have correct `serviceId`
|
||||
- [ ] Tools.access lists all tool IDs
|
||||
- [ ] Tools.config.tool returns correct tool ID
|
||||
- [ ] Tools.access lists all tool IDs (snake_case)
|
||||
- [ ] Tools.config.tool returns correct tool ID (snake_case)
|
||||
- [ ] Outputs match tool outputs
|
||||
- [ ] Block registered in registry.ts
|
||||
- [ ] If icon missing: asked user to provide SVG
|
||||
- [ ] If triggers exist: `triggers` config set, trigger subBlocks spread
|
||||
- [ ] Optional/rarely-used fields set to `mode: 'advanced'`
|
||||
- [ ] Timestamps and complex inputs have `wandConfig` enabled
|
||||
|
||||
## Final Validation (Required)
|
||||
|
||||
After creating the block, you MUST validate it against every tool it references:
|
||||
|
||||
1. **Read every tool definition** that appears in `tools.access` — do not skip any
|
||||
2. **For each tool, verify the block has correct:**
|
||||
- SubBlock inputs that cover all required tool params (with correct `condition` to show for that operation)
|
||||
- SubBlock input types that match the tool param types (e.g., dropdown for enums, short-input for strings)
|
||||
- `tools.config.params` correctly maps subBlock IDs to tool param names (if they differ)
|
||||
- Type coercions in `tools.config.params` for any params that need conversion (Number(), Boolean(), JSON.parse())
|
||||
3. **Verify block outputs** cover the key fields returned by all tools
|
||||
4. **Verify conditions** — each subBlock should only show for the operations that actually use it
|
||||
|
||||
@@ -102,6 +102,7 @@ export const {service}{Action}Tool: ToolConfig<Params, Response> = {
|
||||
- Always use `?? []` for optional array fields
|
||||
- Set `optional: true` for outputs that may not exist
|
||||
- Never output raw JSON dumps - extract meaningful fields
|
||||
- When using `type: 'json'` and you know the object shape, define `properties` with the inner fields so downstream consumers know the structure. Only use bare `type: 'json'` when the shape is truly dynamic
|
||||
|
||||
## Step 3: Create Block
|
||||
|
||||
@@ -436,6 +437,12 @@ If creating V2 versions (API-aligned outputs):
|
||||
- [ ] Ran `bun run scripts/generate-docs.ts`
|
||||
- [ ] Verified docs file created
|
||||
|
||||
### Final Validation (Required)
|
||||
- [ ] Read every tool file and cross-referenced inputs/outputs against the API docs
|
||||
- [ ] Verified block subBlocks cover all required tool params with correct conditions
|
||||
- [ ] Verified block outputs match what the tools actually return
|
||||
- [ ] Verified `tools.config.params` correctly maps and coerces all param types
|
||||
|
||||
## Example Command
|
||||
|
||||
When the user asks to add an integration:
|
||||
@@ -685,13 +692,40 @@ return NextResponse.json({
|
||||
| `isUserFile` | `@/lib/core/utils/user-file` | Type guard for UserFile objects |
|
||||
| `FileInputSchema` | `@/lib/uploads/utils/file-schemas` | Zod schema for file validation |
|
||||
|
||||
### Advanced Mode for Optional Fields
|
||||
|
||||
Optional fields that are rarely used should be set to `mode: 'advanced'` so they don't clutter the basic UI. Examples: pagination tokens, time range filters, sort order, max results, reply settings.
|
||||
|
||||
### WandConfig for Complex Inputs
|
||||
|
||||
Use `wandConfig` for fields that are hard to fill out manually:
|
||||
- **Timestamps**: Use `generationType: 'timestamp'` to inject current date context into the AI prompt
|
||||
- **JSON arrays**: Use `generationType: 'json-object'` for structured data
|
||||
- **Complex queries**: Use a descriptive prompt explaining the expected format
|
||||
|
||||
```typescript
|
||||
{
|
||||
id: 'startTime',
|
||||
title: 'Start Time',
|
||||
type: 'short-input',
|
||||
mode: 'advanced',
|
||||
wandConfig: {
|
||||
enabled: true,
|
||||
prompt: 'Generate an ISO 8601 timestamp. Return ONLY the timestamp string.',
|
||||
generationType: 'timestamp',
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### Common Gotchas
|
||||
|
||||
1. **OAuth serviceId must match** - The `serviceId` in oauth-input must match the OAuth provider configuration
|
||||
2. **Tool IDs are snake_case** - `stripe_create_payment`, not `stripeCreatePayment`
|
||||
2. **All tool IDs MUST be snake_case** - `stripe_create_payment`, not `stripeCreatePayment`. This applies to tool `id` fields, registry keys, `tools.access` arrays, and `tools.config.tool` return values
|
||||
3. **Block type is snake_case** - `type: 'stripe'`, not `type: 'Stripe'`
|
||||
4. **Alphabetical ordering** - Keep imports and registry entries alphabetically sorted
|
||||
5. **Required can be conditional** - Use `required: { field: 'op', value: 'create' }` instead of always true
|
||||
6. **DependsOn clears options** - When a dependency changes, selector options are refetched
|
||||
7. **Never pass Buffer directly to fetch** - Convert to `new Uint8Array(buffer)` for TypeScript compatibility
|
||||
8. **Always handle legacy file params** - Keep hidden `fileContent` params for backwards compatibility
|
||||
9. **Optional fields use advanced mode** - Set `mode: 'advanced'` on rarely-used optional fields
|
||||
10. **Complex inputs need wandConfig** - Timestamps, JSON arrays, and other hard-to-type values should have `wandConfig` enabled
|
||||
|
||||
@@ -147,9 +147,18 @@ closedAt: {
|
||||
},
|
||||
```
|
||||
|
||||
### Nested Properties
|
||||
For complex outputs, define nested structure:
|
||||
### Typed JSON Outputs
|
||||
|
||||
When using `type: 'json'` and you know the object shape in advance, **always define the inner structure** using `properties` so downstream consumers know what fields are available:
|
||||
|
||||
```typescript
|
||||
// BAD: Opaque json with no info about what's inside
|
||||
metadata: {
|
||||
type: 'json',
|
||||
description: 'Response metadata',
|
||||
},
|
||||
|
||||
// GOOD: Define the known properties
|
||||
metadata: {
|
||||
type: 'json',
|
||||
description: 'Response metadata',
|
||||
@@ -159,7 +168,10 @@ metadata: {
|
||||
count: { type: 'number', description: 'Total count' },
|
||||
},
|
||||
},
|
||||
```
|
||||
|
||||
For arrays of objects, define the item structure:
|
||||
```typescript
|
||||
items: {
|
||||
type: 'array',
|
||||
description: 'List of items',
|
||||
@@ -173,6 +185,8 @@ items: {
|
||||
},
|
||||
```
|
||||
|
||||
Only use bare `type: 'json'` without `properties` when the shape is truly dynamic or unknown.
|
||||
|
||||
## Critical Rules for transformResponse
|
||||
|
||||
### Handle Nullable Fields
|
||||
@@ -272,8 +286,13 @@ If creating V2 tools (API-aligned outputs), use `_v2` suffix:
|
||||
- Version: `'2.0.0'`
|
||||
- Outputs: Flat, API-aligned (no content/metadata wrapper)
|
||||
|
||||
## Naming Convention
|
||||
|
||||
All tool IDs MUST use `snake_case`: `{service}_{action}` (e.g., `x_create_tweet`, `slack_send_message`). Never use camelCase or PascalCase for tool IDs.
|
||||
|
||||
## Checklist Before Finishing
|
||||
|
||||
- [ ] All tool IDs use snake_case
|
||||
- [ ] All params have explicit `required: true` or `required: false`
|
||||
- [ ] All params have appropriate `visibility`
|
||||
- [ ] All nullable response fields use `?? null`
|
||||
@@ -281,4 +300,22 @@ If creating V2 tools (API-aligned outputs), use `_v2` suffix:
|
||||
- [ ] No raw JSON dumps in outputs
|
||||
- [ ] Types file has all interfaces
|
||||
- [ ] Index.ts exports all tools
|
||||
- [ ] Tool IDs use snake_case
|
||||
|
||||
## Final Validation (Required)
|
||||
|
||||
After creating all tools, you MUST validate every tool before finishing:
|
||||
|
||||
1. **Read every tool file** you created — do not skip any
|
||||
2. **Cross-reference with the API docs** to verify:
|
||||
- All required params are marked `required: true`
|
||||
- All optional params are marked `required: false`
|
||||
- Param types match the API (string, number, boolean, json)
|
||||
- Request URL, method, headers, and body match the API spec
|
||||
- `transformResponse` extracts the correct fields from the API response
|
||||
- All output fields match what the API actually returns
|
||||
- No fields are missing from outputs that the API provides
|
||||
- No extra fields are defined in outputs that the API doesn't return
|
||||
3. **Verify consistency** across tools:
|
||||
- Shared types in `types.ts` match all tools that use them
|
||||
- Tool IDs in the barrel export match the tool file definitions
|
||||
- Error handling is consistent (logger imports, error checks)
|
||||
|
||||
@@ -0,0 +1,286 @@
|
||||
---
|
||||
description: Validate an existing Sim integration (tools, block, registry) against the service's API docs
|
||||
argument-hint: <service-name> [api-docs-url]
|
||||
---
|
||||
|
||||
# Validate Integration Skill
|
||||
|
||||
You are an expert auditor for Sim integrations. Your job is to thoroughly validate that an existing integration is correct, complete, and follows all conventions.
|
||||
|
||||
## Your Task
|
||||
|
||||
When the user asks you to validate an integration:
|
||||
1. Read the service's API documentation (via WebFetch or Context7)
|
||||
2. Read every tool, the block, and registry entries
|
||||
3. Cross-reference everything against the API docs and Sim conventions
|
||||
4. Report all issues found, grouped by severity (critical, warning, suggestion)
|
||||
5. Fix all issues after reporting them
|
||||
|
||||
## Step 1: Gather All Files
|
||||
|
||||
Read **every** file for the integration — do not skip any:
|
||||
|
||||
```
|
||||
apps/sim/tools/{service}/ # All tool files, types.ts, index.ts
|
||||
apps/sim/blocks/blocks/{service}.ts # Block definition
|
||||
apps/sim/tools/registry.ts # Tool registry entries for this service
|
||||
apps/sim/blocks/registry.ts # Block registry entry for this service
|
||||
apps/sim/components/icons.tsx # Icon definition
|
||||
apps/sim/lib/auth/auth.ts # OAuth scopes (if OAuth service)
|
||||
apps/sim/lib/oauth/oauth.ts # OAuth provider config (if OAuth service)
|
||||
```
|
||||
|
||||
## Step 2: Pull API Documentation
|
||||
|
||||
Fetch the official API docs for the service. This is the **source of truth** for:
|
||||
- Endpoint URLs, HTTP methods, and auth headers
|
||||
- Required vs optional parameters
|
||||
- Parameter types and allowed values
|
||||
- Response shapes and field names
|
||||
- Pagination patterns (which param name, which response field)
|
||||
- Rate limits and error formats
|
||||
|
||||
## Step 3: Validate Tools
|
||||
|
||||
For **every** tool file, check:
|
||||
|
||||
### Tool ID and Naming
|
||||
- [ ] Tool ID uses `snake_case`: `{service}_{action}` (e.g., `x_create_tweet`, `slack_send_message`)
|
||||
- [ ] Tool `name` is human-readable (e.g., `'X Create Tweet'`)
|
||||
- [ ] Tool `description` is a concise one-liner describing what it does
|
||||
- [ ] Tool `version` is set (`'1.0.0'` or `'2.0.0'` for V2)
|
||||
|
||||
### Params
|
||||
- [ ] All required API params are marked `required: true`
|
||||
- [ ] All optional API params are marked `required: false`
|
||||
- [ ] Every param has explicit `required: true` or `required: false` — never omitted
|
||||
- [ ] Param types match the API (`'string'`, `'number'`, `'boolean'`, `'json'`)
|
||||
- [ ] Visibility is correct:
|
||||
- `'hidden'` — ONLY for OAuth access tokens and system-injected params
|
||||
- `'user-only'` — for API keys, credentials, and account-specific IDs the user must provide
|
||||
- `'user-or-llm'` — for everything else (search queries, content, filters, IDs that could come from other blocks)
|
||||
- [ ] Every param has a `description` that explains what it does
|
||||
|
||||
### Request
|
||||
- [ ] URL matches the API endpoint exactly (correct base URL, path segments, path params)
|
||||
- [ ] HTTP method matches the API spec (GET, POST, PUT, PATCH, DELETE)
|
||||
- [ ] Headers include correct auth pattern:
|
||||
- OAuth: `Authorization: Bearer ${params.accessToken}`
|
||||
- API Key: correct header name and format per the service's docs
|
||||
- [ ] `Content-Type` header is set for POST/PUT/PATCH requests
|
||||
- [ ] Body sends all required fields and only includes optional fields when provided
|
||||
- [ ] For GET requests with query params: URL is constructed correctly with query string
|
||||
- [ ] ID fields in URL paths are `.trim()`-ed to prevent copy-paste whitespace errors
|
||||
- [ ] Path params use template literals correctly: `` `https://api.service.com/v1/${params.id.trim()}` ``
|
||||
|
||||
### Response / transformResponse
|
||||
- [ ] Correctly parses the API response (`await response.json()`)
|
||||
- [ ] Extracts the right fields from the response structure (e.g., `data.data` vs `data` vs `data.results`)
|
||||
- [ ] All nullable fields use `?? null`
|
||||
- [ ] All optional arrays use `?? []`
|
||||
- [ ] Error cases are handled: checks for missing/empty data and returns meaningful error
|
||||
- [ ] `createLogger` is imported from `@sim/logger` and used for error logging
|
||||
- [ ] Does NOT do raw JSON dumps — extracts meaningful, individual fields
|
||||
|
||||
### Outputs
|
||||
- [ ] All output fields match what the API actually returns
|
||||
- [ ] No fields are missing that the API provides and users would commonly need
|
||||
- [ ] No phantom fields defined that the API doesn't return
|
||||
- [ ] `optional: true` is set on fields that may not exist in all responses
|
||||
- [ ] When using `type: 'json'` and the shape is known, `properties` defines the inner fields
|
||||
- [ ] When using `type: 'array'`, `items` defines the item structure with `properties`
|
||||
- [ ] Field descriptions are accurate and helpful
|
||||
|
||||
### Types (types.ts)
|
||||
- [ ] Has param interfaces for every tool (e.g., `XCreateTweetParams`)
|
||||
- [ ] Has response interfaces for every tool (extending `ToolResponse`)
|
||||
- [ ] Optional params use `?` in the interface (e.g., `replyTo?: string`)
|
||||
- [ ] Field names in types match actual API field names
|
||||
- [ ] Shared response types are properly reused (e.g., `XTweetResponse` shared across tweet tools)
|
||||
|
||||
### Barrel Export (index.ts)
|
||||
- [ ] Every tool is exported
|
||||
- [ ] All types are re-exported (`export * from './types'`)
|
||||
- [ ] No orphaned exports (tools that don't exist)
|
||||
|
||||
### Tool Registry (tools/registry.ts)
|
||||
- [ ] Every tool is imported and registered
|
||||
- [ ] Registry keys use snake_case and match tool IDs exactly
|
||||
- [ ] Entries are in alphabetical order within the file
|
||||
|
||||
## Step 4: Validate Block
|
||||
|
||||
### Block ↔ Tool Alignment (CRITICAL)
|
||||
|
||||
This is the most important validation — the block must be perfectly aligned with every tool it references.
|
||||
|
||||
For **each tool** in `tools.access`:
|
||||
- [ ] The operation dropdown has an option whose ID matches the tool ID (or the `tools.config.tool` function correctly maps to it)
|
||||
- [ ] Every **required** tool param (except `accessToken`) has a corresponding subBlock input that is:
|
||||
- Shown when that operation is selected (correct `condition`)
|
||||
- Marked as `required: true` (or conditionally required)
|
||||
- [ ] Every **optional** tool param has a corresponding subBlock input (or is intentionally omitted if truly never needed)
|
||||
- [ ] SubBlock `id` values are unique across the entire block — no duplicates even across different conditions
|
||||
- [ ] The `tools.config.tool` function returns the correct tool ID for every possible operation value
|
||||
- [ ] The `tools.config.params` function correctly maps subBlock IDs to tool param names when they differ
|
||||
|
||||
### SubBlocks
|
||||
- [ ] Operation dropdown lists ALL tool operations available in `tools.access`
|
||||
- [ ] Dropdown option labels are human-readable and descriptive
|
||||
- [ ] Conditions use correct syntax:
|
||||
- Single value: `{ field: 'operation', value: 'x_create_tweet' }`
|
||||
- Multiple values (OR): `{ field: 'operation', value: ['x_create_tweet', 'x_delete_tweet'] }`
|
||||
- Negation: `{ field: 'operation', value: 'delete', not: true }`
|
||||
- Compound: `{ field: 'op', value: 'send', and: { field: 'type', value: 'dm' } }`
|
||||
- [ ] Condition arrays include ALL operations that use that field — none missing
|
||||
- [ ] `dependsOn` is set for fields that need other values (selectors depending on credential, cascading dropdowns)
|
||||
- [ ] SubBlock types match tool param types:
|
||||
- Enum/fixed options → `dropdown`
|
||||
- Free text → `short-input`
|
||||
- Long text/content → `long-input`
|
||||
- True/false → `dropdown` with Yes/No options (not `switch` unless purely UI toggle)
|
||||
- Credentials → `oauth-input` with correct `serviceId`
|
||||
- [ ] Dropdown `value: () => 'default'` is set for dropdowns with a sensible default
|
||||
|
||||
### Advanced Mode
|
||||
- [ ] Optional, rarely-used fields are set to `mode: 'advanced'`:
|
||||
- Pagination tokens / next tokens
|
||||
- Time range filters (start/end time)
|
||||
- Sort order / direction options
|
||||
- Max results / per page limits
|
||||
- Reply settings / threading options
|
||||
- Rarely used IDs (reply-to, quote-tweet, etc.)
|
||||
- Exclude filters
|
||||
- [ ] **Required** fields are NEVER set to `mode: 'advanced'`
|
||||
- [ ] Fields that users fill in most of the time are NOT set to `mode: 'advanced'`
|
||||
|
||||
### WandConfig
|
||||
- [ ] Timestamp fields have `wandConfig` with `generationType: 'timestamp'`
|
||||
- [ ] Comma-separated list fields have `wandConfig` with a descriptive prompt
|
||||
- [ ] Complex filter/query fields have `wandConfig` with format examples in the prompt
|
||||
- [ ] All `wandConfig` prompts end with "Return ONLY the [format] - no explanations, no extra text."
|
||||
- [ ] `wandConfig.placeholder` describes what to type in natural language
|
||||
|
||||
### Tools Config
|
||||
- [ ] `tools.access` lists **every** tool ID the block can use — none missing
|
||||
- [ ] `tools.config.tool` returns the correct tool ID for each operation
|
||||
- [ ] Type coercions are in `tools.config.params` (runs at execution time), NOT in `tools.config.tool` (runs at serialization time before variable resolution)
|
||||
- [ ] `tools.config.params` handles:
|
||||
- `Number()` conversion for numeric params that come as strings from inputs
|
||||
- `Boolean` / string-to-boolean conversion for toggle params
|
||||
- Empty string → `undefined` conversion for optional dropdown values
|
||||
- Any subBlock ID → tool param name remapping
|
||||
- [ ] No `Number()`, `JSON.parse()`, or other coercions in `tools.config.tool` — these would destroy dynamic references like `<Block.output>`
|
||||
|
||||
### Block Outputs
|
||||
- [ ] Outputs cover the key fields returned by ALL tools (not just one operation)
|
||||
- [ ] Output types are correct (`'string'`, `'number'`, `'boolean'`, `'json'`)
|
||||
- [ ] `type: 'json'` outputs either:
|
||||
- Describe inner fields in the description string (GOOD): `'User profile (id, name, username, bio)'`
|
||||
- Use nested output definitions (BEST): `{ id: { type: 'string' }, name: { type: 'string' } }`
|
||||
- [ ] No opaque `type: 'json'` with vague descriptions like `'Response data'`
|
||||
- [ ] Outputs that only appear for certain operations use `condition` if supported, or document which operations return them
|
||||
|
||||
### Block Metadata
|
||||
- [ ] `type` is snake_case (e.g., `'x'`, `'cloudflare'`)
|
||||
- [ ] `name` is human-readable (e.g., `'X'`, `'Cloudflare'`)
|
||||
- [ ] `description` is a concise one-liner
|
||||
- [ ] `longDescription` provides detail for docs
|
||||
- [ ] `docsLink` points to `'https://docs.sim.ai/tools/{service}'`
|
||||
- [ ] `category` is `'tools'`
|
||||
- [ ] `bgColor` uses the service's brand color hex
|
||||
- [ ] `icon` references the correct icon component from `@/components/icons`
|
||||
- [ ] `authMode` is set correctly (`AuthMode.OAuth` or `AuthMode.ApiKey`)
|
||||
- [ ] Block is registered in `blocks/registry.ts` alphabetically
|
||||
|
||||
### Block Inputs
|
||||
- [ ] `inputs` section lists all subBlock params that the block accepts
|
||||
- [ ] Input types match the subBlock types
|
||||
- [ ] When using `canonicalParamId`, inputs list the canonical ID (not the raw subBlock IDs)
|
||||
|
||||
## Step 5: Validate OAuth Scopes (if OAuth service)
|
||||
|
||||
- [ ] `auth.ts` scopes include ALL scopes needed by ALL tools in the integration
|
||||
- [ ] `oauth.ts` provider config scopes match `auth.ts` scopes
|
||||
- [ ] Block `requiredScopes` (if defined) matches `auth.ts` scopes
|
||||
- [ ] No excess scopes that aren't needed by any tool
|
||||
- [ ] Each scope has a human-readable description in `oauth-required-modal.tsx`'s `SCOPE_DESCRIPTIONS`
|
||||
|
||||
## Step 6: Validate Pagination Consistency
|
||||
|
||||
If any tools support pagination:
|
||||
- [ ] Pagination param names match the API docs (e.g., `pagination_token` vs `next_token` vs `cursor`)
|
||||
- [ ] Different API endpoints that use different pagination param names have separate subBlocks in the block
|
||||
- [ ] Pagination response fields (`nextToken`, `cursor`, etc.) are included in tool outputs
|
||||
- [ ] Pagination subBlocks are set to `mode: 'advanced'`
|
||||
|
||||
## Step 7: Validate Error Handling
|
||||
|
||||
- [ ] Every tool imports `createLogger` from `@sim/logger`
|
||||
- [ ] Every tool creates a logger: `const logger = createLogger('{ToolName}')`
|
||||
- [ ] `transformResponse` checks for error conditions before accessing data
|
||||
- [ ] Error responses include meaningful messages (not just generic "failed")
|
||||
- [ ] HTTP error status codes are handled (check `response.ok` or status codes)
|
||||
|
||||
## Step 8: Report and Fix
|
||||
|
||||
### Report Format
|
||||
|
||||
Group findings by severity:
|
||||
|
||||
**Critical** (will cause runtime errors or incorrect behavior):
|
||||
- Wrong endpoint URL or HTTP method
|
||||
- Missing required params or wrong `required` flag
|
||||
- Incorrect response field mapping (accessing wrong path in response)
|
||||
- Missing error handling that would cause crashes
|
||||
- Tool ID mismatch between tool file, registry, and block `tools.access`
|
||||
- OAuth scopes missing in `auth.ts` that tools need
|
||||
- `tools.config.tool` returning wrong tool ID for an operation
|
||||
- Type coercions in `tools.config.tool` instead of `tools.config.params`
|
||||
|
||||
**Warning** (follows conventions incorrectly or has usability issues):
|
||||
- Optional field not set to `mode: 'advanced'`
|
||||
- Missing `wandConfig` on timestamp/complex fields
|
||||
- Wrong `visibility` on params (e.g., `'hidden'` instead of `'user-or-llm'`)
|
||||
- Missing `optional: true` on nullable outputs
|
||||
- Opaque `type: 'json'` without property descriptions
|
||||
- Missing `.trim()` on ID fields in request URLs
|
||||
- Missing `?? null` on nullable response fields
|
||||
- Block condition array missing an operation that uses that field
|
||||
- Missing scope description in `oauth-required-modal.tsx`
|
||||
|
||||
**Suggestion** (minor improvements):
|
||||
- Better description text
|
||||
- Inconsistent naming across tools
|
||||
- Missing `longDescription` or `docsLink`
|
||||
- Pagination fields that could benefit from `wandConfig`
|
||||
|
||||
### Fix All Issues
|
||||
|
||||
After reporting, fix every **critical** and **warning** issue. Apply **suggestions** where they don't add unnecessary complexity.
|
||||
|
||||
### Validation Output
|
||||
|
||||
After fixing, confirm:
|
||||
1. `bun run lint` passes with no fixes needed
|
||||
2. TypeScript compiles clean (no type errors)
|
||||
3. Re-read all modified files to verify fixes are correct
|
||||
|
||||
## Checklist Summary
|
||||
|
||||
- [ ] Read ALL tool files, block, types, index, and registries
|
||||
- [ ] Pulled and read official API documentation
|
||||
- [ ] Validated every tool's ID, params, request, response, outputs, and types against API docs
|
||||
- [ ] Validated block ↔ tool alignment (every tool param has a subBlock, every condition is correct)
|
||||
- [ ] Validated advanced mode on optional/rarely-used fields
|
||||
- [ ] Validated wandConfig on timestamps and complex inputs
|
||||
- [ ] Validated tools.config mapping, tool selector, and type coercions
|
||||
- [ ] Validated block outputs match what tools return, with typed JSON where possible
|
||||
- [ ] Validated OAuth scopes alignment across auth.ts, oauth.ts, block, and modal (if OAuth)
|
||||
- [ ] Validated pagination consistency across tools and block
|
||||
- [ ] Validated error handling (logger, error checks, meaningful messages)
|
||||
- [ ] Validated registry entries (tools and block, alphabetical, correct imports)
|
||||
- [ ] Reported all issues grouped by severity
|
||||
- [ ] Fixed all critical and warning issues
|
||||
- [ ] Ran `bun run lint` after fixes
|
||||
- [ ] Verified TypeScript compiles clean
|
||||
+13
-2
@@ -102,8 +102,19 @@ const SCOPE_DESCRIPTIONS: Record<string, string> = {
|
||||
'read:user': 'Read public user information',
|
||||
'user:email': 'Access email address',
|
||||
'tweet.read': 'Read tweets and timeline',
|
||||
'tweet.write': 'Post tweets',
|
||||
'users.read': 'Read profile information',
|
||||
'tweet.write': 'Post and delete tweets',
|
||||
'tweet.moderate.write': 'Hide and unhide replies to tweets',
|
||||
'users.read': 'Read user profiles and account information',
|
||||
'follows.read': 'View followers and following lists',
|
||||
'follows.write': 'Follow and unfollow users',
|
||||
'bookmark.read': 'View bookmarked tweets',
|
||||
'bookmark.write': 'Add and remove bookmarks',
|
||||
'like.read': 'View liked tweets and liking users',
|
||||
'like.write': 'Like and unlike tweets',
|
||||
'block.read': 'View blocked users',
|
||||
'block.write': 'Block and unblock users',
|
||||
'mute.read': 'View muted users',
|
||||
'mute.write': 'Mute and unmute users',
|
||||
'offline.access': 'Access account when not using the application',
|
||||
'data.records:read': 'Read records',
|
||||
'data.records:write': 'Write to records',
|
||||
|
||||
@@ -108,6 +108,7 @@ export const XBlock: BlockConfig = {
|
||||
type: 'short-input',
|
||||
placeholder: 'Enter tweet ID to reply to',
|
||||
condition: { field: 'operation', value: 'x_create_tweet' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'quoteTweetId',
|
||||
@@ -115,6 +116,7 @@ export const XBlock: BlockConfig = {
|
||||
type: 'short-input',
|
||||
placeholder: 'Enter tweet ID to quote',
|
||||
condition: { field: 'operation', value: 'x_create_tweet' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'mediaIds',
|
||||
@@ -122,6 +124,7 @@ export const XBlock: BlockConfig = {
|
||||
type: 'short-input',
|
||||
placeholder: 'Comma-separated media IDs (up to 4)',
|
||||
condition: { field: 'operation', value: 'x_create_tweet' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'replySettings',
|
||||
@@ -136,6 +139,7 @@ export const XBlock: BlockConfig = {
|
||||
],
|
||||
value: () => '',
|
||||
condition: { field: 'operation', value: 'x_create_tweet' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
// --- Tweet ID field (shared by multiple operations) ---
|
||||
{
|
||||
@@ -212,6 +216,7 @@ export const XBlock: BlockConfig = {
|
||||
],
|
||||
value: () => 'recency',
|
||||
condition: { field: 'operation', value: 'x_search_tweets' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
// --- User ID field (shared by many operations) ---
|
||||
{
|
||||
@@ -341,6 +346,7 @@ export const XBlock: BlockConfig = {
|
||||
field: 'operation',
|
||||
value: ['x_get_user_tweets', 'x_get_user_timeline'],
|
||||
},
|
||||
mode: 'advanced',
|
||||
},
|
||||
// --- Time range fields (shared by tweet search and user tweet operations) ---
|
||||
{
|
||||
@@ -357,6 +363,7 @@ export const XBlock: BlockConfig = {
|
||||
'x_get_user_timeline',
|
||||
],
|
||||
},
|
||||
mode: 'advanced',
|
||||
wandConfig: {
|
||||
enabled: true,
|
||||
prompt: `Generate an ISO 8601 timestamp based on the user's description.
|
||||
@@ -386,6 +393,7 @@ Return ONLY the timestamp string - no explanations, no quotes, no extra text.`,
|
||||
'x_get_user_timeline',
|
||||
],
|
||||
},
|
||||
mode: 'advanced',
|
||||
wandConfig: {
|
||||
enabled: true,
|
||||
prompt: `Generate an ISO 8601 timestamp based on the user's description.
|
||||
@@ -425,6 +433,7 @@ Return ONLY the timestamp string - no explanations, no quotes, no extra text.`,
|
||||
'x_get_blocking',
|
||||
],
|
||||
},
|
||||
mode: 'advanced',
|
||||
},
|
||||
// --- Pagination Token (shared by many operations) ---
|
||||
{
|
||||
@@ -448,6 +457,7 @@ Return ONLY the timestamp string - no explanations, no quotes, no extra text.`,
|
||||
'x_get_blocking',
|
||||
],
|
||||
},
|
||||
mode: 'advanced',
|
||||
},
|
||||
// --- Next Token (for search operations that use nextToken instead of paginationToken) ---
|
||||
{
|
||||
@@ -459,6 +469,7 @@ Return ONLY the timestamp string - no explanations, no quotes, no extra text.`,
|
||||
field: 'operation',
|
||||
value: ['x_search_tweets', 'x_search_users'],
|
||||
},
|
||||
mode: 'advanced',
|
||||
},
|
||||
// --- Trends fields ---
|
||||
{
|
||||
@@ -475,6 +486,7 @@ Return ONLY the timestamp string - no explanations, no quotes, no extra text.`,
|
||||
type: 'short-input',
|
||||
placeholder: '20',
|
||||
condition: { field: 'operation', value: 'x_get_trends_by_woeid' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
// --- Usage fields ---
|
||||
{
|
||||
@@ -483,6 +495,7 @@ Return ONLY the timestamp string - no explanations, no quotes, no extra text.`,
|
||||
type: 'short-input',
|
||||
placeholder: '7 (1-90)',
|
||||
condition: { field: 'operation', value: 'x_get_usage' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
],
|
||||
tools: {
|
||||
|
||||
@@ -1841,7 +1841,23 @@ export const auth = betterAuth({
|
||||
tokenUrl: 'https://api.x.com/2/oauth2/token',
|
||||
userInfoUrl: 'https://api.x.com/2/users/me',
|
||||
accessType: 'offline',
|
||||
scopes: ['tweet.read', 'tweet.write', 'users.read', 'offline.access'],
|
||||
scopes: [
|
||||
'tweet.read',
|
||||
'tweet.write',
|
||||
'tweet.moderate.write',
|
||||
'users.read',
|
||||
'follows.read',
|
||||
'follows.write',
|
||||
'bookmark.read',
|
||||
'bookmark.write',
|
||||
'like.read',
|
||||
'like.write',
|
||||
'block.read',
|
||||
'block.write',
|
||||
'mute.read',
|
||||
'mute.write',
|
||||
'offline.access',
|
||||
],
|
||||
pkce: true,
|
||||
responseType: 'code',
|
||||
prompt: 'consent',
|
||||
|
||||
Reference in New Issue
Block a user